A fortified building at the site was attested as early as 1292. In 1300, however, Godefroid of Hellebeke, the first known Seigneur of Beersel, received permission from Jean II, Duke of Brabant, to build a fortification on the present site near his residence.
The castle’s present configuration dates to 1357. It was twice sacked and was subject to significant restorations in 1491 and 1617. Its present condition owes much to a major restoration in 1928–39. Built largely of brick, a rare material for such buildings at the time, around a circular enceinte, its major feature is its three large towers.