In one account of the fall of Mount Ferrant, it is said that in around 1169, William Fossard I (the grandson of the castle-builder) died, leaving the heir, William Fossard II in his minority. This meant that he would become a ward of the King. Thus William was put under the care of William le Gros, Count of Aumale. However he repaid this trust by seducing the Count’s sister. When he was discovered, William II fled the country. In his fury, William le Gros ordered Mount Ferrant to be torn down and the wood from the building to be given to the monks of Meaux Abbey, for building material.

John Leland the Tudor Librarian to Henry VIII who visited the site of castle between 1535 and 1543, relates another version of the legend. As he heard it, this version involved a Bigod of Settrington:
“who did secretely woe and wan the wylle of one of the Albemarles daughters of the Earl of Holderness”.

Roger Dodsworth the Yorkshire antiquary made an exploration of the county in the years from 1619 to 1631 mentions in his Church notes:

...A beck having its head atte Birdsale, where was
Montferrand Castle in...
The fairest lady in all this land,
Was drown’d on the topp of Mount Ferrand.

In Robert Addison’s History he writes “... Tradition says there is a deep well into which a large iron chest filled with gold and precious things was thrown by the occupier of the castle in a time of great danger which was so common in those days but in after ages an attempt to recover was made by persons descended to the bottom and succeeded in putting a strong iron chain around it an to the chain was affixed twenty to thirty strong men, when these put forth their strength they succeeded in raising the chest half way when lamentable to relate the chain broke and down it fell into the well where it remains to this very day”.

From Leavening Through the Ages by Don Howarth