Panagia monastery was built in 1225. The building that visitors see nowadays is of 16th - 18th centuries.
The first buildings must have been parts of an old fortification. This is probably why it was given the name "Palaiokastrita” (Old Castle). The monastery was destroyed by Genoans in 1403. It was rebuilt in 1469 and destroyed for the second time in 1537 by the Turks. It was reconstructed in 1572 and is preserved up to now with many repairs and additions, such as its north entrance (1722).