In reply to kipper12:
The monastery is nice and weird, so worth a stop on your way for lunch at Kosmos which is the village at the top of the pass. It's only open on a couple of days a week.
Swimming at Plaka or Poulithra is quite pleasant too.
A day trip to the amazing Monemvasia is absolutely worthwhile. I really enjoyed it and you get a lovely drive there and back too.