In reply to Midgetgem:
We explored Leonidio recently after many trips to Kalymnos. It's unhelpful to compare the two locations directly, but it is fair to say that Leonidio has lots of good climbing, plenty of places to stay and some great beaches.
The big plus for Leonidio is that you can fly easily to Athens and hire a car cheaply. The drive then is three and a half hours. You need a car at Leonidio but for short drives only. I do love the scooter experience though and the boat trips to Telendos while in Kalymnos.
Something else to consider is going to the other areas in Greece if you are going for a couple of weeks. I've just bought the 2017 topo to Greece and am looking forward to exploring. Napflio was really good if you want to break up the driving, for example. Finding accommodation is really easy.
Crags aren't busy in Leonidio but perhaps become more popular in October. Nowhere as busy as Kalymnos though.
My personal summary would be that for a purely climbing holiday, Kalymnos would be difficult to beat, but if you see yourself mixing cragging with time on the beach etc, try Leonidio. Both are great so don't fret about picking the wrong option.