In reply to Brendan:
Yes you can. You can get the from Telendos (the island opposite, only 700 meters away and 2 euros on a boat to get there.
OR
There is a beach near Skalia, very chilled out and has multiple kayaks and rowing boats etc for you to use, for free!