In reply to AdeleC:
I was going to suggest kayaking, sea kayaking in particular if you're anywhere near a nice bit of coastline.
I had a little dabble for the first time last year and immediately loved it (a 5-day "intromediate" course with these people btw:
http://www.seakayakinganglesey.co.uk - highly recommended).
Cheap though? Tricky. Admittedly I am verging on "all the gear no idea" levels now, but even leaving out a few daft impulse buys I've probably already spent more kitting myself out for paddling than I have over the years on climbing gear, rack, ropes etc.
Overall though, over the next few years the biggest cost of this new hobby for me is likely to be fuel - Sheffield is a ridiculous place to live for sea paddling. Makes about as much sense as getting into climbing if you live in Norwich!