In reply to funkyvin1:
You could easily spend a day in Haddon or Chatsworth with the advantage The Peacock in Rowsley is close.
Matlock Bath has a seaside resort feel (tacky but fun) and there's lots to do. Heights of Abraham is expensive and rather limited unless you are under 10. If you get the chance The Barley Mow in Bonsall is a hidden gem of a pub, just up the road. Crich Tram Museum is excellent.
Of the Castleton caves, Blue John is the most interesting, Peak cavern the most impressive and Speedwell the most unusual (the boat). With a bit of organisation and a sense of adventure you could get a key to Holme Bank chert mine (Bakewell).
If you need a walk in the rain, walk along the river from Cressbrook to Litton Mill (sheltered but muddy) then up onto the old railway where you can walk back though the tunnels, which will be dry (3 mile round). Monsal head pub is decent.