In reply to Cheese Monkey: Port Eynon is nice, close to Boiler Slab, fantastic views across the bay and walk along beach to a pub 10 mins at most.
Then there is Llangennith, beach hidden by the dunes, same sort of walk to pub in the village, good for Falls Bay. This can feel like you're in a festival due to the size of it though.
The camp site above Three Cliffs Bay gets lots of mentions as one of the best in UK for views, but the pub is either down the road (narrow and dangerous to walk imho) or down a path that can get really wet/muddy/slippery. I think it's a bit further than the others too, maybe 20 mins?
There are 2 others, one is a holiday park and the other escapes me,nice, quiet place, no pub though.
Mark