That's right. Not sure about the paying bit - I've only ever used it for letters and as I remember it was the sender that paid everything - I just turned up with ID and picked them up.
If it's amazon you're buying from, they have locations all over France you can pick your stuff up from. You just search for the nearest collection point at the checkout stage.