In reply to cha1n:
The only tap I can think of is at the bivouac at Bourron Marlotte. It's in the car park by the road leading up to the Mare aux Fées. It's a long way from the main climbing areas though. Isn't the tap working at the old wooden market building in the middle of Milly la Foret? It's on the North East corner of the building, at least it was.
The taps and bivvy site at the Cuvier have been closed down for years. It's strange there aren't more taps, I know I've seen one near a Maison Forestière near Fontainbleau town but I'm not sure where... possibly at one marked on the IGN map as "Mais Forest des Huit Routes, just N W of Fontainebleau going toward Mont Ussy. Sorry I can't be more helpful.