In reply to Julian W:
Broke both arms near the wrist in 2011, one a nasty one, both bones in several places. Since the break was quite bad and I was early 20's (post bone growth) I had both of mine plated which helped healing at first and meant no cast. Unfortunately I had to have one plate removed 18 months later due to persistent low level pain.
Bouldering indoors was my rehab, I don't remember the timescales but it was at least 6 weeks after accident and took it steady, built up over time. One thing I remember my physio saying was that soreness after using it was not a bad thing as long as it was gone by the next day. If pain persisted to the next day then I had overdone it. Due to the plate messing me up it took around a year to exceed pre-accident strength but in your case it sounds like it will be much shorter. Aside from a slightly reduced wrist flexibility I am now much stronger than I was before.
I'd say ignore doc and build up carefully and slowly at his own pace by feeling.