In reply to llamaglama: There are plenty of good exercises out there to strengthen up the muscles surrounding your knee but from the sounds of your knee I'd be wary about doing anything too strenuous in case you damage it further. The classics (Squats, lunges, split sqats etc.) put a fair bit of strain on the knee joint.
Search through youtube or look on www.strengthrunning.com for any exercises that don't put any brutal pressure on your knee. Things like bridge, clam, heel dips/raises, leg raise etc.
Sounds like you have quite a specific problem, a trip to the physio might prevent either further injury or hours wasted doing exercises which don't solve it.