In reply to Removed UserBoingBoing:
Where the people late? If they were planning to be back at, say, 14h00 and were now four hours late, I'd probably start to suspect that something had developed -- unless they were trad. climbers. If they weren't planning to be back until 18h00, I would say your course of action was perfectly correct.
If I suspected that there might be developing situation in which a rescue team might play a later role, I would try to get hold of rescue sooner rather than later and give them the known facts: where, who, weather, supposed competence levels, known plans and intentions etc. Make it clear that you have no reason to believe that anyone needs immediate assistance but let them know the facts and they might override your judgement -- they're trained in such judgements, after all.
Just be certain to ring them up, immediately, when facts change -- either because the missed people have appeared or because the situation has deteriorated.