In reply to goldmember:
Setting wooden posts in to concrete isn't the best idea - water runs down the wood but can't escape so rots the wood pretty quickly.
Replace the post. Dig out the concrete "plug" as cleanly as you can; put in new post and support by packing stones (or broken bits of the old concrete plug) around it. If the last pieces of stone you put in are slate like then you can use them to get the post plumb vertical. Tamp the stones in with a sledgehammer.
While this may seem crude it is remarkably effective - we used to place stone gate stoops this way and we'd be hanging 12ft wooden gates off them.