In reply to sticks:
Would back Pete's general point.
I've always got away with tightening the straps down so the pack sits very high, and then having the waistbelt loose so that it just stops the pack from coming over your head if you jump/fall/bend over.
I'm 6-1 and always liked the Inov8 packs - they seemed to scrunch down really small and sit up high on my shoulders without annoying me.
I also had a little Deuter pack for very light outings as you describe, also worn high with a loose belt.
I've never tried a proper vest pack, but they do look like the way forward.