In reply to Offwidth:
...which wasn't me, btw.
The topic is fascinating. I've barely scratched the surface, but through reading about it I have come to understand ( I think) where the Bayesian approach is coming from, and how Ioannidis (he of the paper "Why most published research findings are false" that is mentioned in the Arnold article) arrives at his conclusion.
One thing i like about it all is that, loosely that approach can be couched as "odds that my theory is right, having done my experiment" (posterior odds) = "impact that my experiment has made on those odds" x "odds that it was right before the experiment" (prior odds). Ioannidis claims, and tries to show, if I understand him correctly, that the posterior odds of most of the published research findings that he included in his study were less than one, which means that the findings were more likely wrong than right, due to a combination of poor prior odds - this is the subjective bit - but is what you get in fields without a well established canon of results, and poor experimental design (often due to sample sizes being too small, given the effect size) such that the experiments didn't do enough to transform those prior odds into posterior odds that you would want to bet on.
Using this idea, we are led to the notion that extraordinary claims (really shaky prior odds) require extraordinary evidence (an astonishingly sound experiment) if the claim is to have legs. Do astrologists know this?
Regina Nuzzo has written accessibly on this topic in many news articles in Nature.
