Tom Siegfried at Science News has a rather lengthy and useful post about the nature of and some knotty problems with the concept of statistical significance. Not too much new here — be sure you understand what a P-value really is, don’t conflate statistical and economic significance — as Siegfried points out:
Experts in the math of probability and statistics are well aware of these problems and have for decades expressed concern about them in major journals. Over the years, hundreds of published papers have warned that science’s love affair with statistics has spawned countless illegitimate findings. In fact, if you believe what you read in the scientific literature, you shouldn’t believe what you read in the scientific literature.
The statistics blogosphere, to the extent that there is one, is all a flutter. Andrew Gelman provides a round up on his blog.
My statistician pal generally endorses the Science News article, but I didn’t see his email to me because it got dumped in my junk mail folder. He attributes this to spam filters using Bayesian methods.
No kidding.