Tuesday, January 23, 2007
...type I and type II errors arise because the investigator has attempted to dichotomize the results of a study into the categories "significant" and "not significant." Since this degradation of the study is unnecessary, an "error" that results from an incorrect classification of the study result is also unnecessary.As one who falls squarely on the quantitative side of things - scientifically, what you think is only as interesting to me as the numbers you can show - the reliance on P irks me for just this reason. If you see 15 separate studies of, say, Drug X on T-cell counts, showing a (non-significant) p of .1, to me that's nearly as interesting as two with p=.003.
For gushier stuff than T-cell counts (like cognitive function scores or pain indexes), where measures are necessarily subjective, I would argue that such dichotomization is totally inappropriate. Sadly, that's all that gets published. Unless you're already a super-established expert with bottomless funds who's probably sleeping with the editor.