Critical Thinking About Research - book review
From: Harriet Hall
Date: Wed, 20 Jul 2005 10:58:42 -0700
I want to recommend a book: "Critical Thinking about Research" by Julian Meltzoff published by the American Psychological Association.
Just because a study is published in a peer-reviewed journal doesn't guarantee you can believe its results. This book teaches you how to evaluate scientific evidence, how to read a research article and critique it. It is a beautiful survey of all the things that can go wrong in an experiment, from the logic of design to the mechanics of randomization to the statistical analysis to the interpretation of results to the write-up. Just one very minor example: if all the group 1 data are handled early in the day and all the group 2 data are handled late in the evening, the handling of group 2 data might vary in quality due to researcher fatigue or other subjective factors. It would be easy enough to plan ahead to rule out this possibly confounding factor, but you would have to think of it first.
It includes 16 fake articles with deliberate flaws. You can test yourself and then read the author's critique. It is a wonderful teaching exercise, made even more enjoyable by the puns he includes (Jane Kent-Reid and John E. Duzzent as authors of a study on dyslexia).
There are a lot of studies in print that should never have been published and a lot that should never have been done in the first place. The principles of good research are poorly understood even by many researchers. Psychological pitfalls have betrayed otherwise good scientists (remember N-rays?). I wish this book were required reading for everyone who does science or reads about it - particularly those in alternative medicine who quote flawed studies to support their beliefs. And those who do meta-analyses and lump the good studies with the bad.
Harriet Hall, MD
*********
Used by permission. Links added by PL.
Date: Wed, 20 Jul 2005 10:58:42 -0700
I want to recommend a book: "Critical Thinking about Research" by Julian Meltzoff published by the American Psychological Association.
Just because a study is published in a peer-reviewed journal doesn't guarantee you can believe its results. This book teaches you how to evaluate scientific evidence, how to read a research article and critique it. It is a beautiful survey of all the things that can go wrong in an experiment, from the logic of design to the mechanics of randomization to the statistical analysis to the interpretation of results to the write-up. Just one very minor example: if all the group 1 data are handled early in the day and all the group 2 data are handled late in the evening, the handling of group 2 data might vary in quality due to researcher fatigue or other subjective factors. It would be easy enough to plan ahead to rule out this possibly confounding factor, but you would have to think of it first.
It includes 16 fake articles with deliberate flaws. You can test yourself and then read the author's critique. It is a wonderful teaching exercise, made even more enjoyable by the puns he includes (Jane Kent-Reid and John E. Duzzent as authors of a study on dyslexia).
There are a lot of studies in print that should never have been published and a lot that should never have been done in the first place. The principles of good research are poorly understood even by many researchers. Psychological pitfalls have betrayed otherwise good scientists (remember N-rays?). I wish this book were required reading for everyone who does science or reads about it - particularly those in alternative medicine who quote flawed studies to support their beliefs. And those who do meta-analyses and lump the good studies with the bad.
Harriet Hall, MD
*********
Used by permission. Links added by PL.
posted by Sitemaster | 9:16 PM
<< Home