Policy Point Wednesday: Critical Thought, Scientific Studies and the News

The news is full of attention-grabbing headlines:  Studies Question the Pairing of Food Deserts and Obesity, Study Finds That School Junk Food Doesn’t Lead To Childhood Obesity, and Obesity rate exaggerated to name a few.  Although news sources are supposed to be objective, are you noticing a trend?  Just a few years ago, food deserts and school junk food were in the news as causes of obesity, and current news can't seem to stop discussing the skyrocketing obesity rate.

How is a lay person supposed to wade through all of this information and assess which sources to trust?  I've found two excellent resources on how to assess whether a claim is valid and I'd like to share them with you.  

The first is a tool to assess any type of claim.  Skeptic Magazine offers a summary in their excellent video, the "Baloney Detection Kit"  (The detection kit is available for purchase from Skeptic.com) It consists of the following assessment tools:

1. How reliable is the source of the claim?
2. Does the source make similar claims?
3. Have the claims been verified by somebody else?
4. Does the claim fit with the way the world works?
5. Has anyone tried to disprove the claim?
6. Where does the preponderance of evidence point?
7. Is the source playing by the rules of science?
8. Is the claimant providing positive evidence or just denying evidence?
9. Does the new theory account for as many phenomena as the old theory?
10. Are personal beliefs driving the claim?

The second tool is from the blog Double X Science, (again, please click through and read the article in full, it is well worth it) and is specific to assessing news on science, health and medical information.  It offers a way to find and evaluate the pertinent information in an article.  Here is a broad overview of their process:

1. Skip the headline.
2. What is the basis of the article? (A study, a review, personal experience, opinion?)
3. Look at the words in the article. (Correlation does not mean causation - are they using correlative words?)
4. Look at the original source of the information. 
5. Remember that every single person involved in what you're reading is seeking some kind of return. (Advertisers, click-throughs, etc.)
6. Ask a scientist.

In my own online research, I have found the "sound bite" provided by news outlets offers direction, but little actual insight.  Source material, not surprisingly, usually offers more depth than the "sound bite," and sometimes it's easier to spot bias or missing information there.  Next time you read or hear something that doesn't sound quite right to you, try using these tools to sharpen your critical eye.