The Science behind the Headlines
January 8, 2010
The following is based on an article of the same name published in the International Journal of Obesity on December 1, 2009 by MB Cope and DB Allison of the University of Alabama. Although it was not widely reported in the Asian press it deserves to be in our Science behind the Headlines series because it deals with the important issue of what scientific information can be trusted.
AFIC has from its inception been dedicated to the idea that it is important for the media, the public and policy makers to have access to reliable information about nutrition, food safety and health that is based on sound science. What has become apparent in recent years and which is demonstrated convincingly by this article is that it is becoming much harder to distinguish what is sound science and what is not.
The authors conducted an analysis on scientific publications concerning the role of sugary beverages in causing obesity and publications concerning the role of breastfeeding in preventing obesity. In doing their analysis they coined the term “White Hat Bias” or WHB to describe what they thought might be a tendency among scientists to ignore data that did not support their views on sugary beverages and breastfeeding while overemphasizing data that did support their views. This type of bias is a result of researchers believing that they know what needs to be done to improve public health and are determined that their research will provide the proof. This belief is not necessarily a bad thing unless the researchers deviate from sound scientific practices and ignore data that do not support their conclusions or report their data and others’ in a misleading way.
[NOTE: Those of you who did not grow up in the U.S. during the 1950’s may not understand the significance of the term “white hats”. During the 50’s there were many popular American Western Frontier TV shows and movies where the villains all wore black hats and the heroes all wore white hats. Since then the term “white hat” has been used to describe someone who has good intentions or is heroic in nature. The term is sometimes used in the context of an overly simple world view where people are either good or evil.]
The authors found 4 different examples of how WHB can influence perceptions and outcomes in research and public understanding (or misunderstanding.
1. CITATION BIAS
The first example was an analysis of whether original research reports on these topics were accurately characterized in subsequent research and publications. The authors looked at two papers studying the impact of reducing sugary beverage intake on obesity in children. In one paper, the authors did not find any significant difference in Body Mass Index (a measure used to determine obesity) between children who had reduced intake of sugary beverages and those who did not. The other paper had the same conclusion for the overall group but did note a slight decrease in obesity for a small subgroup with reduced intake of sugary beverages. In looking at how these results were cited in subsequent publications, Cope and Allison discovered that the majority misleadingly characterized the strength of the evidenced in the original research on the impact of reducing intake of sugary beverages.
2. PUBLICATION BIAS
The second example studied publication bias, or the possibility that a study’s chance of publication might depend on its outcome. In the case of sugary beverages, the authors found that most of the published studies showing a positive link between sugary beverages and obesity tended not to be very precise and to come from non-industry sources. By contrast, the studies showing no impact of sugary beverages on obesity tended to have industry funding and be much more precise. The authors’ analysis led them to conclude that non-industry researchers tended not to publish findings that did not show a positive link, although it is difficult to quantify how great the magnitude of this bias might be. The authors found similar results when they analyzed for publication bias on the link between breastfeeding and prevention of obesity.
3. MISCOMMUNICATION IN PRESS RELEASES
The third example was anecdotal rather than systematic but the results were consistent with previous systematic analyses. In this example, the authors cited examples where press releases on papers on obesity research misleadingly characterized the conclusions as more positive than could be supported by the papers themselves. This is possibly a function of the need for academic institutions to successfully compete for talent and research funds.
4. INAPPROPRIATE OR QUESTIONABLE INCLUSION OF INFORMATION
Finally, the authors looked at how research may be misleadingly presented by including incorrect or questionable material in reviews. In review papers, authors should try to objectively and comprehensively summarize all relevant scientific publications in a given area. Cope and Allison found that in a World Health Organization review on breastfeeding to prevent obesity, the authors tended to exclude data that did not support their desired conclusion that there is a connection. They also found this type of bias in reviews related to sugary beverages and in one instance found that a review inappropriately included irrelevant information related to sweetened solid foods.
This paper highlights the quandary that exists for those of us who want to see science presented based on facts rather than opinions. At a minimum it means that even more effort must be devoted to critical examination of new reports that one thing or another is either bad or good for us. Certainly AFIC will redouble our efforts to provide a more objective perspective but in the meantime a little common sense may be in order. It is always wise to have some skepticism about claims that seem either too good or too bad to be true. Also, as AFIC constantly recommends, it is hard to go wrong with a balanced diet. For more information on what a balanced diet might be, see http://afic.org/dietaryguide.php?news_id=248&start=0&category_id=26&parent_id=26&arcyear=&arcmonth=
