Interpreting Research Studies: Part 1
Ann Bartlett encouraged me to write about how to evaluate research papers. How are we to interpret the results? Are they valid? Are the statistical methods appropriate? Is the sample size large enough? Are the results unbiased? And lastly, who is providing the funding for the research? It also is important to ensure that research subjects are protected, ethically and personally. All studies involving humans are subjected to Investigational Review Board approval. For a researcher to study humans, they must review ethical principles and often pass a test in their own institution. Much of the information applies to protection of the research subject including informed consent and the potential benefits and side effects that may occur while the research is conducted.
A recent Washington Post newspaper had an outstanding article by Peter Whoriskey (Washington Post, Sunday, November 25): “Can Drug Research Still be Trusted?” The article focuses on Avandia or Rosiglitazone (a thiozolidinedione) and discusses how the original research was published in the New England Journal of Medicine (NEJM is one of the most prestigious medical journals in the world) noting the benefits of Avandia and disregarding the effects on the cardiovascular system (increased risk of heart attacks). According to the article, the 2006 paper in the NEJM “described a trial that compared three new diabetes drugs and concluded that Avandia, GlaxoSmithKline’s new drug, performed best.” According to Dr. Steven Nissen who was quoted in the article, there were warning signs in the data but they were overlooked. The FDA removed Avandia from the market in 2010 based on review of the data by Dr. Nissen and others, also published in the NEJM.
The title of the article “Can Drug Research Still be Trusted” alludes to the fact that a pharmaceutical company funded this particular study. Many of the paper’s authors had financial ties to the pharmaceutical company as consultants, grant recipients, and employees. (This is a common occurrence in the world of academia). However, does the fact that the research was funded by pharmaceutical company alter the interpretation of the results in this instance? In any instance? One hopes that bias does not enter into the picture. The concern in this instance is that there may have been an unconscious bias in the interpretation or “spin” of the data due to financial relationships with the company. However, it also was pointed out by Mr. Whoriskey that data suggesting increased risk for heart attacks may not have been provided to the researchers in the first place.
The big question is how can one know that results are valid in a scientific study? What are the factors that determine how one can successfully extrapolate meaning from analyzed data?
Some generalizations are in order. Many well-reputed academic scientists and physicians receive funding from pharmaceutical companies to continue their research. Funding is essential and very difficult to obtain. Competition for NIH grants and the like is becoming more and more difficult with fewer grants awarded. Therefore, pharmaceutical companies are becoming more prominent in terms of sponsoring studies, particularly drug trials, in both animals and humans. The key is for the pharmaceutical company to provide the funding, let the researchers do the data collection, statistical analysis, and provide the results with the appropriate interpretation regardless of success or not.
After many years in the medical world, reading and writing scientific papers, and in my role as an associate editor for a Diabetes journal, I feel comfortable critiquing papers for their validity and overall applicability. Scientific article assessment is very important and one must learn how and what to evaluate. Next week, I will provide specific suggestions to better understand and evaluate research studies.