When determining how generalizable a research study is to the outside world, it’s important to focus on the “Methods” section of a study. Though it can appear to be pretty bland, it actually paints a bigger picture of how the results can be interpreted. For instance, take a look at this CNN Health report on a study connecting ibuprofen to male infertility:

​

​

 

 

 

 

 

 

 

 

 

Studies are often viewed within a (social) media context, so being able to analyze a study’s methods in either context is key to understanding the research.

​

After reading the Methods section in the above image, do you notice anything a little peculiar about the study sample?

• The story’s claim is based on only 31 men, all from Denmark, all white, and no one older than 35 years old.

​

How generalizable are these results, then? Could there be different results for men older than 35? What about men living in different countries/cultures?​

​

While these may seem like questions that are just being asked for the sake of being asked, studies like this one show that people of different ancestral descent who carry a certain gene type for schizophrenia are, depending on if they are mainly of European or East Asian descent, either at risk for or protected from schizophrenia when that gene type is present. If such  differences are happening at the gene level, it’s safe to question if there are differences based on ancestry, age, upbringing, etc., that are happening when a drug, ibuprofen, enters the body.

           

Next, take a look at this Methods section from a study done to examine the effects of poverty on children’s brain development.

​

​

 

 

 

 

 

 

 

 

 

 

 

 

The study did a good job of including children from different racial backgrounds, but they selected their sample from an ongoing longitudinal study being done in St. Louis, Missouri. So, they are only looking at children from one geographical location, which means we can’t know how the results may have looked similar or different if they had pulled from a rural town in, for example, Michigan’s Upper Peninsula.

 

Even more important to note, is that the original longitudinal study oversampled for preschoolers with depressive symptoms, meaning the sample for this study included preschoolers with a clinical diagnosis of depression. Couldn’t this presence of depression have affected children’s brain development? Or, at least, the brain imaging scans taken from those children with depression?

 

As a reader of research, careful reading of the Methods section of a journal article or media summary of research can help you better determine whether you feel the research findings are actually generalizable to those outside of a certain population, or even generalizable outside of the specific people in the study.