The Atlantic ran an awesome takedown on a recently published study on breastfeeding. While I am interested in the subject to a degree what impressed me more was how they reported on this study. Which is to say they ignored the obvious click-bait headline and dug into the research itself. One thing my research methods prof drilled into my head was that there are very few breakthrough studies. Instead science relies on a consensus, so the more labs and studies that come up with a similar result the more that the findings may be considered reliable or fact.
So while the article isn't perfect it's closer to how i'd love to see science handled in the press. Here are a few of my favorite paragraphs. Be sure to go read the rest!
Even with small rates of loss, significant biases are possible. And the number of people who drop out of a study is actually less important than how those departures are related to the findings. The contributing factors that make someone lost to follow-up are often the same factors that would affect the outcome of the study.
In the context of a study about breastfed babies, the obvious example is socioeconomic status. That's because researchers tracked key socioeconomic markers—income levels, duration of schooling—that could also influence whether a person is findable for a follow-up. So, in theory, the breastfed babies who wouldn't have ended up earning as much or performing as well on an IQ test could have been the ones lost to the study. Which is why these sorts of losses to follow-up are considered one of the main threats to validity in clinical trials, and they rarely occur randomly. "The only way to ensure that losses to follow-up have not biased study results," wrote the authors of a 1998 study about tracking research cohort, "is to keep all losses to an absolute minimum."
Relying on a research subject's memory can be a major threat to validity, and breastfeeding mothers have provided inaccurate estimates to scientists before. "When mothers are asked retrospectively in breastfeeding studies about breastfeeding duration, the reports become increasingly inaccurate with increasing time since cessation," according to a 2008 paper about research biases by Lars Fadnes. "Mothers who had breastfed their infants for longer systematically underreported their breastfeeding duration; while in contrast, those who had breastfed for a shorter time over-reported the duration."
In other words, because mothers who breastfeed are also more likely to enjoy a higher socioeconomic status, maybe the breastfed babies have higher IQs and better income in life because of advantages—other than breast milk—associated with that higher socioeconomic status. Does the extra $100 a month that breastfed babies apparently end up making, according to this study's findings, really come form the fact that they were breastfed—or from other advantages they had over time? Correlation, as they say, is not causation.