“It's not an experiment if you know it's going to work.” – Jeff Bezos
Imagine you are a high school teacher with a class of a few dozen teenagers. The district is putting together a track team and the principal of your school has given you the task of determining which of your students is the fastest runner. On its face, the problem seems simple enough. Line the kids up, let them run a certain distance, observe who finishes first, and report back to your boss accordingly.
Not so fast. Unless one of the students is an absolute standout who renders the decision obvious, your assignment can quickly become complicated. What distance should the students run? Being a good sprinter is different from excelling at long distances, after all. What if two runners are extremely close in skill but the winner is more affluent and has superior running shoes? What if the race is repeated several times and one kid wins a few of the races but finishes in the back of the pack in others, while another finishes second in every race? What if the parents of the sporadic finisher are major donors to the school? Getting selected for the track team sure would look good on that Ivy League college application, wouldn’t it?
Who is the “fastest runner” now?
This thought exercise lays bare several material challenges that plague the experimental sciences. These include isolating control variables from those being intentionally probed, applying proper statistical analyses to data sets, and checking biases that may contaminate both the experimental design and the interpretation of the results, the former often facilitating a prejudiced assessment of the latter. Non-scientists would be shocked at how susceptible the entire profession is to basic human failings, social pressures, and economic incentives. This becomes especially true for fields of study in which variables are nearly impossible to control, giving researchers substantial leeway in how results are presented and what conclusions are drawn from them.
To wit, we observe the media hyperventilating over the latest salvo from radical environmentalists in their ongoing war against natural gas. An international team of scientists led by researchers at the Rocky Mountain Institute (RMI) – an organization founded by Amory Lovins, best known for its longstanding opposition to nuclear energy – recently published a short paper in an obscure journal with some explosive claims. Here’s how the Washington Post described it (emphasis added throughout):
“Gas-burning stoves in kitchens across America are responsible for roughly 12.7 percent of childhood asthma cases nationwide — on par with the childhood asthma risks associated with exposure to secondhand smoke, according to a study. The peer-reviewed study, published in the International Journal of Environmental Research and Public Health, adds fuel to a burgeoning debate over the potential threats that gas stoves pose to the planet and public health.
It comes as scientists and activists cheer the Consumer Product Safety Commission’s recent decision to weigh new regulations on indoor air pollution from gas stoves, even as the natural gas industry fights to keep the signature blue flames of the appliances in American homes.”
This “research” is little more than flimsy propaganda. The entire episode – from how the research was conducted, where it was published, and how the media’s fear machine was kicked into overdrive – is a quintessential example of all that is wrong with the modern academia-media-government complex, a behemoth where actual science is rarely practiced but the word itself is cynically abused in the name of raw politics.
Get your steak knives out, it’s time to dig in.