Ezra Opens a Second Front in the War on Science

Ezra’s latest post complains that the media doesn’t pay enough attention to partisan thinktanks. The idea is that it doesn’t matter that a study is hopelessly biased, as long as it has 500 pages of pretty graphs. Now, I know it’s attractive to project the diametric opposite of your own view onto the mainstream media. However, that’s not a very good methodology for discussing politics.

Social sciences have an outrageously low signal to noise ratio. How couldn’t they? They a) attract people who’re on average less smart than those who do natural sciences, b) give people a lot more incentive to stick to indefensible conclusions, and c) has a proliferation of methodological minutiae that can get you any result you want. For instance, multiple regression analysis depends on the variables you enter so much that the same data can be shown to lead to wildly different conclusions.

With so much chaff and so little wheat, people have to develop mechanisms of throwing out the junk. One good mechanism is the rule of thumb that if a thinktank disagrees with an academic study, the thinktank is always wrong; to take a slight refinement, academics whose chairs are funded by ideologically motivated thinktanks, like John Lott, are barely more reliable than thinktanks.

Right now, the media gives equal credence to peer-reviewed studies and to thinktank hatchet jobs. Even if it does what you imply it should, it won’t improve matters much. Laypersons won’t be any more informed, and experts won’t learn anything new.

A single news article can only offer a cursory analysis of scientific debates, and is hardly more informative than, say, direct-to-consumer drug advertising. It’s impossible to write a 1,000-word New York Times article doing justice to, say, debates within biological community about evolutionary developmental biology. And that’s an issue where it’s relatively easy to produce rigorous research; when you venture into economics, things get exponentially harder.

It’s possible for a newspaper to publish a debate between Lewontin and Dawkins, or even to commission both to write regular features about evolution. But what’s the point? Professional biologists, who know all the relevant facts and have been familiar with those debates for decades have already decided. There’s no point in a rematch in a far less professionally competent arena.

The same principle applies to social sciences. When a thinktank publishes a research, the ideal media’s reaction should be to ignore it. There’s nothing that privileges thinktank fellows over real economists enough to exempt them from peer review. Likewise, there’s nothing that privileges ordinary people’s views. Serious scholars may not be able to describe poverty or unemployment in lurid detail, but frankly the media could use fewer heartwrenching stories and more facts.

The layperson doesn’t need to know anything more than what mainstream expert opinion is, and to what extent expert opinion can be trusted (more so on climatology and evolutionary biology than on welfare economics and political science). If he cares enough to delve into the subject, that’s what books and professional reviews are for. The mainstream media can never be a substitute for scholarly books and articles; even semi-popular magazines like Scientific American are inferior to actually reading what current research is.

Check out my post on religion and welfare. The post has 1,500 words; counting graphs and tables as a number of words taking equal space, the study it critiques has 15,000. And the post doesn’t do anything that I’d expect of similar coverage in the mainstream media: literature review, quoting other experts, actually checking the data, fleshing out alternative theories. In its most cursory form, therefore, a satisfactory critique needs to be about a tenth its object study’s size. When the study in question has the 500 pages the hack Ezra quotes brags about, it takes 50 to take it apart.