Sunday, August 04, 2013

Perspective III

[Edit: Whoa, it's been a while, and I should have a title.  Status of the hiatus?  Who knows, but I'd be pretty happy to get Paul Ryan's mug off the front page one of these days.]

In 2002, Jan Hendrik Schön, a researcher at what was left of Bell Labs, was accused of scientific misconduct by a formal investigative committee. He was caught falsifying data, sometimes drawing curves and pretending they were measured, other times swapping them from other pieces of data, and it was a huge scandal at the time, judged among the worst things you can do in science, or at least in premiere science. He was spotted because other researchers noticed that some of his plots looked exactly the same, even when he was reporting different conditions or phenomena. That is, they did not follow the same predictable shape, as you might hope data will do, but they bounced around their trendline in exactly the same way in both plots. This is what I remember of the scandal back then: one of my coworkers smugly pointing this out to me. See those two graphs, Keith? There's no way they should look exactly the same. (That's not just a fraud, but a lazy one. For the dedicated forger, how hard would it have been to introduce false instrument noise?)

When the Lucent committee looked more closely at his work, they found the bullshitting to be pervasive. No wonder he was able to publish a paper every week! Since this stuff was getting cranked through Science and Nature--relatively groundbreaking results in what was a sexy field at the time--Schön was ridden out of the scientific community on a rail. His awards and even his doctorate were rescinded (but the latter was re-awarded; evidently the guy's been fighting for his degree as recently as three years ago). The journal editorials tend along reputation-preserving retrospectives to the tune of how the hell did this happen, and let's never allow physics to be besmirched this way again.

I'll add that Schön is roughly my age, and even though I was a total academic nobody, there's still only a degree or two of separation between us. We worked in a similar field I am sure I read some of his papers late in my grad school days (which isn't to say I remember them specifically). My advisor had done his post-doc at Bell Labs, and some of his remaining friends there, to whom I'd been introduced, would become Schön's occasional co-authors. In defense of that community, the fact that Schön's data were hard to replicate isn't a total indictment by itself. Studies of organic electronics were notoriously difficult to reproduce in those days (and probably are now too), even for the same people, doing the same experiments.  This was much more a matter of trying to force soft, quasi-pure, and reluctantly ordered systems to behave like near-perfect semiconductor crystals than it was a matter of dishonesty. Electrical behavior that was extremely sensitive to barely-tangible properties of an interface was hardly a rare thing. And in defense of Schön's co-authors, it's actually really easy to get listed as a middle contributor in a bustling, incestuous place like Bell Labs. That place (in that field at least, and in the 1990s) was more liberal than anywhere when it came to spreading names across the mastheads. You might get listed as an author for having prepped one or two samples, for overseeing one of the instruments, or contributing a paragraph of text from one of your publications.  Take that nice piece of resume fluff, and move on, it's not terribly necessary to know what the lead was ever really up to. And more than that, Bell Labs was in its sad death spiral in the late 90s. It's got to be hard to care about what the guy down the hall is doing when you're investing all your free minutes in an effort to get yourself out from under the headsman's axe. Schön was dishonest as fuck-all, but he was in a niche where he could ramp it up a little higher than usual before getting caught.

So.  It's months old now, but the economics world was rocked--no, better to say that it should have been rocked--by two researchers who published a report explaining that national debt levels above 90% of GDP cross a drastic threshold that's correlated with low or negative growth.  Now, this was published in a conference proceedings (they give you big grains of salt at the door to these things, along with the pens and other swag--you publish in proceedings when you don't want to go through all the work of satisfying reviewers), so it's not, I sure hope, premiere economics, except that it became a leading-edge study because it got swiped up as a wand of legitimacy by pro-austerity policymakers and pundits, and we have all had to keep hearing about this crap.  But when Reinhart and Rogoff's analyses were put under any kind of scrutiny at all, it was, well, not good: bizarre weighting procedures and cherry-picked data ranges, along with spreadsheet manipulation (is it really cool with Harvard that its researchers chuck numbers into Excel and call it research?), later claimed as a mistake, that was painfully obviously designed to produce the foregone conclusion.  And even if they've claimed only correlation, causation has been implicit when the work is reported by their supporters. 

(I'll add too, that I don't think national debt is awesome myself.  I see it as a trick that clears itself up so long as there's GDP growth, but there's no good reason to monkey with the accounting like that, well, other than to be obfuscatory about imbalances.  The solution isn't to starve the lower orders; it is support them while showing the books properly.  That debt doesn't correlate with growth, or that it correlates much more weakly than they claimed, is supportive of the "black box" theory.)

Anyway, my argument isn't so much what happened.  You get academic tendentiousness for all sorts of reasons, and even thinking people are still hierarchical herd animals in lots of ways.  Their rise went farther than it should have too.  I've said before that since it's fundamentally based in the logic of agreements, economics is more like law than it is like science, which is to say that it's a matter of both rationality and, to a much higher degree than physics is, advocacy.  Reinhart and Rogoff were probably not as dishonest as Jan Schön was--it was more likely they were just caught being lazy--but on the other hand, it doesn't seem to have hurt their careers, but for a few necessary defensive letters.  Their work appears to remain influential. 

I mean, fucking seriously.  Here's Larry Goddamn Summers defending it, doing the opposite of protecting his own intellectual reputation and that of the field.  They are friends, Summers says, and he openly admits that their work supports his agenda.  Why does this guy, in the face of bad data, and his own awful history of being wrong policy-wise--not to mention occupying the smug embodiment of shitty low-empathy ethics of austerity--retain so much pull?  Why was he a chief economic advisor, and why is that slimy piece of crap on the short list to run the Fed?  I mean we know why, but still. 

Economics is far from useless, but it's also farther from hard science than it pretends.  In more ways than one.

                

8 comments:

Archaeopteryx said...

Honestly, I'm just commenting because I can't find a thumbs-up button. You're right that economics is not useless, but in many cases, it's worse than useless.

Keifus said...

Oh man, thanks Arch.

Inkberrow said...

It's not causation so much as cause---to wit, is it just?

Schon, Reinhart, and Rogoff have their methodological colleagues in the global warming and anthropogenic climate change community too, and there an analogous set of pro-austerity types need by definition only their gatekeeping political or socio-ethical premises. When power, acclaim, and the Good of the World are at stake, it's superfluous to really try and falsify the dull hypothesis.

Keifus said...

Hi Inkberrow, I suppose I could have added that everyone who works at this sort of thing has a little bit of a Procrustean impulse at work, even for noble reasons. We want the data to support the hypothesis. But fabricating it outright is a big no-no. And fudging it (and making convenient errors) to drastically alter your conclusions is also a step beyond mere bias.

I don't believe that climate studies fall into the latter category. (Certainly they don't fall into the former.) I am familiar with some of the ways Mann et al. have had their statistical methodologies attacked (and I think it's fair game as far as it's an honest attack), and I think their rebuttal in support of the model (i.e., that adding more principal components, which their critics say they shoulda, doesn't change the outcome much) was convincing. But that's high-end statistics, and I'm not an expert on it, and the necessity to deal with proxy data is never fun. In any case, that's a much different argument than R&R's rather embarrassingly obvious ass-pulling.

The power of the climate change lobby? Really?

Inkberrow said...

Agreed, the power of the climate change lobby is not as readily apparent as one might expect, given the Grave, Immediate Existential Crisis that Everyone Who Matters Accepts Axiomatically. (Not knowing much more than a jot about hard sciences, that's exactly why I'm an AGW agnostic, because if it was really proven, bright well-advised leaders like President Obama and Hu Jintao would be doing something programmatic about it right now by executive order or otherwise). But it's the potential power it represents, a scientist's dream: green Dr. Strangeloves presiding over the necessary reordering of civilization.

hipparchia said...

You might get listed as an author for having prepped one or two samples, for overseeing one of the instruments, or contributing a paragraph of text from one of your publications. Take that nice piece of resume fluff, and move on, it's not terribly necessary to know what the lead was ever really up to.

this happens a lot in science, or used to anyway. I've been away for a while, so maybe not so much now, and it does seem to me I've read that some are moving away from this model.

as for rogaine and braveheart, they seem to have pulled a different kind of fraud. apparently they were more circumspect and cautious when discussing their results in academic circles but placing very different interpretations on their work when talking to policy people. and since the academics seldom mingle with the likes of congresscritters and policy wonks, nobody was catching them at it.

Keifus said...

Yes, the judgment of "lazy" was indeed generous. (If you prefer, outright corruption, but with the American veneer of plausible deniability!)

Always nice to see the Cynic show her face!

hipparchia said...

cynic, me?

;)