Seth’s blog

It would have been harsh to title this post “Marc Hauser, RIP”. However, unless  the following is shown to be in error, I’ll never believe anything he writes or has written:

According to the document that was provided to The Chronicle, the experiment in question was coded by Mr. Hauser and a research assistant in his laboratory. A second research assistant was asked by Mr. Hauser to analyze the results. When the second research assistant analyzed the first research assistant’s codes, he found that the monkeys didn’t seem to notice the change in pattern. In fact, they looked at the speaker more often when the pattern was the same. In other words, the experiment was a bust.

But Mr. Hauser’s coding showed something else entirely: He found that the monkeys did notice the change in pattern—and, according to his numbers, the results were statistically significant. If his coding was right, the experiment was a big success.

The second research assistant was bothered by the discrepancy. How could two researchers watching the same videotapes arrive at such different conclusions? He suggested to Mr. Hauser that a third researcher should code the results. In an e-mail message to Mr. Hauser, a copy of which was provided to The Chronicle, the research assistant who analyzed the numbers explained his concern. “I don’t feel comfortable analyzing results/publishing data with that kind of skew until we can verify that with a third coder,” he wrote.

A graduate student agreed with the research assistant and joined him in pressing Mr. Hauser to allow the results to be checked, the document given to The Chronicle indicates. But Mr. Hauser resisted, repeatedly arguing against having a third researcher code the videotapes and writing that they should simply go with the data as he had already coded it. After several back-and-forths, it became plain that the professor was annoyed.

“i am getting a bit pissed here,” Mr. Hauser wrote in an e-mail to one research assistant. “there were no inconsistencies! let me repeat what happened. i coded everything. then [a research assistant] coded all the trials highlighted in yellow. we only had one trial that didn’t agree. i then mistakenly told [another research assistant] to look at column B when he should have looked at column D. … we need to resolve this because i am not sure why we are going in circles.”

The research assistant who analyzed the data and the graduate student decided to review the tapes themselves, without Mr. Hauser’s permission, the document says. They each coded the results independently. Their findings concurred with the conclusion that the experiment had failed: The monkeys didn’t appear to react to the change in patterns.

They then reviewed Mr. Hauser’s coding and, according to the research assistant’s statement, discovered that what he had written down bore little relation to what they had actually observed on the videotapes. He would, for instance, mark that a monkey had turned its head when the monkey didn’t so much as flinch. It wasn’t simply a case of differing interpretations, they believed: His data were just completely wrong.

As word of the problem with the experiment spread, several other lab members revealed they had had similar run-ins with Mr. Hauser, the former research assistant says. This wasn’t the first time something like this had happened. There was, several researchers in the lab believed, a pattern in which Mr. Hauser reported false data and then insisted that it be used.

If taken literally, this description seems to imply that Hauser was making up data — writing down results much more favorable to his career than the actual results — and not realizing it! As if someone else was marking the data sheet. Since the videotapes are being coded by more than one person the fabrication/delusion/whatever would come to light, you might think, but he does it anyway! And then gets “a bit pissed” when things don’t work out perfectly.

I would love to hear Hauser’s side of this story, and see the videotapes being coded. So far Hauser has said nothing to make me doubt the straightforward interpretation: He made up data. After Saul Sternberg and I published a paper implying that Ranjit Chandra had made up data, Chandra retired.

Derek Bickerton says Hauser “fell victim to a soon-to-be-outdated view of evolution”. I am more interested in what this says about Harvard and Hauser’s co-authors. In particular, I wonder what Noam Chomsky, one of Hauser’s co-authors, will say. The incident makes Chomsky look bad. Hauser appears to be a person who pushes aside the truth of things. That Chomsky wrote a major paper with him suggests that Chomsky failed to notice this.

Thanks to Dave Lull and Language Log.

• A new paper debunks Michael Mann’s Hockey Stick global temperature graph. “Climate scientists have greatly underestimated the uncertainty of proxy-based reconstructions and hence have been overconfident in their models.” Very well written.
• “Obscure, contemporary ethics books . . . were actually about 50% more likely to be missing than non-ethics books.” Paper. The study was done entirely online and covered 32 large university libraries.
• Gladys Reid, Australian discoverer of benefits of feeding zinc to farm animals. “Reid was reluctant to make direct dose recommendations after claiming the Director General of Agriculture had told her she would be taken to court for misleading practices if she did. However she won followers from farming wives in particular. Many would call asking for zinc advice after tiring of seeing suffering livestock and husbands on the brink of suicide from crippling stock and production losses.”
• Using a treadmill while working
• The Potti Scandal continues
• How loud are Sunchips?

Thanks to Don Sheridan and Melissa Francis.

A paper in Cognition by Harvard professor Marc Hauser and others has been retracted:

The paper tested cotton-top tamarin monkeys’ ability to learn generalized patterns, an ability that human infants had been found to have, and that may be critical for learning language. The paper found that the monkeys were able to learn patterns, suggesting that this was not the critical cognitive building block that explains humans’ ability to learn language.

The note to be published about the retraction says almost nothing about why: “An internal examination at Harvard University . . . found that the data do not support the reported findings.”

Several other papers from Hauser’s lab have also been questioned.

The usual explanation would be that someone in Hauser’s lab made the results better than they actually were. A co-author of the paper said Hauser had told him “there were problems with the videotape record of the study”.  That’s consistent with the usual explanation: Someone edited the tapes (via deletions) to make the results appear better than they were. But it’s also possible that many tapes are missing, which might be an accident. When The New Yorker archives were moved from Building A to Building B several years ago, much of the archives was lost.

Thanks to Aaron Blaisdell.

I came across the Potti scandal while trying to find out about the trouble faced by a woman named Joan Evans because a statistical analysis couldn’t be reproduced. Robert Gentleman had mentioned this in a talk at the Joint Statistical Meetings in Vancouver. Look for The Cancer Letter, Gentleman said.

I now realize that Joan Evans is Joe Nevins, who co-authored a major paper with Potti.

Speaking of Potti, members of the Duke administration are said to “have warned people not to even Google the name ‘Anil Potti,’”

A Duke University associate professor named Anil Pott who does cancer research turns out to have fabricated numerous details on applications for research money. The first fabrication to be noticed was that he had received a Rhodes Fellowship.

This is interesting because Duke had previously investigated him:

Late last year [2009], there was a crescendo that caused Duke to stop clinical trials on three of his research programs, two involving lung cancer and one involving breast cancer. In each program, Potti was giving patients chemotherapy — determining what drugs might work best and in what dosage — based upon his genome research.

In January Duke let these programs resume after an internal review. [emphasis added] And these are the precise programs where Duke — for the second time — has now suspended new [emphasis added] enrollments. . . . In an official statement on the winter review, Duke said it had determined Potti’s approaches were “viable and likely to succeed.”

Someone who appears to be a total fraud is called to Duke’s attention — and they find him innocent! This is what happened with the SEC and Madoff and Memorial University and Ranjit Chandra. Chandra’s research assistant, a nurse, told Memorial something was wrong and Memorial did nothing, or very little. Chandra then sued the nurse. He went on to write the paper that Saul Sternberg and I investigated.

Someone lies on his resume — it happens. That a prestigious institution like Duke let him continue to get away with it, possibly endangering patients and surely wasting vast resources, after it’s brought to their attention — not so well-known. So far, the New York Times has only covered the false-resume side of the story. You may recall how poorly Duke responded to charges against its lacrosse team.

As this unfolded, Duke had the following headline on its website: “Crisis management 101: What can BP CEO Hayward’s mistakes teach us”. From a CNN story in which a Duke expert was quoted.

Duke.Fact.Checker notes that Potti’s papers have at least 26 co-authors! Many with M.D.’s, who have or will tell thousands of trusting patients “you should take Drug X”. The patient endangerment is not trivial.

The Cancer Letter on Potti. Another issue of The Cancer Letter about it.