Unfortunate Obituaries: The Case of David Freedman
One of my colleagues at Berkeley didn’t return library books. He kept them in his office, as if he owned them. He didn’t pay bills, either: He stuck them in his desk drawer. He was smart and interesting but after he failed to show up at lunch date — no explanation, no apology — I stopped having lunch with him. He died several years ago. At his memorial service, at the Berkeley Faculty Club, one of the speakers mentioned his non-return of library books and non-payment of bills as if they were amusing eccentricities! I’m sure they were signs of a bigger problem. He did no research, no scholarly work of any sort. When talking about science with him — a Berkeley professor in a science department — it was like talking to a non-scientist.
David Freedman, a Berkeley statistics professor who died recently, was more influential. He is best known for a popular introductory textbook. The work of his I found most interesting was his comments on census adjustment: He was against adjusting the census to remove bias caused by undercount. This was only slightly less ridiculous than not returning library books — and far more harmful, because his arguments were used by Republicans to block census adjustment. The undercounted tended to vote Democrat. The similarity with my delinquent colleague is the very first line in Freedman’s obituary: He “fought for three decades to keep the United States census on a firm statistical foundation.” Please. A Berkeley statistics professor, I have no idea who, must have written or approved that statement!
The obituary elaborates on this supposed contribution:
“The census turns out to be remarkably good, despite the generally bad press reviews,” Freedman and Wachter wrote in a 2001 paper published in the journal Society. “Statistical adjustment is unlikely to improve the accuracy, because adjustment can easily put in more error than it takes out.”
There are two kinds of error: variance and bias. The adjustment would surely increase variance and almost surely decrease bias. The quoted comments ignore this. They are a modern Let Them Eat Cake.
Few people hoard library books, but Freedman’s misbehavior is common. I blogged earlier about a blue-ribbon nutrition committee that ignored evidence that didn’t come from a double-blind trial. Late in his career, Freedman spent a great deal of time criticizing other people’s work. Maybe his critiques did some good but I thought they were obvious (the assumptions of the statistical method weren’t clearly satisfied — who knew?) and that it was lazy the way he would merely show that the criticized work (e.g., earthquake prediction) fell short of perfection and fail to show how it related to other work in its field — whether it was an improvement or not. As they say, he could see the cost of everything and the value of nothing. That he felt comfortable spending most of his time doing this, and his obituary would praise it (”the skeptical conscience of statistics”), says something highly unflattering about modern scientific culture.
For reasonable comments about census adjustment, see Eriksen, Eugene P., Kadane, Joseph B., and Tukey, John W. (1989). Adjusting the 1980 census of population and housing. JASA, 84, 927-943.
December 3rd, 2008 at 10:01 am
You seem to be saying that Freedman was a bad person because you disagree with his view on censuses and because republicans agree with him. You think it obvious that he was wrong, but I don’t see anything obviously wrong with it.
December 3rd, 2008 at 1:31 pm
“Bad person”? No, bad behavior. You’re right, Dennis, that he was wrong might not be obvious to a someone not professionally involved with estimation — and I suppose at least a few statistics professors even agreed with him. (But when I brought up Freedman’s position with my friends who were statistics professors, they agreed with me.) That’s why I gave a citation with a detailed explanation of why his position was wrong. No one doubted that the census undercounted some people more than others. Freedman was against fixing this so that you could no longer predict who was undercounted. Suppose a store systematically overcharged you. What would you think of someone who opposed solving the problem using the best methods available so that, on average, you were charged the correct price? That Republican politicians agreed with him is not part of why I dislike his behavior. It was why his behavior was influential.
December 3rd, 2008 at 2:44 pm
Seth,
He may have been a bastard, he may have been ideologically blinded. But the time to address those issues was when he was alive.
Obituaries are romanticized for the benefit of the survivors.
–Ed
December 3rd, 2008 at 3:41 pm
Ed, I made these points when Freedman was alive, too. Many times. I hope you’re not saying no one should criticize a dead person or the culture that produced him.
I’m not saying he was a bastard or ideologically blinded, certainly not. For all I know he was a Democrat.
The problem with the obituary is not “romanticization”. My point is not that Freedman did this stuff but that other professors saw this as okay, even good. My point is about the culture, in other words. It is one reason academia is called the ivory tower. Romanticization in obituaries is fine. Romanticization would have been to leave the census stuff out of the obituary. Freedman did a lot of perfectly good other stuff.
December 4th, 2008 at 9:16 am
Seth,
I didn’t mean to say that one shouldn’t criticize the dead.
And you didn’t call Freedman a bastard. Anyone’s obituary, bastard or ideologue is romanticized.
I think I understand now… The issue is the “Census Stuff” that should have been frowned upon and challenged, wasn’t, even worse it was celebrated in his obituary. Didn’t get that on first or second reading of the post, maybe I need more flaxseed oil.
Anyway, thank you for your posts on academia.
–Ed
December 18th, 2008 at 1:40 am
It is not clear if census adjustment would have reduced bias (at the cost of variance) as you state. Indeed my bet is that David Freedman is correct: model adjustment would have made the bias worse. The issue is not one of just random sampling. Much model based extrapolation was being proposed. The devil is in the details, and vague gestures do not an argument make. A hint of the complexity is that by the 2000 census the partisan impact of the undercount was probably the reverse of what you conjecture. For the purposes of apportionment, the south west (e.g., states like Texas) would have gained Congressional seats and hence electoral college votes at the expense of the north east. So, a correction would have been a win for the Republicans in national elections. Note that apportionment counts illegal immigrants, although they cannot vote. Much like many academics, the political parties took their positions based on weak evidence and held their ground even when the facts shifted. Of course the scientific issue is that the undercount is difficult to fix and who is being undercounted is constantly changing. The problem is more difficult than calculating the partisan impact, and most people can’t even get that part right.
December 18th, 2008 at 5:52 pm
Jas, where is it argued that census adjustment was more likely to increase bias than reduce it? I have never heard that argument nor do I understand it.
December 20th, 2008 at 5:46 pm
Freedman argued that census adjustment would introduce bias of unknown magnitude. He proved this by following essentially the same methodology but making minor changes, and showing that substantial differences would arise in the adjustment. As Jas says, the devil is in the details, which must be understood here to follow what is going on.
Following the census, there is a survey which attempts to measure the undercount which occurred in the census. This survey itself is subject to the same undercount problem as the census. A fundamental assumption which makes adjustment possible is that these two undercounts are INDEPENDENT. Freedman argues that this is not true — a person who is undercounted in the census is also likely to be undercounted in the subsequent survey. This makes eminent sense. If there is correlation between the undercounts, this will lead to bias in the adjustment. How large this bias can be is assessed in the paper cited below, which shows that it is likely to be so large as to make the adjustment worse than useless.
K.W. Wachter and D.A. Freedman. “The fifth cell: Correlation bias in U.S. census adjustment.” Evaluation Review, vol. 24 (2000) pp. 191–211
December 30th, 2008 at 8:49 pm
I don’t think anyone is disputing whther biases exist in the census adjustments. On any given census you can be sure that it will be biased one way or another depending on the methodologies used. Given the likelihood that undercounts are correlated, I agree with Mr. Zaman, Wachter and Freedman, in that the adjustments are likely “less than useless”.
taw
December 30th, 2008 at 10:01 pm
Terry, you wrote “given the likelihood that undercounts are correlated” — I don’t think anyone disputes this either. This is the puzzle behind Freedman’s criticisms: he seemed to equate lack of perfection with uselessness. I haven’t read the Wachter and Freedman yet, but I will, and then after that maybe I will understand Freedman’s criticisms better.
December 31st, 2008 at 1:20 pm
Once the decision is made to adjust the census results, the adjustment will be just one more political issue. There is no guarantee that unbiased statistical methods will be used. Do you think the average (or above-average) Congressman understands the difference between variance and bias? Would he even want to? All he has to know is how to get his party’s statisticians on the committee that does the adjustments.
On a related question, how would everyone feel about using statistical methods to adjust the vote totals in elections? The arguments would be similar. A lot of people don’t vote, we could get a more accurate outcome with adjustment. Right?