http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/ Self-Experimentation, Scientific Method, the Shangri-La Diet, etc. Thu, 20 Nov 2008 18:53:30 +0000 http://wordpress.org/?v=2.0.7 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50926 Thu, 11 Oct 2007 12:32:38 +0000 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50926 [...] The Twilight of Expertise (part 12: Super Crunchers) [...] […] The Twilight of Expertise (part 12: Super Crunchers) […]

]]> http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50828 Wed, 10 Oct 2007 19:45:26 +0000 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50828 [...] The Twilight of Expertise (part 12: Super Crunchers) [...] […] The Twilight of Expertise (part 12: Super Crunchers) […]

]]> http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50789 Wed, 10 Oct 2007 15:23:08 +0000 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50789 You're linking two transitions: (A) from hunter-gatherer to agricultural economics, and (B) from expert to math-model decision-making. Okay, each involves a replacement of creativity/expertise by a routine. We might throw in (C) the industrial revolution, where Adam Smith's pin factory replaces craftmanship by assembly-line production. However, the "Super-Cruncher" transition (B) seems to me to be fundamentally different in that the routine is to be carried out by a computer. The expert is upset at losing status, but the resulting society is not one of increased drudgery; it's simply a society in which people (patients, sports teams, investors, customers, governments) depend less on human experts and more on their "own" (computer) resources, as fed by their own intuitions and web-collected data from all over the world. See page 124: "The most important thing that is left to humans is to use our minds and our intuitions to guess at what variables should and should not be included in statistical analysis." This may be intrinsically closer to a hunter-gatherer structure after all; there's no hierarchy of experts, perhaps no hierarchy of authority at all. Of course we will need Gelmans for a while, but in the end perhaps we can breed good modeling algorithms via genetic algorithms; if experts are replaced by algorithms, meta-experts may be replaced by meta-algorithms. And then the Singularity? Well, maybe. It is a step in that direction, whether the journey continues or not. You’re linking two transitions: (A) from hunter-gatherer to agricultural economics, and (B) from expert to math-model decision-making. Okay, each involves a replacement of creativity/expertise by a routine. We might throw in (C) the industrial revolution, where Adam Smith’s pin factory replaces craftmanship by assembly-line production. However, the “Super-Cruncher” transition (B) seems to me to be fundamentally different in that the routine is to be carried out by a computer. The expert is upset at losing status, but the resulting society is not one of increased drudgery; it’s simply a society in which people (patients, sports teams, investors, customers, governments) depend less on human experts and more on their “own” (computer) resources, as fed by their own intuitions and web-collected data from all over the world. See page 124: “The most important thing that is left to humans is to use our minds and our intuitions to guess at what variables should and should not be included in statistical analysis.” This may be intrinsically closer to a hunter-gatherer structure after all; there’s no hierarchy of experts, perhaps no hierarchy of authority at all.
Of course we will need Gelmans for a while, but in the end perhaps we can breed good modeling algorithms via genetic algorithms; if experts are replaced by algorithms, meta-experts may be replaced by meta-algorithms. And then the Singularity? Well, maybe. It is a step in that direction, whether the journey continues or not.

]]> http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50788 Wed, 10 Oct 2007 15:19:12 +0000 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50788 Philip Tetlock's book on expert political prediction "Expert Political Judgment: How Good Is It? How Can We Know?” found (http://www.newyorker.com/archive/2005/12/05/051205crbo_books1) it to be largely bogus, as I understand it. The New Yorker summarized his findings. Even rats made better predictions than Yale students at a particular task. From Louis Menand's New Yorker article: "It is the somewhat gratifying lesson of Philip Tetlock’s new book, “Expert Political Judgment: How Good Is It? How Can We Know?” (Princeton; $35),that people who make prediction their business—people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables—are no better than the rest of us. When they’re wrong, they’re rarely held accountable, and they rarely admit it, either. They insist that they were just off on timing, or blindsided by an improbable event, or almost right, or wrong for the right reasons. They have the same repertoire of self-justifications that everyone has, and are no more inclined than anyone else to revise their beliefs about the way the world works, or ought to work, just because they made a mistake. No one is paying you for your gratuitous opinions about other people, but the experts are being paid, and Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote. Our system of expertise is completely inside out: it rewards bad judgments over good ones." Why am I not surprised? Philip Tetlock’s book on expert political prediction “Expert Political Judgment: How Good Is It? How Can We Know?” found
(http://www.newyorker.com/archive/2005/12/05/051205crbo_books1)
it to be largely bogus, as I understand it. The New Yorker summarized his findings. Even rats made better predictions than Yale students at a particular task.
From Louis Menand’s New Yorker article:
“It is the somewhat gratifying lesson of Philip Tetlock’s new book, “Expert Political Judgment: How Good Is It? How Can We Know?” (Princeton; $35),that people who make prediction their business—people who appear as experts on television, get quoted in newspaper articles, advise governments and businesses, and participate in punditry roundtables—are no better than the rest of us. When they’re wrong, they’re rarely held accountable, and they rarely admit it, either. They insist that they were just off on timing, or blindsided by an improbable event, or almost right, or wrong for the right reasons. They have the same repertoire of self-justifications that everyone has, and are no more inclined than anyone else to revise their beliefs about the way the world works, or ought to work, just because they made a mistake. No one is paying you for your gratuitous opinions about other people, but the experts are being paid, and Tetlock claims that the better known and more frequently quoted they are, the less reliable their guesses about the future are likely to be. The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge. People who follow current events by reading the papers and newsmagazines regularly can guess what is likely to happen about as accurately as the specialists whom the papers quote. Our system of expertise is completely inside out: it rewards bad judgments over good ones.”
Why am I not surprised?

]]> http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50774 Wed, 10 Oct 2007 12:23:48 +0000 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50774 You get some of the elements of hunting-gathering, at least of the gathering part, by shopping, hence, presumably, its popularity. Other elements can come from frantically running around to do small one-shot freelance jobs and tasks, such as being a taxi driver searching for fares. Video games presumably satisfy other elements of the faux-hunter lifestyle, or certain physical recreations such as hide-and-seek. You get some of the elements of hunting-gathering, at least of the gathering part, by shopping, hence, presumably, its popularity. Other elements can come from frantically running around to do small one-shot freelance jobs and tasks, such as being a taxi driver searching for fares. Video games presumably satisfy other elements of the faux-hunter lifestyle, or certain physical recreations such as hide-and-seek.

]]> http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50764 Wed, 10 Oct 2007 10:58:17 +0000 http://www.blog.sethroberts.net/2007/10/10/the-twilight-of-expertise-part-12-super-crunchers/#comment-50764 Sure, but what you call "the math models" (I would call them "statistical models" since I think when you fit a mathematical model to data, it's a statistical model) need experts to run them well. A good algorithm doesn't come from nowhere. Sure, but what you call “the math models” (I would call them “statistical models” since I think when you fit a mathematical model to data, it’s a statistical model) need experts to run them well. A good algorithm doesn’t come from nowhere.

]]>