Comments on: Twisted Skepticism

by: Mike Brown

Sat, 12 Apr 2008 15:05:58 +0000

This may be tangential to your discussion, but have you run across http://www.institutionalreviewblog.com/ The blogger there is a history teacher at Geo. Mason who has been thwarted by IRB rules intended, of course, to protect human subjects, but misapplied (he feels) to the social sciences. I’m currently going through the online IRB training through CITI and, while I totally agree that psychological and medical experiments need oversight, is it really necessary if I’m testing a user interface for data entry?

For whatever reason, I’m seeing parallels between this series of posts and my recent reading of horror stories related to getting IRB approvals. Another article about the blog is at http://insidehighered.com/news/2007/01/19/irb

by: Gustavo Lacerda

Mon, 07 Apr 2008 19:28:14 +0000

Darius,

This link seems to have the full article: “Creativity versus skepticism within science: more harm has been done in science by those who make a fetish out of skepticism, aborting ideas before they are born, than by those who gullibly accept untested theories.”
http://www.encyclopedia.com/doc/1G1-154238670.html

by: Darius Bacon

Mon, 07 Apr 2008 06:33:47 +0000

I just came across a ref to an article by V.S. Ramachandran titled “Creativity versus skepticism within science: more harm has been done in science by those who make a fetish out of skepticism, aborting ideas before they are born, than by those who gullibly accept untested theories.” That reminded me of this discussion — perhaps you’ll find it of interest.

http://www.amazon.com/Creativity-versus-skepticism-within-science/dp/B000KIXERI/

by: Andrew Gelman

Sat, 15 Mar 2008 22:09:44 +0000

Hi, Seth. The rest of my comment got cut off because I used the “less than” sign, which got interpreted as html. But I think I made the basic point above. It’s fine to respect unorthodox research, but, at some point, the work is so crappy that it’s the equivalent of reading tea leaves, or throwing darts at a newspaper and using the words to write poetry. It might provoke interesting thoughts, but I don’t consider it science. It’s more like literature, or philosophical speculation, leavened with statistical errors and irrelevant data.

by: Andrew Gelman

Sat, 15 Mar 2008 00:05:17 +0000

Seth,

There are no “Kanazawa data.” Kanazawa analyzed existing public data sources. You could give these data to your psych undergrads too, and if they know SPSS, they might come up with some interesting things too. What made Kanazawa’s work break the attention barrier (so that people like you and me have heard of him) was that his findings were surprising. He got surprising findings by making statistical errors. If he’d gathered his own data, I’d respect his work more. The point of my paper is not about p

by: Tony

Fri, 14 Mar 2008 22:02:39 +0000

Susan Haack was once taking questions after a talk, and was asked “What, essentially, do you think the scientific method is?” Her (prescriptive) response was: “Trying really hard to figure out the truth.”

The problem I have with this is that many scientific advances are made when scientists almost dogmatically hold onto beliefs in the teeth of contrary evidence. Their perseverance leads to the uncovering of evidence that does support their position …

by: seth

Fri, 14 Mar 2008 20:24:45 +0000

Andrew, thanks for clarifying that. Your paper argues that the Kanazawa study was too small to have a good chance of finding results with p < .05 given the likely size of the effect. Sure. I don’t agree that this means its info is useless. You write: “A study of this size is not fruitful for estimating variation on this scale.” That is too strong, I believe. I don’t think it’s true that nothing can be learned from the Kanazawa data. The Kanazawa data make some ideas more plausible, other ideas less plausible.

by: Andrew Gelman

Fri, 14 Mar 2008 19:28:27 +0000

Seth,

You question my claim, “Some studies just don’t have enough data to be relevant” and write, “What’s an example?”

Follow the link that I put in my comment. It gives an example, in excruciating detail.

by: seth

Fri, 14 Mar 2008 18:28:49 +0000

Were I to teach the basics of science, I would tell a bunch of stories chosen to show the value of a wide range of evidence, experimental, non-experimental, case reports, etc. I would say: we need all types, and explain why. A story-driven approach is a lot different than the current approach (e.g., evidence-based medicine) with its emphasis on rules (do this, do that) and value judgments (this is good, that is bad).

by: michael vassar

Fri, 14 Mar 2008 14:39:36 +0000

It’s definitely worse to build bikes with permanent training wheels than to teach people to ride bikes without training wheels, but better than to have a population of bikers who are always falling down. The question is, do we (collectively) know any fairly reliable way to teach people to use th “no training wheels” version of rationality instead of just “faking it” with the “scientific method”. If so, do we (personally) know how to get there from here, e.g. how to influence the culture in the relevant manner. Honestly, I think ‘no’ and ‘no’, but I think I’m closer to the second bit than to the first.