Seth’s blog

In 1977, John Tukey published a book called Exploratory Data Analysis. It introduced many new ways of analyzing data, all relatively simple. Most of the new ways involved plotting your data. A few involved transforming your data. Tukey’s broad point was that statisticians (taught by statistics professors) were missing a lot: Conventional statistics focussed too much on confirmatory data analysis (testing hypotheses) to the omission of exploratory data analysis — data analysis that might show you something new. Here are some tools to help you explore your data, Tukey was saying.

No question the new tools are useful. I have found great benefits from plotting and transforming my data. No question that conventional statistics textbooks place far too little emphasis on graphs and transformations. But I no longer agree with Tukey’s exploratory versus confirmatory distinction. The distinction that matters — at least to historians, if not to data analysts — is between low-status and high-status. A more accurate title of Tukey’s book would have been Low-Status Data Analysis. Exploratory data analysis already had a derogatory name: Descriptive data analysis. As in mere description. Graphs and transformations are low-status. They are low-status because graphs are common and transformations are easy. Anyone can make a graph or transform their data. I believe they were neglected for that reason. To show their high status, statistics professors focused their research and teaching on more difficult and esoteric stuff — like complicated regression. That the new stuff wasn’t terribly useful (compared to graphs and transformations) mattered little. Like all academics — like everyone — they cared enormously about showing high status. It was far more important to be impressive than to be useful. As Veblen showed, it might have helped that the new stuff wasn’t very useful. “Applied” science is lower status than “pure” science.

That most of what statistics professors have developed (and taught) is less useful than graphs and transformations strikes me as utterly clear. My explanation is that in statistics, just as in every other academic area I know about, desire to display status led to a lot of useless highly-visible work. (What Veblen called conspicuous waste.) Less visibly, it led to the best tools being neglected. Tukey saw the neglect –  underdevelopment and underteaching of graphs, for example — but perhaps misdiagnosed the cause. Here’s why Tukey’s exploratory versus confirmatory distinction was misleading: Because the tools that Tukey promoted for exploration also improve confirmation. They are neglected everywhere. For example:

1. Graphs improve confirmatory data analysis. If you do a t test (or compute a p value in any way) but don’t make an associated graph, there is room for improvement. A graph will show whether the assumptions of the computation are reasonable. Often they aren’t.

2. Transformations improve confirmatory data analysis. That a good transformation will make the assumptions of the test more reasonable many people know. What few people seem to know is that a good transformation will make the statistical test more sensitive. If a difference exists, the test will be more likely to detect it. This is like increasing your sample size at no extra cost.

3. Exploratory data analysis is sometimes thought of as going beyond the question you started with to find other structure in the data — to explore your data. (Tukey saw it this way.) But to answer the question you started with as well as possible you should find all the structure in the data. Suppose my question is whether X has an effect.  I should care whether Y and Z have an effect in order to (a) make my test of X more sensitive (by removing the effects of Y and Z) and (b) assess the generality of the effect of X (does it interact with Y or Z?).

Most statistics professors and their textbooks have neglected all uses of graphs and transformations, not just their exploratory uses. I used to think exploratory data analysis (and exploratory science more generally) needed different tools than confirmatory data analysis and confirmatory science. Now I don’t. A big simplification.

Exploration (generating new ideas) and confirmation (testing old ideas) are outputs of data analysis, not inputs. To explore your data and to test ideas you already have you should do exactly the same analysis. What’s good for one is good for the other.

Likewise, Freakonomics could have been titled Low-status Economics. That’s essentially what it was, the common theme. Levitt studied all sorts of things other economists thought were beneath them to study. That was Levitt’s real innovation — showing that these questions were neglected. Unsurprisingly, the general public, uninterested in the status of economists, found the work more interesting than high-status economics. I’m sensitive to this because my self-experimentation was extremely low-status. It was useful (low-status), cheap (low-status), small (low-status), and anyone could do it (extremely low status).

More Andrew Gelman comments. Robin Hanson comments.

Alexandra Carmichael is one of the founders of CureTogether.com, whom I met at a Quantified Self meeting last year. A few days ago, she left an interesting comment on one of my posts:

I practice random acts of kindness, with a goal of helping at least 10 people a day (and at least 1 person I don’t know). I find this helps my mood toward the end of the day, when it is most likely to fall - no matter what else has happened that day, at least I’ve helped 10 people.

I asked her about it:

SETH Where did the idea come from?

ALEXANDRA It goes all the way back to my grandparents being Scout leaders - I was never in the Scouts myself but I observed how helpful and supportive they always were. Then during my university years when I was forming my life philosophy, I got to attend an incredible lecture by Jane Goodall. Her organization Roots & Shoots inspires people around the world to give back to the earth, animals, and people around them, with her amazing presence and the quote “Every individual can make a difference.” Service learning is also one of the things we thread into homeschooling our two daughters, along with design, simple living, and non-violent communication.

The specific goal of helping 10 people a day started last summer during a goal-setting weekend. I was curious to see if formalizing and quantifying something I had been doing in a fuzzier way would make a difference in my life, if measuring acts of kindness would result in an increased number of acts, or more friends, or help me with my chronic depression - plus I love quantifying things! I don’t find it necessary to actually record how many people I help in a day, but I keep a rough running tally in my head as I go through the day to make sure it’s at least 10 - my kids like to help with this count too.

SETH What are some examples of these acts?

ALEXANDRA I do a lot of different things. If I get extra free tickets to events or conferences, I will pass them along to people who I think would love to go; I will offer to take a picture of a tourist family where one person inevitably gets left out behind the camera; I will connect people who I think would benefit from knowing each other; I will take two hours to listen and hug and support a child who is having a hard time learning a new skill; I will answer a newbie entrepreneur’s questions about how to get started in business or help them spread their message; I will help coordinate gatherings that I believe in (such as Quantified Self); I will hold the door for someone. It can be anything really, no matter how small.

SETH How have people reacted when you tell them about this?

ALEXANDRA The most frequent reaction is “That sounds too challenging to do every day - 10 people? Why not 1 or 2?” The second most frequent reaction is “You are inspiring me to make positive changes in my own life.” My answer to both is “I love helping people!”

SETH What have you learned?

ALEXANDRA if you help people, without wanting anything in return, you get help when you need it - often surprising help, and often more than you gave. I learned that helping people seems to make them like you more, so my number of online friends has skyrocketed (1500 on Twitter, 800 on Facebook, 500 on LinkedIn) - but close “in person” friends I choose to limit to a handful because of my tendency to get overwhelmed by frequent or shallow social situations. I learned that helping people does help with depression, because (a) you have something else to focus on outside of yourself and (b) you go through the day with an expectant air of wonder at who will be the next person you can help. I also learned that helping 10 people a day is really not a lot, and I often wind up helping 20 or more people in a day. Of course, this is only from my perspective - I can’t guarantee that all of these people actually feel helped, I just know that I tried to help.

SETH When you say “if you help people, without wanting anything in return, you get help when you need it - often surprising help, and often more than you gave” I’m not sure I understand. Can you give some examples?

ALEXANDRA It’s not so much that the people I help help me in return, but more that by spreading goodwill and being tuned in to what others need, I also became more aware of my own needs and started to feel a greater sense of self-worth, like I deserved to have my needs met. This is not something I was taught growing up, and I went through two bouts of major postpartum depression without asking for or getting the support I needed. I feel much more open about my needs now, which perhaps makes it easier for others to help me. So the change was more in me than in others.

In terms of specific examples, when I learned that I have a Tourette’s spectrum disorder, and tweeted that, I made an incredible new friend who has been through similar neurological issues, and who in our conversations of support and empathy has helped me more than I can ever thank him for. Also, when I decided to find some consulting work to support my family while we build CureTogether, a very welcoming door opened (soon to be made public), and offered me basically a dream position. I guess I needed to learn to ask for and accept help as well as to give it.

SETH Thanks, Alexandra. It’s especially interesting that helping others raised your feeling of self-worth. I wouldn’t have guessed it would have that effect.

In the current New Yorker, James Surowiecki writes:

The bailout of the auto industry, after all, was as unpopular as the bailout of the banks, even though it was much tougher on the companies (G.M. and Chrysler went bankrupt; shareholders were wiped out, and C.E.O.s pushed out), and even though the biggest beneficiaries of the deal were ordinary autoworkers. You might have expected a deal that helped workers keep their jobs to play well in a country spooked by ballooning unemployment. Yet most voters hated it.

Yes, rewarding failure doesn’t play well. The voters were right. The same money that was used to give a few giant companies a second (or third) chance could have been used to give many thousands of very small companies a first chance. It could have been used to help many thousands of people start new small businesses (often one-person businesses) or keep their new small business afloat. All those small businesses would have provided plenty of jobs. and they would have had a far more promising future, far more room for growth, than the Big Three, being both far more diverse and having not already failed. The many thousands of people who wanted to start small businesses were unable to get together and make themselves visible, so the failure of government to help them went unnoticed. Their diversity was economic strength but political weakness.

It’ isn’t surprising things happened as they did — the Big Three (not to mention Wall Street) were bailed out, small businesses were ignored — but it is an indication of how poorly our economy is managed in the most basic ways. I’m not even an economist and I understand this simple point. Bernanke and Summers do not.

It’s easy for me to understand because the same thing happens in science. Government support of research is a good idea, but the money is misspent, in the same way. Grant support goes to a few large projects — generally to people who have already failed (to do anything useful) — rather than to a large number of small projects that haven’t yet failed. The way to support innovation is to place many small bets not a few big ones. That’s one thing I learned from self-experimentation, which allowed me to place many small bets.

Kathleen Hillers posted this on a website called The Intention Experiment:

I just read a book called 29 Gifts: How a Month of Giving Can Change Your Life by Cami Walker. The author of the book has ms and was seeking natural healing. She was told by a “wise woman” from South Africa that if she gave a gift everyday for the next 29 days that it would have a healing effect in more ways than one. It’s a great book, but if you don’t want to read it, start giving a gift everyday and make a journal of every gift you give and the circumstances involved. If you miss a day, you have to start over because you have to keep the flow of giving constant. The gifts do not have to be materialistic. You can give some one a phone call, a ride, encouragement, whatever. I just started doing this on Feb 1st and my life is already getting better. The day before I started, I was in a panic. I couldn’t sleep, and I was completely broke . The day I started, i actually started feeling much better, and things are already looking up.

Regression to the mean, maybe. But maybe not. The idea has some plausibility: The Chinese character that means “happy” is a combination of a character that means “owe” and a character that means “again”.

A new study in the Archives of General Psychiatry, summarized in the Wall Street Journal:

Researchers in the new study identified 81 people, ages 13 to 25, with warning signs of psychosis, including sleeping much more or less than usual, growing suspicious of others, believing someone is putting thoughts in their head or believing they have magical powers. Forty-one were randomly assigned to take four fish oil pills a day for three months. The other patients took dummy pills.

After a year of monitoring, 2 of the 41 patients in the fish oil group, or about 5%, had become psychotic, or completely out of touch with reality. In the placebo group, 11 of 40 became psychotic, about 28%.

The study is impressive not only because it uses ordinary food (fish oil) rather than  dangerous drugs (such as Prozac) but also because it studies prevention. Just as the ketogenic diet suggests a widespread animal-fat deficiency, so this study suggests a widespread omega-3 deficiency, which won’t surprise any reader of this blog. Completing the picture — I believe most Americans eat far too little animal fat, omega-3, and fermented food — baker’s yeast is being studied as a cure for cancer.

Thanks to Oskar Pearson and Chris.