“The lack of a real control group, subject self-selection, and other threats to
internal validity mentioned above make it impossible to distinguish regression to the mean from a treatment effect… However, this study takes place in the real world in which the prospects for good multi-year studies are severely limited by funding, compliance with protocols, and cooperative participants. Policy decisions need to be made even on programs whose effectiveness have not been evaluated at all. Assessing the study from a pragmatic policy-making perspective, interventions such as these have great promise… so some generosity should be shown in interpreting the results…[The program] is relatively low cost compared with alternatives … and unlike these programs, it has been studied. We can’t prove that the intervention had any effect … , but it may very well be the best option among the set of alternatives.”

Or perhaps activities that lead to greater observation or awareness of phenomena, or to further hypotheses. Part of what you’re identifying is disjunctive thinking and an unwillingness in scientists to acknowledge a huge grey area.There is a huge continuum between partial confirmation of a hypotheses and totally unquestionable science. What determines what falls where on that continuum? Of course a controlled 5 year study with thousands of subjects might be ideal in confirming the Shangri-la Diet, but as more people have success with it over longer periods of time, it becomes more and more trivial.

A glaring error related to people who read about your diet: there seem to be doctors and experts who dismiss it as “placebo” when they could confirm that extra light olive oil, and associated methods, reduce hunger in the time between breakfast and lunch. Can you articulate the mind-set that leads to such intellectual incompetence? An enthusiastic teenager might be more likely to confirm what’s going on with the olive oil than many educated or mis-educated people. Much of the problem seems to be letting the perfect be the enemy of the good. There is a belief that because I know the hypothesis that I am trying to confirm, I cannot experiment on myself. But it would be pretty bizarre if my beliefs could alone take away hunger.

Part of the problem is that you are threatening some peoples’ sense of their own authority and power. To try out your method and see that it reduces hunger would be ego threatening.

It’s true that one person cannot completely confirm something through self experimentation, because there may be something unusual about them resulting in the effect. But even 5 or 10 people all getting the same effect in self experimentation gives me, at least, a lot of confidence that something interesting is going on, though we always have to be careful about self fulfilling prophecies.

It isn’t easy to argue how more evidence can be worse. It’s not “results” that are false (unless fabricated); it’s the conclusions drawn from them.

Did you mean “only if the scale is ratio or ordinal” here?