16 February 2018

Anecdotal evidence


Events were quite conflicting since I was talking with you last time. We had the breathtaking success of SpaceX, successful launching of a Falcon Heavy that attracted the same crowds like the beginning of the space exploration. And then current USA administration reminded me why I started this blog and this activism, by publicizing their budget proposals which included severe cuts to science. Yes, I know that the signed budget does not cut science, but Trump and company are still allocating money.
So, this time I decided to help you stay smart by explaining why an average scientist does not take anecdotal evidence seriously. 
Often various quacks will back up their claims with the anecdotal evidence. They will show you a photo of random humans, or testimonies that whatever they are peddling worked for someone. That is anecdotal evidence, even when quacks present you with multiple testimonies. It is still anecdotal.
Here I will discuss the situation when presented testimony is actually true because I’m certain you are capable to identify scam when quacks are lying to you.
In essence, anecdotal evidence is presenting examples where the outcome was positive to whatever quack is trying to peddle.
Scientists do not take such evidence seriously because it does not follow the scientific method.
First, one example does not prove a hypothesis. Let us talk about quacks peddling medicine. They will give you photos and testimonies of people who were cured by this medicine. And let us say that the people in examples really had an illness and really got cured. That still does not mean the quack’s cure is what cured them. But it means that quack knows human nature, and knows that one human face has a more emotional hold on an average human than numbers describing the analysis of the cure effects.
Second, the methodology of collecting anecdotal evidence is faulty. When quack is giving you an example of people helped by cure instead of results of analysis of the cure effectivity, alarms should ring. Because quack will not tell you how many cases cure did not help. If a cure helps only one person in billion, that is not really an effective cure, because it will be more likely you will end up among the billions for which cure does not work. If quack does not tell you when the cure does not work, chances are, you are being swindled.
Third, no controls. Even in the cases when quack’s medicine seemingly worked, we still do not have proof that it was that particular medicine that worked. It could be something else. After all, placebo works. A significant percentage of people will get better because they are thinking someone is giving them a cure. That’s why every official medicine has to perform better than placebo before it is approved for use. Quack’s cures are often effective only as a placebo. And some of the quacks will actually admit it convinced that's good.
Fourth, no replication of the results. Anecdotal evidence often cannot be reproduced, even if one makes all conditions the same. Mostly because not all conditions were taken into the account. But let us go even more basic. The truth, reality tends to be repeatable. Meaning if something is really true it will happen over and over again, regardless of who is doing the procedure.
Just think, a pebble you drop here in an everyday environment on the Earth will fall down, regardless of who is letting things fall. And that is because gravity is real.
That’s why there is a big insistence in science on making the whole experiment repeatable. That’s why good scientific paper always reports on the methodology and controls used in the research. And that’s why every research results have to be interpreted within limits of used methodology and controls.
Therefore, in the eyes of any scientist, anecdotal evidence is ridiculous. Yes, a scientist knows that the majority of people are more swayed by anecdotes and emotions. That’s simply human nature. So we need to be trained not to be swayed by anecdotal evidence, especially when is offered to us by trusting, helpful human face.
I can tell you that even I tend to fight those natural impulses. It is hard not to agree with trusting human face. What I found to help me is not to rush with my decision. I take time and do my research before I commit or make a decision. When a person is honest, they usually do not mind if I take my time. Crooks tend to create a sense of urgency. They do not wish you to think.
All this here is a summation of several different research works. If you wish to learn more about human irrationally and how to beat it, start your search by reading works of Dan Ariely.


No comments:

Post a Comment