CHAPTER 2:
To believe or not to believe
• The importance of “not always believing what we read in the papers” cannot be overstated.
• This lesson applies not just to newspapers but to academic research papers in journals also.
• In fact it applies everywhere someone presents us with a conclusion based on a study.
• When we are presented with a study we should examine 7 components of the study:
1. The source of the research and of the funding.
2. The researchers who had contact with the participants.
3. The individuals or objects studied and how they were selected.
4. The exact nature of the measurements made or the questions asked. 5. The setting in which the measurements were asked.
6. The extraneous differences in groups being compared.
7. The magnitude of any claimed effects or differences.
1. The source of the research and of the funding.
Research costs money, researchers need to be paid, equipment has to be bought, subjects need to be found.
We should always be aware of this fact when we examine the claims made by researchers.
•
•
•
•
We should look differently on research conducted by
Independent agencies e.g. CSO, Eurostat,NSC
Academics (who funds them?)
Companies trying to convince consumers to buy their product instead of a competitors.
Journalists
2. The researchers who had contact with the participants.
When a study is conducted, the responses of an individual may depend on who asks the questions.
Question: How much money do you earn?
Response to a Revenue Commissioner will probably be different than to a friend or date whom you are trying to impress. Would you trust a study on the use of illegal drugs which was carried out by Gardai knocking on peoples’ doors.
3. The individuals or objects studied and how they were selected. • THE INDIVIDUALS STUDIED
Can we apply the results of a study conducted on men to women? Does a study conducted on Americans apply to Irish people? • HOW THEY WERE SELECTED
Many studies rely on volunteers, is this wise?
Is there not a difference between the kind of people who volunteer for a study and those who don’t.
Consider what would happen if someone came up to you in the street with a questionnaire.
4. The exact nature of the measurements made or the questions asked.
You should be aware that the wording and the ordering of questions can influence responses.
How do you feel about all those foreigners coming over here taking our jobs?
What is your opinion on the plight of refugees forced by war in Darfur to flee their homes and come to Ireland?
Do you have confidence in the Government?
How do you feel about the corruption unearthed by recent tribunals? 5. The setting in which measurements were taken.
When and where was the study conducted.
Studies conducted at certain times of the day may exclude certain elements of the target population.
@3.00pm many employed people are at work, so a study conducted on Grafton St. at that time will probably not be representative. Phone surveys conducted during the day will also probably under-represent the views of “the working stiff”.
Capilano Suspension Bridge
How you would reply to certain questions posed in a police interrogation room would probably differ to how you would answer those same questions in a pub.
6. The extraneous differences in groups being compared.
CONFOUNDING FACTORS
A study shows that exam scores among marijuana smokers are lower than among non-smokers.
A conclusion is drawn that marijuana, impairs exam performance. We should however consider that the type of person who smokes dope may be the kind of unmotivated slacker who doesn’t do enough study for their exams irrespective of whether they smoked or not.
7. The magnitude of any claimed effects or differences.
Newspapers seldom say how large the effects of a statistical study are.
Are the results STATISTICALLY SIGNIFICANT?
UK & IRISH General