cheatsheet_lesson2

Critical Thinking Cheat Sheet: Lesson 2 - Evaluating Evidence

Evidence Hierarchy (Strongest to Weakest)

  1. Direct Empirical Evidence: Observable through senses or measuring instruments
  2. Peer-Reviewed Scientific Research: Studies scrutinized by experts and published
  3. Expert Consensus: Agreement among specialists in a field
  4. Official Records: Information from established institutions with verification processes
  5. Journalistic Reporting: Information gathered by professional journalists
  6. Eyewitness Testimony: First-hand accounts (often unreliable despite seeming compelling)
  7. Anecdotal Evidence: Personal stories or isolated examples
  8. Testimonials: Personal statements often used in marketing
  9. Appeals to Tradition/Popularity: Claims based on longevity or popularity

The CRAAP Test for Source Evaluation

Letter
Stands For
Key Questions
C
Currency
When was it published/updated? Is it recent enough for your topic?
R
Relevance
Does it directly address your question? Is it at an appropriate level?
A
Authority
Who is the author/publisher? What are their credentials? Any conflicts of interest?
A
Accuracy
Is it supported by evidence? Has it been reviewed or fact-checked? Can you verify it?
P
Purpose
Why was this created? To inform, teach, sell, entertain, or persuade?

Red Flags: Warning Signs of Misleading Evidence

  • Cherry-Picking: Selecting only data that supports a position
  • Correlation ≠ Causation: Assuming that because two things happen together, one causes the other
  • Small/Biased Samples: Drawing broad conclusions from too few examples
  • Misleading Statistics: Numbers presented without context or manipulated
  • Appeal to False Authority: Citing experts speaking outside their expertise
  • Weasel Words: Vague language creating impressions without verifiable claims
  • Emotional Manipulation: Using emotional language to override critical thinking

Scientific Evidence Hierarchy

  1. Systematic reviews and meta-analyses
  2. Randomized controlled trials
  3. Cohort studies
  4. Case-control studies
  5. Cross-sectional surveys
  6. Case reports
  7. Expert opinions
  8. Animal or laboratory studies

Practical Strategies for Information Overload

  • Prioritize scrutiny for information that influences important decisions
  • Use trusted information filters (fact-checking organizations, reputable outlets)
  • Look for consensus rather than outliers
  • Practice “satisficing” for low-stakes matters
  • Remember: The goal isn’t perfect information but appropriate confidence based on evidence quality