Prior Probability: The Dirty Little Secret of “Evidence-Based Alternative Medicine”

“Evidence-based alternative medicine” sounds like a unicorn you can bill to insurance: magical, comforting, andif you squintscientific-ish.
The pitch usually goes like this: We ran a study. The p-value was under 0.05. Boom. Science approved.
But here’s the awkward part nobody puts on the brochure: evidence doesn’t float in a vacuum. Evidence lands in a world where some claims are
wildly plausible and others are… basically asking physics to “just be cool for a second.”

The missing ingredient is prior probabilitysometimes called prior plausibility or pre-test probability.
It’s the “before we saw this study, how likely was this claim to be true?” question. And it matters because
a statistically significant result can still be a false alarmespecially when the claim started out unlikely.
That’s the dirty little secret: if your prior probability is tiny, you need extraordinary evidence to move the needle.

This isn’t a rant against curiosity or new ideas. It’s a reminder that medicine isn’t just “data” and “vibes.”
It’s biology, chemistry, physics, and clinical outcomesplus a whole lot of human bias.
If you want to separate promising complementary practices from expensive placebo theater, you have to talk about priors.
Yes, even at the risk of sounding like the person who brings a calculator to a drum circle.

What “Prior Probability” Actually Means (Without Making Your Eyes Glaze Over)

Prior probability is your starting plausibility estimate based on everything we already know:
basic science, previous studies, established mechanisms, and how often similar claims have panned out.
In Bayesian terms, your prior combines with new evidence to produce a posterior probabilityyour updated belief after seeing the data.
In everyday terms: it’s the difference between “possible” and “possible but also my cat might file taxes.”

A quick example: seatbelts vs. magic water

• Claim A: “Seatbelts reduce injury in car crashes.”
  The prior plausibility is high because it matches physics, biomechanics, and decades of real-world data.
• Claim B: “Water remembers healing intentions after being diluted until no molecules remain.”
  The prior plausibility is extremely low because it conflicts with chemistry and known physical behavior of liquids.

Both claims could be tested, but they don’t start on equal footing. Treating them like they do is how you end up
“evidence-based” your way into nonsense with a straight face.

Why P-Values Can’t Save Implausible Claims

A lot of “evidence-based alternative medicine” marketing leans on p-values. The unspoken assumption is:
p < 0.05 means it works. That’s not what a p-value means. A p-value is a measure of how compatible the data are
with a particular statistical model (often assuming no effect). It is not the probability the treatment works.

Even worse: in research areas with many low-plausibility hypotheses (or lots of small studies, multiple outcomes, flexible analyses, and publication bias),
“statistically significant” results are more likely to be false positives. This is why discussions of false discovery rate,
false positive risk, and pre-study odds show up whenever serious methodologists talk about reliability.
When the base rate of true effects is low, you can “discover” your way into a pile of mirages.

The base-rate problem in plain English

Imagine testing 1,000 claims. If only 10 are actually true (low prior probability across the set), and your study design and analysis
create a 5% false-positive rate, you can easily end up with a bunch of “positive” results where most are wrong.
That’s not a scandal; it’s math. It’s also why “but there are studies!” is not the slam-dunk people think it is.

The “Evidence-Based” Label Gets Weird in Alternative Medicine

In mainstream evidence-based medicine, the “evidence” is supposed to sit on a foundation of plausible mechanisms and prior knowledge.
In many alternative medicine subcultures, “evidence” becomes a costume: put a lab coat on a claim, run a small trial, and call it validated.
That’s how we get the same wordevidenceused to mean two very different things:
responsibly accumulated knowledge versus any study-shaped object.

Some complementary practices are plausible (and sometimes helpful) because they overlap with established physiology:
movement, mindfulness, sleep hygiene, stress management, physical therapy-style interventions.
Others depend on mechanisms that don’t map onto reality as we understand itlike ultra-dilutions, energy fields with no measurable properties,
or distant intention changing tissue healing. Those require a far more skeptical prior.

Three buckets: plausible, unclear, and implausible

1. Plausible adjuncts: approaches with known or reasonable mechanisms (e.g., certain mind-body practices for stress reduction,
   movement-based therapies, some nutritional interventions when evidence supports them).
2. Unclear-but-testable: interventions with incomplete mechanisms but not obviously impossible. These deserve careful trials
   with good controls and transparent reporting.
3. Implausible claims: interventions that would require rewriting basic physics/chemistry/biology to be true. These start with
   a very low prior probability and need exceptionally strong, reproducible evidence to be taken seriously.

Concrete Examples: How Priors Change the Interpretation

Example 1: Homeopathy

Homeopathy’s core ideas include “like cures like” and remedies diluted to the point where no molecules of the original substance remain.
The claimed mechanism is incompatible with basic chemistry as we currently understand it, which drives the prior probability down.
So even if a study reports a modest benefit, the rational question is: is that a true effector a product of bias, random variation,
poor blinding, selective reporting, or placebo effects?

This doesn’t mean you stop asking questions. It means you demand higher-quality evidence: larger studies, rigorous controls,
preregistered outcomes, independent replication, and transparent data. If the effect is real and robust, it should survive those conditions.

Example 2: Acupuncture

Acupuncture is interesting because it straddles the plausibility line. Needle stimulation can plausibly influence pain perception
via nerves and neurochemical pathways. That gives some prior plausibility to certain outcomes (like pain modulation).
But claims tied to specific meridians, invisible energy flows, or organ “rebalancing” can drift into lower-plausibility territory.

The evidence picture often depends on the outcome, the condition, and how well studies control for expectation effects.
“It helps some people with some kinds of pain” is a different claim from “it fixes a wide range of internal diseases by correcting qi.”
Priors help you keep those claims from being mashed into one overconfident headline.

Example 3: Distant prayer, reiki, and “energy healing”

Claims that healing can be transmitted at a distance without a known physical mechanism typically start with an extremely low prior probability.
That doesn’t make studies impossible, but it changes what counts as persuasive evidence.
A single “positive” trialespecially if small, noisy, or difficult to replicateshould be treated as a signal to investigate,
not a green light to declare victory.

“But It’s Evidence-Based!”: The Usual Talking Points (And the Reality Check)

Talking point: “There are peer-reviewed studies.”

Reality check: peer review is quality control, not truth certification. It can filter obvious problems, but it doesn’t erase bias,
small sample sizes, questionable analyses, or selective publication of positive results.
A field can accumulate a stack of “positive” papers and still be wrong if the underlying signal is weak and the noise is loud.

Talking point: “It’s statistically significant.”

Reality check: statistical significance is not clinical significance, and it’s not the probability the treatment works.
With enough participants, even tiny differences can become significant. With enough analytical flexibility,
almost anything can be made to sparkle.

Talking point: “It’s safe, so why not try it?”

Reality check: “safe” is not a free pass. Some interventions are directly risky (contamination, interactions, delay of effective treatment),
while others are indirectly harmful by replacing proven care or draining time and money.
Plus, “why not” is not how we should decide what becomes a healthcare standard.

How to Evaluate Claims Without Becoming a Full-Time Skeptic Monk

You don’t need to memorize Bayes’ theorem to think Bayesian. You just need a habit of asking better questions.
Here’s a practical checklist you can use when you see “evidence-based” attached to a complementary or alternative therapy.

1) Start with plausibility

• Does the claim fit with established biology and chemistry?
• Is there a reasonable mechanism, or is it hand-waving?
• Have similar claims worked before, or is the field full of vanishing effects?

2) Look for rigorous study design

• Randomization, blinding, and appropriate controls (including credible sham controls when needed)
• Preregistered outcomes and transparent reporting
• Large enough sample sizes to avoid “small study exaggeration”

3) Demand replication and consistency

• Do independent groups find the same result?
• Do meta-analyses show a stable effect, or does the signal shrink as studies improve?
• Are negative results published, or does everything mysteriously “work”?

4) Separate “feels good” from “works”

People can feel better for real reasonsattention, reassurance, supportive touch, relaxation, better sleep, hopeful framing.
Those can matter for well-being. But they are not proof that a specific proposed mechanism is true, and they don’t automatically
justify medical claims about treating disease.

Important note: This is general education, not personal medical advice.
If you’re considering any therapyespecially if you have a health conditiontalk with a licensed healthcare professional.
“Natural” and “alternative” can still interact with medications or delay needed care.

So Why Is Prior Probability the “Dirty Little Secret”?

Because it forces honesty. If we openly weigh plausibility, some beloved “alternative” claims start with priors so low
that the usual level of evidence used in medicine is nowhere near enough. That can feel unfair to proponents:
“Why do we need more proof than everyone else?”

The answer is blunt but fair: because you’re making a more extraordinary claim.
If a therapy implies a modest tweak to a known biological pathway, it starts closer to “maybe.”
If it implies water has memory, energy fields are clinically manipulable but undetectable, or intention acts at a distance,
it starts closer to “extraordinary.” Extraordinary claims don’t get a participation trophy; they get a heavier burden of proof.

The good news is that this isn’t anti-innovation. It’s pro-reliability. Prior probability helps science spend its limited time,
money, and patient participation on hypotheses that have a realistic chance of paying offor on truly radical hypotheses only
when the evidence is strong enough to justify the leap.

Real-World Experiences: Where Prior Probability Shows Up

To make this less abstract, here are a few real-life-style scenarios (the kind you’ll hear in clinics, families, and research meetings)
where prior probability quietly determines whether “evidence-based alternative medicine” is a breakthrough… or just a well-lit mirage.

1) The “My neighbor swears by it” moment

Someone you trust says an herbal supplement “changed their life.” They’re not lying. Their experience is genuine.
But the prior probability question is: how often do people improve for reasons unrelated to the supplement?
Symptoms can naturally fluctuate. Many conditions improve with time. People try new routines at the same time (sleep, diet, stress reduction),
and we tend to credit the newest, most “active” thing. When the prior plausibility is moderate, a personal story might be a useful clue.
When it’s low, a story is more likely to reflect placebo effects, regression to the mean, or coincidence.

2) The “We finally ran a study!” celebration

A small clinic runs a trial and finds a statistically significant benefit. Everyone’s excitedand they should be, at least a little.
But a researcher in the room asks: “How many outcomes did we measure? Were all outcomes reported? Was the sham control credible?”
This isn’t cynicism; it’s quality control. In low-prior settings, small studies are especially vulnerable to false positives.
The experience of many research teams is that early positive results can fade when trials become larger, cleaner, and independently replicated.
That doesn’t mean the team did anything maliciousit often means the first signal was a mix of a tiny effect and lots of noise.

3) The “It can’t hurt” therapy that quietly does

A person with a serious illness tries an alternative therapy “alongside” standard care. At first, it seems harmless.
But then appointments pile up, money disappears, and the patient delays a proven treatment because the alternative plan feels more hopeful
and less scary. This is where prior probability intersects with human psychology: low-plausibility therapies often come wrapped in high-certainty marketing.
The experience clinicians report again and again is not just physical harmthough that can happenbut opportunity cost:
time, energy, and trust diverted away from interventions that have a stronger evidence base.

4) The placebo effectexperienced as real relief

Many people have experienced this: a caring practitioner, a calming ritual, and a sense of being listened to can reduce pain, stress,
and symptom distress. That relief is real. The mistake is leaping from “I feel better” to “the proposed mechanism is true.”
Prior probability helps separate those two. If the mechanism is implausible, a positive experience is more parsimoniously explained by
expectation effects, conditioning, attention, and supportive carepowerful forces that can be ethically incorporated into good medicine
without pretending that physics has been rewritten.

5) The integrative clinic that gets it right

There are also positive experiences. Some integrative programs focus on interventions with plausible mechanisms and growing evidence:
supervised exercise, nutrition counseling based on established science, mindfulness-based stress reduction, sleep interventions,
and physical therapy-style approaches. Patients often report better quality of life, less anxiety, and improved function.
The difference is that these programs tend to avoid grandiose claims and anchor recommendations in biology and reproducible outcomes.
In other words, they behave like healthcaremeasured, transparent, and willing to change when the evidence changes.

Across all these experiences, the lesson is the same: prior probability isn’t an academic luxury.
It’s the hidden steering wheel of interpretation. Ignore it, and you’ll repeatedly mistake noise for signalespecially in fields where
extraordinary claims are common and rigorous replication is rare. Use it, and you can stay open-minded without becoming gullible.

Conclusion: Evidence Doesn’t Count the Same for Every Claim

“Evidence-based alternative medicine” can be a sincere effort to test ideasor it can be a branding strategy that treats any positive study
as a permission slip. Prior probability is the difference between those two worlds. It forces you to ask:
How plausible is this claim before the study? and Is the evidence strong enough to overcome that starting point?

If we want healthcare that’s compassionate and accurate, we need both good evidence and honest priors.
Otherwise, we’re not doing evidence-based medicinewe’re doing “evidence-themed” medicine, which is like a costume party where the placebo effect
is the DJ and the laws of chemistry are stuck outside looking for parking.