What Does 'Evidence-Based' Actually Mean?

a quick primer on the evidence-based paradigm

The term ‘evidence-based’ is thrown around a lot in almost every field these days. We have evidence-based medicine, evidence-based policy, evidence-based education, evidence-based management and even evidence-based strength training. If you ask someone what ‘evidence-based practice’ means you will usually receive an unhelpful answer like “It’s about doing things that are supported by evidence”. Everyone, even those who criticise the evidence-based paradigm, claims to base their actions on the evidence available to them.

There are three components to the evidence-based paradigm that I’ll elucidate upon in this post:

1. An emphasis on trying to find what methods/practices/tools etc. work with a deemphasis on figuring out exactly how and why these things work. (Deprioritise mechanism-based reasoning)

2. Putting less weight on factors like custom, tradition and the anecdotal experiences of individual professionals in the field, e.g. Deemphasising claims like “I should do P because I tried it once and I think it works.” and “I should do P because that’s the done thing among my colleagues.” (Deprioritise anecdotal experience and tradition)

3. A hierarchy of evidence that tells us about the relative differences in strength between results from different kinds of research studies, expert opinion and mechanism-based reasoning. (Evidence Pyramid)

Classical Schools of Medicine

The evidence-based paradigm first emerged in the field of medicine during the 1990s, so I think it’s good to start by looking at the history of different medical paradigms. In ancient Rome, there were many different schools of medical thought. The Greek physician and philosopher Galen wrote about two different schools: the Rationalists (also known as Dogmatists) and the Empiricists.¹

The Rationalists believed that medical theory and mechanism-based reasoning should guide medical practice. They cared intensely about finding the hidden causes of illnesses and their corresponding treatments. The Rationalists were interested in grand theories of disease like Humourism and Empedocles’ view that illness is caused by an imbalance of the four classical elements in one’s body. Medicine, to them, was more like philosophy than a trade.

The Empiricists thought this theorising was a waste of time. They argued that medical advancements emerge from experimentation, not armchair reasoning. The precise nature of the human body is beyond our understanding. What matters is what treatments work, not how or why they work. Empiricist doctors would use the following process to treat an illness:

1. Observe the patient’s symptoms and the results of any treatments. (Rely on personal observation – Autopsia)

2. Collate and read a record of prior observations made by other doctors. (Rely on the observations of others – Historia)

3. If the illness is new or unknown, then prescribe a treatment linked to a known illness most similar to it (Treat by analogy – Epilogismos)

Both the Rationalist and Empiricist schools faced multiple problems. Many of the Rationalist theories, like Humourism, turned out to be wrong. The Rationalists derived bad theories from false premises and allowed these theories to inform their practice. Adherents were resistant to modifying these theories even when presented with contrary evidence. They would occasionally sacrifice effective treatment practices in the name of theoretical consistency.

The Empiricists, meanwhile, had a purely observational approach that placed limits on the development of new preventative medicine. They didn’t have a serious framework for systematically exploring new phenomena or adjudicating between conflicting findings. Making observations yourself and reading about prior observations is not enough – you need a way of synthesising and evaluating these findings. Just looking at ‘what works’ leaves you vulnerable to cognitive and statistical biases like observer bias, placebo effect, post hoc bias and so on. In practice, many Empiricists ended up overrelying on their own personal experiences.

Evidence-Based Medicine

Even two millennia on from the BC era, we don’t have a successful grand theory of medicine. Many diseases and corresponding treatments have poorly understood mechanisms. In the absence of good theory, modern medicine couldn’t adopt the Rationalist paradigm. But we also knew it couldn’t adopt the Empiricist paradigm either. So, instead, evidence-based medicine was developed in the 1990s.

The evidence-based paradigm combines insights about the limits of our mechanistic theories and the limits of pure observation. Evidence-based medicine retains the Empiricist emphasis on observation but supplements it with a more sophisticated understanding of causality, statistical analysis and research methodology. It employs a hierarchy of evidence – with expert opinion and mechanism-based reasoning at the bottom and systematic reviews of randomised controlled trials (RCTs) at the top. The use of mechanism-based reasoning as a starting point gives modern practitioners more to go off of than the Empiricists. This hierarchy reflects increasing confidence that observed outcomes represent meaningful causal relationships rather than mere coincidence.

Evidence-based medicine isn’t perfect. The ‘gold standard’ of randomised controlled trials struggles with complex interventions involving multiple components or requiring contextual adaptation like chronic disease management plans. Randomised controlled trials also can’t practically or ethically be used to evaluate everything – you can’t conduct large RCTs on extremely rare diseases or withhold establishment treatments from patients. Focusing on average treatment effects across populations also means that evidence-based medicine can struggle to deal with the fact that individuals often respond to treatment differently, although practitioners can employ N-of-1 trials (trials with a single patient being allocated an experimental and control treatment in a random order). There are other epistemic problems with evidence-based medicine, but it has the advantage of being better than the alternatives.

The Evidence-Based Paradigm Outside of Medicine

Medicine isn’t the only field lacking good big theories and struggling with the limits of personal observation. Economics, education, public policy and sociology have all embraced the evidence-based paradigm to some degree. We know that these fields suffer from inaccurate and imprecise modelling – economic models, for example, are notorious for relying on unrealistic assumptions. The evidence-based paradigm provides a useful way of figuring out what policy interventions work without overrelying on flawed theories about how they work or deferring to the anecdotal experiences of experts.

Robust empirical work can combat false prevalent beliefs, like the false belief that the minimum wage reduces employment or that ‘Scared Straight’ programs makes youths less likely to commit crimes in the future. But, again, sometimes you can’t practically or ethically conduct RCTs and they don’t give you the information you might want in all cases. The evidence-based paradigm can’t tell you which policy objectives you ought to pursue either.

Anyway, I hope this gives you a better idea of what ‘evidence-based’ means than answers like “It’s about basing your ideas on the available evidence”.

1. He also wrote about a third Methodist school, but for the sake of simplicity I’m going to ignore them.