“Illness is the doctor to whom we pay most heed; to kindness, to knowledge, we make promise only; pain we obey.â€
― Marcel Proust
An essay on ADHD in the New York Times launched an interesting Twitter exchange with Steve Silberman and a medical blogger PalMD on how well we understand psychiatric disorders and treatment.
In the article, Dr. Sroufe concludes that since there is no evidence for longterm use of ADHD medication, their use should be abandoned. He is right that the evidence of efficacy is all short term. Over the long term, no benefit has been shown. Of course almost no one dealing with the issue on a day to day basis would agree. Parents, teachers and physicians all agree that these medications have a use to improve the lives of these children. Count me among those who believe it is highly probable that treatment over the course of months and years has utility, but is hard to prove.
As a problem in decision making, this is a good example of the difference between believing and knowing.
There is a difference between the practice of science and an absolutist approach to truth. In decision making, we must be practical. As Williams James said, “Truth is what works.” He believed that science was a pragmatic search for useful models of the world, including mind. Those that look for abstract, absolute truth in clinical research will be confused, misguided and as often as not, wrong in their decisions. Truth is something that happens to a belief over time as evidence is accumulated, not something that is established by a single positive experiment.
Belief in the usefulness of therapy in medicine follows this model of accumulation of belief. The complexity and variability of human behavior demands a skeptical approach to evidence and a sifting through to discover what works.
Clinical trials for drugs to affect behavior are generally relatively small, short experiments that measure a change from baseline in some clinically meaningful variable. These trials are clinical pharmacology studies in the classic sense, studies in patients (clinical) of drug effect (pharmacology). No one is expecting cure or even modification of the disease. The benefit is short term symptom relief, so the trial examines short term symptom relief. In the case of a pain reliever, we ask whether patient’s self reports of pain are decreased by therapy compared to before therapy. In ADHD, we ask whether a group of target behaviors is changed by treatment compared to baseline.
This approach of measuring change from baseline has a host of pitfalls that limit the generalizability of clinical trials to real life medicine. First, baseline measures are subject to large amounts of bias. One of the worst sources of bias in these trials is the patient and physician’s joint desire to have the patient meet the severity required to be enrolled. The investigator is under pressure to contribute patients to the trial. The patient hopes to gain access to some new therapy, either during the trial or during some subsequent opportunity. Both of these factors pressure patients to maximize the severity of their complaint at baseline. How do you get into a trial? Exaggerate your problem! Even without conscious or unconscious bias from patients, any trial will enroll patients that happen to be worse than their average state. When measured repeatedly over time, the scores will tend to drop- a classic regression to the mean. If you select more severe outliers, they will tend to look more average over time.
Second, diseases are not stable over time. Without any intervention, measures of a disease will be most highly correlated when measured with a short duration between assessments. The longer you wait to measure again, the lower the correlation. Measuring a drug effect in a controlled trial accurately depends on a high level of correlation. All else being equal, the longer one treats, the harder it will be to measure the effect of the drug. This is the major pitfall of depression trials. Episodes are usually limited in duration, so most patients will get better over time without treatment.
So perhaps its not surprising that its very hard to measure the effect of ADHD drugs after months or years in chronic therapy trials. These kids get better over time both from regression to the mean and the natural history of the disease.
Another important issue in ADHD research is that these drugs have effects in healthy volunteers. As Dr. Sroufe points out, amphetamines help college students study for exams- no diagnosis of ADHD needed. This makes it easier to do pharmacology studies, but means that diagnosis in those studies doesn’t really matter- the pharmacology is largely independent of any real pathological state. One could never study a cancer drug in some one without cancer, but this is not true of a cognitive enhancing drug. Its probably most likely that the kids with ADHD don’t have a single pathophysiology, but rather a combination of being at one end of a normal spectrum of behavior plus stress or lack of coping mechanisms that create problems for them in the school environment where those behaviors are disruptive to their learning and that of others. The pharmacology of stimulants helps then all- after all it helps even neurotypic college students and computer programmers.
Treatment response does not confirm diagnosis in ADHD as it does in some other neurological diseases like Parkinson’s Disease. While we’d like to call ADHD a disease or at least abnormal brain state, we have no routine way of assessing the current state of a child’s brain. We have even less ability to predict the state of the brain in the future. Thus diagnosis, in the real meaning of the word- “dia” to separate and “gnosis” to know, is something we can’t do. We don’t know how to separate these kids from normal or into any useful categories. And we have no way of describing prognosis- predicting their course. So a trial that enrolls children on the basis of a behavior at a moment in time and tries to examine the effects of an intervention over the long term is probably doomed to failure. Many of those enrolled won’t need the intervention over time. Many of those who don’t get the intervention will seek other treatment methods over time.
With all of these methodological problems, we can’t accept lack of positive trials to be proof that drugs are ineffective long term. We can’t even prove that powerful opioid pain relievers have longterm efficacy. In fact, it was not too long ago that we struggled with a lack of evidence that opioids were effective even over time periods as short as 12 weeks.
Our short term data in ADHD provides convincing evidence of the symptomatic effects of treatment. Instead of abandoning their use, we should be looking at better ways to collect long term data and test which long term treatment algorithms lead to the best outcomes. And we should be using our powerful tools to look at brain function to understand both the spectrum of ADHD behaviors and the actions of drugs in specific brain regions.
