More atrocious antivaccine science promoted by Robert F. Kennedy, Jr [Respectful Insolence]

It always amuses me how antivaccine activists have such a love-hate relationship with academia, particularly academia at the higher echelons. On the one hand, they routinely denigrate academics because inevitably well-designed, well-executed epidemiological studies testing the hypothesis that vaccines are correlated with the risk of autism always come up empty. That’s because vaccines don’t cause autism. I used to hedge a bit when I said that, but over the 12 years I’ve been doing this, I’ve covered more studies than I can remember testing this very hypothesis, and a clear pattern has emerged. The best studies were entirely negative, and only crap studies by “scientists” associated with the antivaccine movement showed anything resembling a positive result. So antivaxers detest, disparage, and otherwise denigrate medical academia because it doesn’t support their delusion that vaccines are harmful and cause autism.

On the other hand, antivaxers are desperate for validation. They crave any evidence that real scientists take them seriously or, even better, have produced evidence that supports their delusion that vaccines cause autism. That’s a good explanation for an article by President Donald Trump’s new BFF, antivaccine activist Robert F. Kennedy, Jr., Yale University Study Shows Association Between Vaccines and Brain Disorders. RFK Jr. is practically giddy with validation:

A team of researchers from the Yale School of Medicine and Penn State College of Medicine have found a disturbing association between the timing of vaccines and the onset of certain brain disorders in a subset of children.

Analyzing five years’ worth of private health insurance data on children ages 6-15, these scientists found that young people vaccinated in the previous three to 12 months were significantly more likely to be diagnosed with certain neuropsychiatric disorders than their non-vaccinated counterparts.

This new study, which raises important questions about whether over-vaccination may be triggering immune and neurological damage in a subset of vulnerable children (something parents of children with autism have been saying for years), was published in the peer-reviewed journal Frontiers in Psychiatry, Jan. 19.

Hmmm. Frontiers in Psychiatry? I’ve encountered Frontiers journals before. Suffice to say, I have not been impressed. Other Frontiers journals, for instance, have shown an unfortunate susceptibility to antivaccine pseudoscience. Be that as it may, my skeptical antennae always start to twitch whenever I see someone like RFK Jr. exulting over a study. Let’s just say it’s a long history of seeing the sort of execrable “science” he routinely embraces, as long as it supports his belief that mercury in vaccines causes autism, even though the mercury-containing thimerosal preservative that contains mercury was removed from childhood vaccines 15 years ago.

Before I examine the study itself, which, not surprisingly, is nowhere near as convincing as RFK Jr. portrays it, let’s see what RFK Jr. thinks of it:

More than 95,000 children in the database that were analyzed had one of seven neuropsychiatric disorders: anorexia nervosa, anxiety disorder, attention deficit and hyperactivity disorder (ADHD), bipolar disorder, major depression, obsessive-compulsive disorder (OCD) and tic disorder.

Children with these disorders were compared to children without neuropsychiatric disorders, as well as to children with two other conditions that could not possibly be related to vaccination: open wounds and broken bones.

This was a well-designed, tightly controlled study. Control subjects without brain disorders were matched with the subjects by age, geographic location and gender.

As expected, broken bones and open wounds showed no significant association with vaccinations.

New cases of major depression, bipolar disorder or ADHD also showed no significant association with vaccinations.

I could tell from RFK Jr.’s description that there was very likely a confounder that could explain the results of this study. In actuality, there are a number of issues with the study that make it far much less of a slam dunk than RFK Jr. and other antivaccinationists seem to think that it is. If you don’t believe me, let’s head on over to the original study by Leslie et al, Temporal Association of Certain Neuropsychiatric Disorders Following Vaccination of Children and Adolescents: A Pilot Case–Control Study, and take a look. It’s open access; so you can all read as much (or as little) of the study as you like.

My first question, before I even started to read the paper, was: Why was this study necessary? The answer is quite simple. It wasn’t. There’s already copious evidence that vaccines are not associated with autism or other neurodevelopmental disorders. For instance, a large and far study in 2007 quite emphatically did not support a causal relationship between vaccines and neurodevelopmental disorders other than autism, while the followup study to that in 2010 just as emphatically did not support a potential causal relationship between vaccines and autism. Both studies were far better designed than this one. So how do the authors justify doing yet another study to study what’s been studied ad nauseam with negative results? I’m going to use a longer quote than usual because it’s important:

In light of the role of the immune system in these central nervous system (CNS) conditions, the impact of vaccines on childhood-onset neuropsychiatric diseases had been considered and was mainly addressed with regards to the administration of the measles, mumps, and rubella (MMR) vaccine (and its various components) and the subsequent development of autism spectrum disorder (ASD). Although the controversy over MMR vaccination and ASD still exists for some members of the public, this association has been convincingly disproven (9, 10). On the other hand, the onset of a limited number of autoimmune and inflammatory disorders affecting the CNS has been found to be temporally associated with the antecedent administration of various vaccines (11). These disorders include idiopathic thrombocytopenic purpura, acute disseminated encephalomyelitis, and Guillain–Barré syndrome among others (12–16). More recently, data have emerged indicating an association between the administration of the H1N1 influenza vaccine containing the AS03 adjuvant and the subsequent new onset of narcolepsy in several northern European countries (17, 18). The immune mechanisms and host factors underlying these associations have not been identified or fully characterized, although preliminary data are beginning to emerge (18–23).

Given this growing body of evidence of immunological involvement in CNS conditions, and despite the controversy concerning the link between ASD and MMR and the clear public health importance of vaccinations, we hypothesized that some vaccines could have an impact in a subset of susceptible individuals and aimed to investigate whether there is a temporal association between the antecedent administration of vaccines and the onset of several neuropsychiatric disorders, including OCD, AN, tic disorder, anxiety disorder, ADHD, major depressive disorder, and bipolar disorder using a case–control population-based pediatric sample (children aged 6–15 years). To assess the specificity of any statistical associations, we also determined whether or not there were any temporal associations between antecedent vaccine administration and the occurrence of broken bones or open wounds.

One can’t help but note that the disorders listed that occur in the CNS after vaccines are actually quite rare, particularly Guillain-Barré. The authors of the article referenced found only 71 cases between 1979 and 2013. That’s 71 cases out of billions of doses of vaccines administered over 34 years. Remember, what this paper is claiming to look at is not serious demyelinating reactions to vaccines, which are very rare, but the reaction between vaccination and common conditions, like compulsive disorder (OCD), anorexia nervosa (AN), anxiety disorder, chronic tic disorder, attention deficit hyperactivity disorder, major depressive disorder, and bipolar disorder. One notes that the authors didn’t look at autism, and they didn’t really explain why. One also notes that antivaxers flogging this paper are annoyed that the authors didn’t look at autism, even though the idea that vaccines cause autism is the central myth of the antivaccine movement.

As for using the association of the H1N1 influenza vaccine with narcolepsy as a justification, it’s important to note that this is a strange case. The association was only observed in specific countries and not in others in which the vaccine does not appear to be a consistent or unique risk factor for narcolepsy in these populations. Overall, it was a confusing set of data to derive any clear picture of whether the H1N1 vaccine was a true risk factor. On the other hand, there are data suggesting that Pandemrix triggers antibodies that can also bind to a receptor in brain cells that help regulate sleepiness in genetically susceptible people. Be that as it may, the result with narcolepsy is nonetheless thin gruel to justify a study like this.

But what about the study itself? Basically, it’s a case control study. As you recall, a case control study is a form of epidemiological study that looks at risk factors in those who are diagnosed with a condition (cases) compared to those who are not (controls). For instance, a case control study might find that people with lung cancer (cases) are far more likely to have a significant smoking history than those without (controls). One thing about a case control study is that the selection of controls is critical, as it is impossible to completely randomize. The controls must be chosen to be as similar as possible to the cases in everything other than the condition being examined. This is not as easy as it sounds.

This particular case control study used data from 2002 to 2007 from the MarketScan® Commercial Claims and Encounters database, which is constructed and maintained by Truven Health Analytics. MarketScan consists of de-identified reimbursed health-care claims for employees, retirees, and their dependents of over 250 medium and large employers and health plans. Individuals included in the database are covered under private insurance plans; no Medicaid or Medicare data are included. The database includes claims information describing the health-care experiences for approximately 56 million covered patients per year and is divided into subsections, including inpatient claims, outpatient claims, outpatient prescription drug claims, and enrollment information. Claims data in each of the subsections contain a unique patient identifier and information on patient age, gender, geographic location (including state and three-digit zip code), and type of health plan.

You can see one thing right away. This is a select population, only patients with private insurance belonging to these plans. Another issue is that this is what we in the biz call administrative data. Administrative data are data collected not for research purposes, but for administrative purposes, primarily registration, transaction and record keeping, usually during the delivery of a service. In this case, the authors used a health insurance administrative datbase. That means it’s just diagnoses, procedures and interventions, some demographic data, and billing information. On the one hand, administrative data allow for huge numbers, is unobtrusive given that these are data that are collected anyway, and can uncover information that a study subject might not provide in an interview. However, there are disadvantages, too, and they’re not small. One very common drawback to using administrative data is that a lot of potentially relevant clinical and demographic data aren’t captured. In other words, the data are restricted to just the data needed to administrative purposes, and therefore the amount of data and the definitions of conditions are often insufficiently granular. There are lots of problems using administrative data. For example, the use of administrative data can frequently provide an incorrect estimate for various conditions and risk factors, as has been described for sickle cell disease, where administrative data grossly underestimated the rate of transfusion. (Discussions of the advantages and disadvantages of using administrative data can be found here and here.)

Now here’s how the authors did the study:

The study sample consisted of children aged 6–15 with a diagnosis of one of the following conditions (ICD-9 codes in parentheses): OCD (300.3), AN (307.1), anxiety disorder (300.0–300.2), tic disorder (307.20 or 307.22), ADHD (314), major depression (296.2–296.3), and bipolar disorder (296.0–296.2, 296.4–296.8). To test the specificity of the models, we also included children with broken bones (800–829) and open wounds (870–897). To identify new cases, we further limited the sample in each diagnostic group to children who were continuously enrolled for at least 1 year prior to their first diagnosis for the condition (the index date). Next, a matched one-to-one control group was constructed for each diagnostic group consisting of children who did not have the condition of interest and were matched with their corresponding case on age, gender, date of the start of continuous enrollment, and three-digit zip code. Because vaccines tend to occur during certain times of year (such as before summer camps or the beginning of school), controls were also required to have an outpatient visit at which they did not receive a vaccine within 15 days of the date that the corresponding case was first diagnosed with the condition, in an effort to control for seasonality. The date of this visit was the index date for children in the control group.

Can you see some problems already? First, in a case control study it is often desirable to use more controls than cases; that wasn’t done here. That isn’t a horrible flaw, just one that I question given that one of the key advantages of an administrative database is large numbers of subjects. More important is how few descriptors were used to match the controls: age, gender, date of insurance, and zip code. I’ll be honest and say that I’m not sure if the way they tried to control for seasonality of vaccine administration is valid or not. I will for now assume it was, as that’s not necessary for my conclusion that this paper is pretty crappy.

Now let’s look at how the analysis was done:

The analyses were performed for each diagnostic group (and their controls) separately. Children with multiple conditions (e.g., ADHD and tic disorder) were included in each of the corresponding analytic groups. First, the proportion of children who were exposed to vaccines in the period before the index date was compared across the case and control groups. Next, bivariate conditional logistic regression models were estimated to determine the hazard ratios (HRs) and 95% confidence intervals (95% CIs) associated with the effect of vaccine exposure on having the condition of interest. Separate models were run for the 3-, 6-, and 12-month periods preceding the index date for each diagnostic group.

This leads to the results as reported:

Subjects with newly diagnosed AN were more likely than controls to have had any vaccination in the previous 3 months [hazard ratio (HR) 1.80, 95% confidence interval 1.21–2.68]. Influenza vaccinations during the prior 3, 6, and 12 months were also associated with incident diagnoses of AN, OCD, and an anxiety disorder. Several other associations were also significant with HRs greater than 1.40 (hepatitis A with OCD and AN; hepatitis B with AN; and meningitis with AN and chronic tic disorder).

So the authors found some associations. Whoopee. RFK Jr. characterizes this paper as a ” well-designed, tightly controlled study.” No it wasn’t. Not at all. RFK Jr. wouldn’t know a well-designed study if it bit him in the posterior. The only reason he thinks this study is “well designed” and “tightly controlled” is because it provides results that he likes. In fact, what the authors did is the same thing authors of a recent acupuncture study I noted did: p-hacking. They did a whole bunch of comparisons using a nominal p<0.05 and don’t correct for multiple comparisons. In other words, it’s almost certainly statistical noise, given that most of the associations are modest and that the associations are all over the place.

Check out Table 2. It is the very definition of a p-hacking table. All the bold results are “statistically significant.” Just peruse the table. You don’t even have to look that closely to see that the receipt of any vaccine within 3, 6, and 12 months is correlated with almost every condition examined, including broken bones and open wounds. (Yes, if you accept this study’s results you have to conclude just as much that vaccines are a risk factor for broken bones! A modest one, true, but a risk factor nonetheless. Yes, that’s sarcasm.) You can look down the list at individual vaccines and see that the influenza vaccine is associated with almost as many conditions. In fact, one thing the authors seem not to mention is that administration of any vaccine seems to modestly decrease the risk of major depression and bipolar disorder!

What we’re looking at is almost certainly nothing more than statistical noise.

But wait, there’s more. Let’s take a look at the funding sources:

This research was funded by donations from RK, BR, and Linda Richmand.

Now this is bizarre. Two of the authors (Robert Kobre and Brian Richmand) self-funded the work. Linda Richmand is almost certainly Brian Richmand’s wife. This is all very strange until you realize that Brian Richmand is a lawyer, not an epidemiologist or health researcher. And guess what. Surprise! Surprise! He’s authored at least one paper promoting the vaccine-autism link! Guess what journal it appeared in? I even wrote about his awful paper when it came out! Like, wow, man. Maybe Deepak Chopra is right, and the universe really is interconnected. Whoa. It gets worse. though. Robert Kobre is not a scientist either. He is the Managing Director at Credit Suisse Securities (USA) LLC and is also chairman of the board of directors of the Global Lyme Alliance, which from its website appears to be very much into chronic lyme disease woo.

So how is it that a lawyer from Stanford and an investment banker at Credit Suisse are listed as being affiliated with the Yale Child Study Center when neither of their names appear in an online list of faculty there? Inquiring minds want to know. Yes, precious, they do.

There are so many dodgy things about this paper that I could continue to go on, but for purposes of a wrap-up, what you need to know is that, no, it doesn’t show that vaccines cause anorexia nervosa or tics, or the other neurological disorders linked to them; that it’s funded by two of the authors and the wife of one of the authors; that one of the authors has a history of writing antivaccine articles for Medical Hypotheses; and, finally, that the other author is chairman of the board of directors for a lyme disease charity that appears to be heavily into chronic lyme disease woo. Bad science and undisclosed conflicts of interest, plus authors claiming the mantle of a university they don’t belong to! When I decided to look at this paper, I hit the jackpot in terms of—shall we say?—teaching opportunities in critical thinking. Thanks, RFK, Jr.!