Déjà vu all over again, part deux: Yet another Internet survey on vaccinations [Respectful Insolence]

One of the best things about blogging is that I don’t feel obligated to cover a topic completely in one post because I know I can always write another one or revisit the topic later. It also allows me to look at what I like to call “variations on a theme” of various kinds of quackery (or anything else, for that matter). View this as a post looking at one such variation on a theme.

The theme this time is the tendency of antivaccine activists to demonstrate their utter cluelessness when it comes to designing clinical studies. This cluelessness virtually always manifests itself in the frequent call by antivaccine quacks for what they seem to view as the Holy Grail of evidence to “prove” that vaccines cause autism, other neurodevelopmental disorders, sudden infant death syndrome, asthma, diabetes, autoimmune diseases, and just about any other chronic condition you can think of, namely the so-called “vaxed/unvaxed” study. The reason they seek such a study is because it is an article of faith among the antivaccine movement that vaccines are degrading the health of our children, making them the “sickest generation ever” and causing all the aforementioned conditions.

As I mentioned the last time I discussed this, years ago (and sometimes even now) antivaccinations would call for an actual randomized, double-blind, placebo-controlled “vaxed versus unvaxed” clinical trial. Of course, as I mention every time this topic comes up, antivaccine activists appear to be completely ignorant of the concept of clinical equipoise, which renders such a trial completely unethical and therefore undoable, at least in developed countries with actual laws protecting the welfare of clinical trial subjects, particularly children. Remember: Clinical equipoise means that there must be genuine uncertainty over which group in a clinical trial is receiving the superior treatment, and there is no doubt that leaving one group of children unvaccinated leaves them susceptible to vaccine-preventable diseases and thus does them harm. That’s why doing such a trial would be completely unethical and we have to rely on the accumulated knowledge from epidemiological studies to determine whether vaccines are causing adverse outcomes.

Some antivaccine activists have realized that there’s no way a randomized clinical “vaxed vs. unvaxed” trial will ever be done. Grudgingly (and I do mean grudgingly), they accept that virtually every other physician does; so they try to find evidence of vaccine harm that they so desperately seek in other ways. Most recently, it was a survey whose purpose was so incredibly obviously antivaccine that it was even more risibly pathetic than previous efforts, such as J.B. Handley’s vaxed versus unvaxed phone survey and a hilariously incompetent survey by a German homeopath named Andreas Bachmair for VaccineInjury.info, both of which were presented as showing that unvaccinated children are healthier than vaccinated children and both of which showed nothing of the sort. This time around, it’s a “study” being promoted by the surprisingly honestly named Vaccine Resistance Movement (VRM) called the VRM Autism Study. Not surprisingly, this new antivaccine “study” follows the long, dishonorable tradition of horribly designed and outright incompetent “studies” done by antivaccinationists since time immemorial (at least to me).

I’ll show you what I mean.

Whenever a clinical trial is designed there are several important things that need to be prespecified. To put it simply, you need:

• Hypothesis: What is the question being asked?
• Inclusion criteria: What subjects will be included and why.
• Exclusion criteria: Which subjects will be excluded and why.
• Primary endpoints: What is/are the most important specific endpoint(s) you are comparing between the control group and the experimental group?
• Secondary endpoints (optional): Additional things you want to compare.
• Protocol: What specific treatments and procedures will subjects undergo? In the case of observational studies: What specific observations will be made on each group?
• Sample size: How many subjects will you need in each group to have a reasonable chance of producing a statistically significant result?

Now, to be fair in the case of pilot observational studies, the need for a specific hypothesis can be somewhat less critical, as it can be quite possible to do such studies as a “hypothesis-generating” study. Even so, you have to have some. So does the VRM study specify these key variables? What do you think?

Let’s take a look at the VRM announcement of its study on its Facebook page:

The purpose of this worldwide study is to determine the incidence/rate of autism amongst those unvaccinated children & adults surveyed. We are also concurrently tracking the incidence/rate of autism amongst those vaccinated children & adults surveyed; cross-referencing dietary factors, pre-existing medical conditions/allergies, family health/vaccine history (multiple generations), breast/bottle feeding, We hope to gain new insights into the causality & manifestations of autism with an exhaustive, scientific approach.

“Scientific approach.” You keep using that term. I do not think it means what you think it means.

Elsewhere, VRM declares:

It is our goal, through this study, to determine an accurate percentage of those unvaccinated autistic children & adults vs. those unvaccinated children & adults who have not become autistic. Essentially what we are trying to identify is the healthy trend in unvaccinated children & adults.

OK, fair enough—on first glance. The devil, as always, is in the details. That’s something that’s true of any sort of science. How does VRM plan to measure these percentages? Well, that’s where the big problem is. As was the case with all the previous studies I’ve discussed, apparently VRM thinks that an Internet survey is a valid way to collect data that can be used to up with accurate estimates of the prevalence of autism:

Just to clarify this study is open to EVERYONE. The absence of autism in your family/household or the fact you haven’t vaccinated your children does not preclude you from taking part in the study. The control parameters are wide open this time. All the data we receive will be pertinent in determining the paths that lead to autism. VRM will be continuing to collect your invaluable data for as long as possible. That being said we still encourage everyone to enroll in the study while it’s openly available.

And:

In order to guarantee accuracy of the data acquired study participants must allow for access to family medical records (*Note: only in cases where autism has occurred). This disclosure will be entirely confidential in nature and at the discretion of the study organizers. Any shared information will be used strictly to confirm an autism diagnosis. No data beyond that which the family agrees to disclose shall be included in the study results.

All participants in this study will be designated a number to assure complete anonymity. Any sensitive information including family names, personal references, e-mail addresses etc will be inaccessible to the public. Only study administrators will be privy to access. Contact will remain exclusively confidential between participants & VRM. At any time a participant may withdraw from the study group; whereupon their data will be returned to them and/or destroyed.

Out of curiosity, I took a look at the survey that VRM was using. As you can see if you take a look yourself, this “study” (such as it is), it’s yet another example of bad experimental design coupled with bad survey design. Take a look:

Category of Participant (choose one selection):

• Received standard vaccinations (full schedule of shots) between 0-7 years; also officially diagnosed with Autism Spectrum Disorder
• Received standard vaccinations (full schedule of shots) between 0-7 years; no official diagnosis of Autism Spectrum Disorder
• Received standard vaccinations (partial schedule) between 0-7 years; also officially diagnosed with Autism Spectrum Disorder
• Received standard vaccinations (partial schedule of shots) between 0-7 years; no official diagnosis of Autism Spectrum Disorder
• Received standard vaccinations (full schedule of shots); also officially diagnosed with Autism Spectrum Disorder
• Received standard vaccinations (full schedule of shots); no official diagnosis of Autism Spectrum Disorder occurring
• Received standard vaccinations (partial schedule of shots); also officially diagnosed with Autism Spectrum Disorder
• Received standard vaccinations (partial schedule of shots): no official diagnosis of Autism Spectrum Disorder occurring
• Received standard vaccinations (full schedule) between 0-7 years; also suspect/unofficially diagnosed with Autism Spectrum Disorder
• Received standard vaccinations (partial) between 0-7 years; also suspect/unofficially diagnosed with Autism Spectrum Disorder
• Received standard vaccinations (full schedule of shots); also suspect/unofficially diagnosed with Autism Spectrum Disorder
• Received no vaccinations whatsoever between 0-7 years; also officially diagnosed with Autism Spectrum Disorder
• Received no vaccinations whatsoever between 0-7 years; no official diagnosis of Autism Spectrum Disorder occurring
• Received no vaccinations whatsoever between 0-7 years; also suspect/unofficially diagnosed with Autism Spectrum Disorder
• Received no vaccinations whatsoever; also officially diagnosed with Autism Spectrum Disorder
• Received no vaccinations whatsoever; no official diagnosis of Autism Spectrum Disorder occurring
• Received no vaccinations whatsoever; also suspect/unofficially diagnosed with Autism Spectrum Disorder

In clinical trials and observational studies, it is absolutely critical to define one’s terms. What does VRM mean by “partial schedule of shots”? It’s not defined. Then, later on, the survey lists the individual vaccines and when the person received them. Of course, anyone should be able to see another problem here. This survey includes “ALL AGES.” The vaccine schedule has changed significantly over the years, as new vaccines were added and the recommended timing was changed over the years. So what it means to receive a “partial schedule of shots” will be very different depending upon how hold the subject is.

That’s not all (of course). Later in the survey, the subjects are asked for “pre-existing medical conditions/genetic predisposition.” The list includes a veritable laundry list of conditions that antivaccinationists consider to be “predisposing” to autism, including mercury amalgams, “medicated childbirth” (e.g., induced labor), antibiotics, and more. Elsewhere, under dietary history, the subjects are asked, for example, if they:

• dairy-rich diet
• use margarine regularly
• use Crisco regularly
• eat junk food regularly
• eat fast food regularly
• avoid junk food
• avoid fast food
• use corn syrup regularly
• use fructose regularly
• avoid fructose
• meat-eater
• vegan
• vegetarian
• semi-vegetarian
• eat strictly organic food
• eat non-organic food
• eat soy products
• regular alcohol user
• recreational drug user
• heavy drug user
• regular caffeine user
• drink pop regularly
• consume Aspartame containing products
• avoid Aspartame containing products
• drink from plastic containers
• avoid using microwave in cooking
• avoid Genetically Modified Foods/GMO
• do not avoid Genetically Modified Foods
• Sodium Fluoride in toothpaste/drinking water
• frequent cell phone user
• work in front of computer

There are more—oh, so many more—examples listed. I simply chose some of the sillier ones. Let’s just put it this way. The survey is so vague as to be meaningless. After all, most people don’t even know what is and isn’t a “genetically modified food,” and there’s no evidence that GMOs are associated with autism.

As with all these antivaccine vaccinated versus unvaccinated”studies,” the key problem (besides the undefined hypothesis and endpoints) is that the subjects are self-selected. More than that, they’re self-selected from people who tend to be antivaccine and learn about such surveys through antivaccine websites and Facebook pages. Add to that the multiple comparisons, and the “researchers” (and I do use that term very loosely) are virtually guaranteed to find all sorts of spurious associations. They’ll be meaningless, of course, because the methodology is so poor, so utterly lacking in anything resembling valid methods. None of this stops antivaccine groups, though. They forge boldly ahead into the depths of pseudoscience in search of “evidence,” no matter how risibly bad, that vaccines cause autism.

Seeing such perversions of science, I sometimes wonder if scientists should just do a large epidemiological study of vaccinated versus unvaccinated children, if only to shut antivaccinationists up and shut down studies like this, even though such a study would be expensive and require a lot of subjects. (Also, there are no compelling preliminary data to justify such a study.) Then I remember how antivaccine activists react to negative studies. They ignore them or attack them as fatally flawed, usually due to “bias” or—of course!—the influence of big pharma. Scientifically pandering to antivaccine loons achieves exactly nothing. Unfortunately, that means we’re likely to continue to see such epically silly self-selected Internet polls masquerading as research.

And don’t even get me started on where the Institutional Review Board approval is for this VRM “study.”