This will be a “big think” blog post that shares my thinking about this March 2022 Nature Human Behavior paper titled “The data revolution in social science needs qualitative research”.
Permit me to focus on one example. Consider a sample of 5,000 equally talented and ambitious 18 year olds. Each has graduated from High School and each is considering applying to the University of California. The students differ that some are Asian and some are Hispanic. To simplify, let’s assume everyone is a member of one of these two groups.
The researcher observes that 72% of Asians 18 year olds in the sample apply to the University of California while only 32% of Hispanic 18 year olds in the sample do.
A qualitative researcher would take the next step of interviewing a random subset of the Asians and the Hispanics in this sample to ask them various questions about their beliefs, life goals, family circumstances and several other nuances that cannot be captured by a standard demographic survey.
A field experiment researcher would proceed with a different strategy. She would take a new sample of 5,000 equally talented and ambitious 18 year old Hispanic and Asian students in the next year and randomly assign a subset of each group to a specific treatment such as a 20 minute information course on understanding the application process to the University of California and the gains to attending an elite 4 year college. The control group would not receive any of this information
The field experiment researcher would observe the following pieces of information;
X1 = percentage of Asians who apply to the University of California given random assignment to the treatment group.
X2 = percentage of Asians who apply to the University of California given random assignment to the control group.
X3 = percentage of Hispanics who apply to the University of California given random assignment to the treatment group.
X4 = percentage of Hispanics who apply to the University of California given random assignment to the control group.
Suppose for this new sample of young people that the researcher observes the following facts;
X1 = 74%
X2 = 72%
X3 = 70%
X4 = 32%
These data immediately indicate that the intervention had a much larger impact on the Hispanic teens than on the Asian teens. The intervention closed the gap.
What is My Point?
I am interested in asymmetries in social science. Economists assume that people know themselves and have a life plan and a “conception of the good life” and they strategically make their choices such as who they marry, where they live and how much education to attain based on their plan. The observer knows that she does not know what is each person’s goal. Social scientists learn about people based on the choices we observe them make.
In the example sketched out above, a field experiment researcher seeks to explain the “education gap” and to test for what might be cost-effective strategies for closing this gap. How did the field experiment researcher choose the specific intervention that turned out to be effective?
One answer is introspection. Another answer is journalistic accounts explaining why some talented teens do not apply to great schools. Another answer is qualitative research on a larger sample of people.
The question here pertains to how quickly will the Big Data researcher zero in on the right treatment to pilot? If the Big Data field experiment researcher is baffled concerning what treatment to pilot, then qualitative research is crucial for narrowing down the set of strategies.
Given the publication incentives of academic researchers and given their finite research budgets, they have strong incentives to pursue treatment effect designs that are effective. Researchers cannot publish papers that say; “I tried this crazed treatment and it turned out to have no effect.”. The self doubt of the field experiment researcher leads her to pursue qualitative strategies (at least on a small sample) to reduce her risk exposure of investing her time and $ in a project that doesn’t yield credible statistically significant results.
“Dynamic” Facts in the Social Science
Note that the effective intervention means that the “Old fact” (that eligible Asians are much more likely to apply to the great colleges than eligible Hispanics) is no longer a fact going forward as the effective intervention brings about convergence. In Physics, this does not occur.
When the field experiment researcher knows that she does not know the causes of behavior (so in this case why Hispanics are not often applying to the UCs), she has an incentive to invest in qualitative research.
Please Help Support BeforeitsNews by trying our Natural Health Products below!
Order by Phone at 888-809-8385 or online at https://mitocopper.com M - F 9am to 5pm EST
Order by Phone at 866-388-7003 or online at https://www.herbanomic.com M - F 9am to 5pm EST
Order by Phone at 866-388-7003 or online at https://www.herbanomics.com M - F 9am to 5pm EST
Humic & Fulvic Trace Minerals Complex - Nature's most important supplement! Vivid Dreams again!
HNEX HydroNano EXtracellular Water - Improve immune system health and reduce inflammation
Ultimate Clinical Potency Curcumin - Natural pain relief, reduce inflammation and so much more.
pathogens and gives you more
energy. (See Blood Video)
Oxy Powder - Natural Colon Cleanser! Cleans out toxic buildup with oxygen!
Nascent Iodine - Promotes detoxification, mental focus and thyroid health.
Smart Meter Cover - Reduces Smart Meter radiation by 96%! (See Video)