Read the story here. Advertise at Before It's News here.
Profile image
By Environmental and Urban Economics (Reporter)
Contributor profile | More stories
Story Views
Last hour:
Last 24 hours:

The Role of Qualitative Data in Academic Field Experiment Research

% of readers think this story is Fact. Add your two cents.

This will be a “big think” blog post that shares my thinking about this March 2022 Nature Human Behavior paper titled “The data revolution in social science needs qualitative research”.

Permit me to focus on one example.  Consider a sample of 5,000 equally talented and ambitious 18 year olds.  Each has graduated from High School and each is considering applying to the University of California.  The students differ that some are Asian and some are Hispanic.  To simplify, let’s assume everyone is a member of one of these two groups.  

The researcher observes that 72% of Asians 18 year olds in the sample apply to the University of California while only 32% of Hispanic 18 year olds in the sample do.

A qualitative researcher would take the next step of interviewing a random subset of the Asians and the Hispanics in this sample to ask them various questions about their beliefs, life goals, family circumstances and several other nuances that cannot be captured by a standard demographic survey.

A field experiment researcher would proceed with a different strategy.  She would take a new sample of 5,000 equally talented and ambitious 18 year old Hispanic and Asian students in the next year and randomly assign a subset of each group to a specific treatment such as a 20 minute information course on understanding the application process to the University of California and the gains to attending an elite 4 year college.  The control group would not receive any of this information

The field experiment researcher would observe the following pieces of information;

X1 =  percentage of Asians who apply to the University of California given random assignment to the treatment group.

X2 =  percentage of Asians who apply to the University of California given random assignment to the control group.

X3 =  percentage of Hispanics who apply to the University of California given random assignment to the treatment group.

X4 =  percentage of Hispanics who apply to the University of California given random assignment to the control group.

Suppose for this new sample of young people that the researcher observes the following facts;

X1 =  74%

X2 =  72%

X3 = 70%

X4 =  32%

These data immediately indicate that the intervention had a much larger impact on the Hispanic teens than on the Asian teens.  The intervention closed the gap.

What is My Point?

I am interested in asymmetries in social science.  Economists assume that people know themselves and have a life plan and a “conception of the good life” and they strategically make their choices such as who they marry, where they live and how much education to attain based on their plan.  The observer knows that she does not know what is each person’s goal. Social scientists learn about people based on the choices we observe them make.

In the example sketched out above, a field experiment researcher seeks to explain the “education gap” and to test for what might be cost-effective strategies for closing this gap.  How did the field experiment researcher choose the specific intervention that turned out to be effective?

One answer is introspection. Another answer is journalistic accounts explaining why some talented teens do not apply to great schools.  Another answer is qualitative research on a larger sample of people.

The question here pertains to how quickly will the Big Data researcher zero in on the right treatment to pilot?   If the Big Data field experiment researcher is baffled concerning what treatment to pilot, then qualitative research is crucial for narrowing down the set of strategies.  

Given the publication incentives of academic researchers and given their finite research budgets, they have strong incentives to pursue treatment effect designs that are effective. Researchers cannot publish papers that say; “I tried this crazed treatment and it turned out to have no effect.”.   The self doubt of the field experiment researcher leads her to pursue qualitative strategies (at least on a small sample) to reduce her risk exposure of investing her time and $ in a project that doesn’t yield credible statistically significant results.

“Dynamic” Facts in the Social Science

Note that the effective intervention means that the “Old fact” (that eligible Asians are much more likely to apply to the great colleges than eligible Hispanics) is no longer a fact going forward as the effective intervention brings about convergence.   In Physics, this does not occur.   

Final Point

When the field experiment researcher knows that she does not know the causes of behavior (so in this case why Hispanics are not often applying to the UCs), she has an incentive to invest in qualitative research.


Before It’s News® is a community of individuals who report on what’s going on around them, from all around the world.

Anyone can join.
Anyone can contribute.
Anyone can become informed about their world.

"United We Stand" Click Here To Create Your Personal Citizen Journalist Account Today, Be Sure To Invite Your Friends.

Please Help Support BeforeitsNews by trying our Natural Health Products below!

Order by Phone at 888-809-8385 or online at M - F 9am to 5pm EST

Order by Phone at 866-388-7003 or online at M - F 9am to 5pm EST

Order by Phone at 866-388-7003 or online at M - F 9am to 5pm EST

Humic & Fulvic Trace Minerals Complex - Nature's most important supplement! Vivid Dreams again!

HNEX HydroNano EXtracellular Water - Improve immune system health and reduce inflammation

Ultimate Clinical Potency Curcumin - Natural pain relief, reduce inflammation and so much more.

MitoCopper - Bioavailable Copper destroys pathogens and gives you more energy. (See Blood Video)
Oxy Powder - Natural Colon Cleanser!  Cleans out toxic buildup with oxygen! 
Nascent Iodine - Promotes detoxification, mental focus and thyroid health.
Smart Meter Cover -  Reduces Smart Meter radiation by 96%!  (See Video)

Report abuse


    Your Comments
    Question   Razz  Sad   Evil  Exclaim  Smile  Redface  Biggrin  Surprised  Eek   Confused   Cool  LOL   Mad   Twisted  Rolleyes   Wink  Idea  Arrow  Neutral  Cry   Mr. Green

    Load more ...




    Email this story
    Email this story

    If you really want to ban this commenter, please write down the reason:

    If you really want to disable all recommended stories, click on OK button. After that, you will be redirect to your options page.