Al Pacino's Serpico and the Economics of Rotten Apples and the Missing Data Problem.

Almost 20 years ago, Brian Jacob, Steve Levitt and Mark Duggan taught economists how to detect cheating teachers and cheating sumo wrestlers.    Why haven’t urban economists been as successful in identifying corrupt police officers?

The answer must be that the data on their daily performance output are not available in the public domain.  In past research, I have examined the annual compensation for the Police in Baltimore, Boston and New York City.  You can read our report here.

Even with those data, the administrative data does not report the worker’s age or gender.  These data provide no performance information.

Basic economics teaches us that an employer will create a job if the extra revenue a worker generates is greater than the cost of hiring her.

For a non-profit, how do we measure “extra revenue”?  When a city hires one more police officer, how much safer is the city?  This is a very difficult question to answer.

In earlier work, Steve Levitt tried to answer this by estimating cross-city crime “production functions” where he correlates a city’s crime rate with the number of police hired.  Ignoring chicken and egg reverse causality issues (i.e cities with more crime hire more cops), this is a very challenging exercise because does each city have the same “crime production function”?  I don’t think so but researchers have to pool observations in order to simplify the statistical inference problem.

If researchers could collect daily data on public sector worker performance, we could do a much
better job estimating person specific production functions.  The challenge would arise that those
who collect the data would know that an academic would be processing the data and this would
create an incentive to misreport the data to protect the public sector worker.

The key here is the word “public”.  A private sector firm maximizes profit and the residual claimant has strong incentives to identify productive workers and to get rid of bad workers.  In the absence of strong liability incentives, does the public sector have strong accountability incentives?

Economists celebrate the “Big Data” access we have today but note that we do not have access to such public sector data. I am arguing that this is no accident.  How do quantitative researchers advocate for more accountability through more public release of productivity data while protecting worker privacy?

There would be fewer “bad apples” if there are Big Data screening techniques to identify them. Such individuals would not join government and they would act differently in the “sunshine” as the quants identify them and track them.

For those who need an Al Pacino refresher on his movie Serpico click here.