AidData: Why it is not Wikipedia

% of readers think this story is Fact. Add your two cents.

The debate about AidData’s “crowd-sourcing” methodology is heating up (see comments on my post from April 30). A recent note on Development Gateway in support of AidData compared AidData’s methodology to Wikipedia, saying that this kind of crowd-sourcing of information can be productive.

I’m actually a big fan of Wikipedia. It is open, transparent, and often very clear and helpful. But I am not using it as a database for cross-country regressions. Many Wikipedia articles are “stubs” or have notes that controversy exists. We can take that into consideration when we read the article. That’s also possible if someone is “reading” AidData’s dataset. But let’s face it, the purpose of having this data is to test relationships. I would be very surprised if users are going to “read” the data and pick and choose which numbers to include.

As Philippa Brant pointed out at the Lowy Institute, nearly half of the cases in the dataset — 47 percent — have only one source. That’s almost 800 “projects” (1700 * .47). That level of uncertainty is not good enough for publication. AidData should have done more cleaning themselves before publishing this data.

AidData has responded to my critique in part by pointing out that they realize that many deals are not confirmed. That’s not how it looks to me. The table we are discussing has a column on status. Of the nearly 1700 projects, 687 have a status of “Pipeline: Commitment” or “Pipeline: Pledge”. None were coded “Pipeline: Vague” or “Suspended”.  As many others have noted, an African ministry official’s press conference about a popular public works project does not mean a commitment or pledge has occurred. Those of us who work on China and Africa have seen this multiple, multiple times. (Note: it is very rarely a Chinese announcement of a project commitment: when the Chinese government announces that it is financing a deal, a project is much more likely — although not always — goes ahead).

Finally, although AidData authors stress in their comments that their results are tentative, that they can’t vouch for much of the data, they didn’t write like that when discussing the numbers. We read about the “top recipients” of Chinese finance as though each of the numbers aggregated on their list is equally firm. That’s simply not the case, as their own database points out.