Do you believe that you are a super-forecaster ? Then sign up for the IARPA good judgement project

The Good Judgment Project is a four-year research study organized as part of a government-sponsored forecasting tournament. Thousands of people around the world predict global events. Their collective forecasts are surprisingly accurate.

The Intelligence Advanced Research Projects Activity (IARPA) invests in high-risk, high-payoff research programs that have the potential to provide the United States with an overwhelming intelligence advantage over future adversaries. IARPA works very closely with the various members of the Intelligence Community to ensure that its programs address relevant future needs and to facilitate the transition of demonstrated capabilities. However, IARPA is not an operational organization, and it neither collects raw intelligence nor produces and disseminates intelligence analyses. To ensure organizational agility, IARPA focuses on long-term, 3-5 year programs rather than the short-term time horizons

The goal of the ACE Program is to dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts. The ACE Program seeks technical innovations in the following areas: (a) efficient elicitation of probabilistic judgments, including conditional probabilities for contingent events; (b) mathematical aggregation of judgments by many individuals, based on factors that may include: past performance, expertise, cognitive style, metaknowledge, and other attributes predictive of accuracy; and (c) effective representation of aggregated probabilistic forecasts and their distributions. The ACE Program will build upon technical achievements of past research and on state-of-the-art systems used today for generating probabilistic forecasts from widely-dispersed experts. The program will involve empirical testing of forecasting accuracy against real events.

In previous years, superforecasters organized into elite teams outperformed average (wisdom of overall crowds) by 65% and beat the algorithms of four competitor institutions by 35-60% and beat two prediction markets by 20-35%.

“Super forecasters” are distinguished by three characteristics:
(1) an intense curiosity about the workings of the political-economic world;
(2) an intense curiosity about the workings of the human mind;
(3) cognitive crunching power (“fluid intelligence” and a capacity for “timely self correction”).
Read more »