What share of results replicate in different literatures in some of the experimental social sciences? The lecture will discuss several large replication projects in mainly psychology and economics, where the coauthors and prof Dreber have redone experiments published in high impact journals with new and larger samples to see whether the main result replicates. It will also cover the studies on forecasting like prediction markets and forecasting surveys, where researchers are demanded to predict scientific results, and discuss our recent work on the possibility of using decision markets to select which studies to replicate. In the prediction markets, researchers are given monetary endowments to bet on whether results picked for replication will replicate with the definition of successful replication being a statistically significant effect in the same direction as the original study. In decision markets, we use market prices to decide which studies to replicate. The lecture will also discuss work on analytical heterogeneity and design heterogeneity. These are projects where researchers are asked to test the same hypotheses on the same experimental or non-experimental data, or where researchers are asked to design experiments to test the same hypothesis. The results suggest that a large share of results do not replicate and that this is to some extent predictable. The results also suggest that there is substantial analytical and design heterogeneity, limiting the generalizability of individual results.
About the speaker:
Anna Dreber Almenberg is a professor of economics at the Stockholm School of Economics. Her research is mainly in meta science, exploring topics such as the replicability and generalizability of scientific results and the potential use of prediction and decision markets for replications. Anna also does work in behavioral economics on topics such as hormones and decision making (with mainly null results from high-powered studies) and economic preferences more broadly. Anna is an editor at the Journal of Political Economy Microeconomics, a relatively new journal which encourages replications and Registered Reports, and she is a member of the Royal Swedish Academy of Sciences and a Wallenberg Scholar.