Inspired by a tweet from our social media team, Replication Markets PI Charles Twardy gives a lesson on what it means to “defend your position.” The trade of the next forecaster replaces the trade you just made. If you still have points, return to defend your position. Many smaller nudges have more power than a…
We have noticed some behavior in the markets that leads us to believe that some of our forecasters have missed the “market” part of the project. There are many null trades where a forecaster has submitted no change at all. Three null trades on one claim. Only “bjaeger” made a forecast. The markets are not…
When Round 3 Markets opened, 203 of our forecasters got a bonus: 30 extra points to invest in the markets! Forecasters who trade enough on multiple rounds earn points based on their longevity and activity. After at least 2 qualifying rounds, you earn a 10% bonus (30 points). After at least 4 qualifying rounds, you…
We have added leaderboards to track Markets vs. Surveys and per-Round scores. This should give all forecasters (not just those who joined at the beginning of the project) a better idea of where they stand. The “Overall Market” is the default. It shows the total Expected Value of your entire market portfolio, using the current…
Important Prize Clarifications The total prize amount for a Round is determined by the number of resolving claims in that Round. Payout is proportional to all winning shares in the round, not per-claim. The Round 0 Market will pay $360 x 12 = $4,320 total. If you have 5% of winning shares, you will get 5% of $4,320 = $216, regardless…
If you’ve just read What is a high-quality replication, you know that our replications will be high-power — usually much higher than the original study — but not quite the “100 replications” ideal set out by DARPA: Assume 100 replications of this study were performed, and a weighted average of their results (i.e., a meta-analysis) was…
What is high quality replication?
In brief: A good-faith, high-power attempt to reproduce a previously-observed finding.
[Updated Jan. 2020 to cover unified definition, and clarify] Concept Replication is when you repeat a previous study to see if you get the same results. Ideally this happens a lot in science. In practice, not as much. But what counts? According to Brian Nosek (“What is replication?”, 2019), people commonly say replication is repeating…
Replication Markets is a government funded research project to forecast the outcomes of thousands of potential social science replications to be done from mid-2019 to mid-2020.