{"id":892,"date":"2019-06-23T21:04:39","date_gmt":"2019-06-23T21:04:39","guid":{"rendered":"http:\/\/replicationmarkets.com\/?p=892"},"modified":"2019-09-04T09:22:21","modified_gmt":"2019-09-04T13:22:21","slug":"welcome-to-replication-markets","status":"publish","type":"post","link":"https:\/\/replicationmarkets.org\/index.php\/2019\/06\/23\/welcome-to-replication-markets\/","title":{"rendered":"Welcome to Replication Markets!"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"892\" class=\"elementor elementor-892\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-0f78111 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"0f78111\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-d445871\" data-id=\"d445871\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-0dffdd2 elementor-widget elementor-widget-text-editor\" data-id=\"0dffdd2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\">Replication Markets is a government funded research project to forecast the outcomes of thousands of potential social science replications to be done from mid-2019 to mid-2020. As part of the project, the Center for Open Science has selected 3,000 studies for us to forecast, from the last 10 years of published social science. \u00a0They will select a representative sample (unknown to us) of 100+ studies for direct replication, and supervise those replications. The results will tell us if we have met or exceed the 70-80% accuracy of our <a href=\"http:\/\/www.citationfutures.com\" target=\"_blank\" rel=\"noopener\">previous replication markets<\/a>, and inform Confidence Scores on the reliability of the claims themselves.<\/span><\/p><p><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\"><b>We are still awaiting final IRB approval and hope to begin forecasting in August. In the meantime, see the link at the bottom to follow the project.<\/b><\/span><\/p><h4>\u00a0<\/h4><h4><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\">Approach<\/span><\/h4><p><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\">Our approach will use both surveys and markets. Based on our previous work we expect the markets to have a small edge, but we are also working on smarter ways to aggregate the surveys. \u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-49f7baf elementor-widget elementor-widget-blockquote\" data-id=\"49f7baf\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"blockquote.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t \t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-484fc6e elementor-widget elementor-widget-text-editor\" data-id=\"484fc6e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\">Prediction markets and surveys are used to forecast the outcome of a wide variety of topics from elections to geopolitics to delivery schedules. A <a href=\"http:\/\/journal.sjdm.org\/18\/18919\/jdm18919.html\" target=\"_blank\" rel=\"noopener\">recent paper<\/a> from the Good Judgment Team provides the best head-to-head comparison yet, and shows that clever averaging outperforms markets, which outperform simple averages. We will be testing the following improvements over our previous work:<\/span><\/p><ul><li><span style=\"font-family: Montserrat, sans-serif; font-size: 12pt; letter-spacing: 0px; white-space: pre-wrap; background-color: transparent;\">Better aggregation methods for the surveys.<\/span><\/li><li><span style=\"font-family: Montserrat, sans-serif; font-size: 12pt; letter-spacing: 0px; white-space: pre-wrap; background-color: transparent;\">Starting the markets with informed prices, not just 50%. \u00a0<\/span><\/li><li><span style=\"letter-spacing: 0px; font-size: 12pt; white-space: pre-wrap; background-color: transparent; font-family: Montserrat, sans-serif;\">Allowing bots to trade in the market, with some rules of engagement. <\/span><\/li><li><span style=\"letter-spacing: 0px; font-size: 12pt; white-space: pre-wrap; background-color: transparent; font-family: Montserrat, sans-serif;\">Allowing connections between claims (starting later in the year)<\/span><\/li><li><span style=\"letter-spacing: 0px; font-size: 12pt; white-space: pre-wrap; background-color: transparent; font-family: Montserrat, sans-serif;\">Because only about 5% of the claims will resolve, we are using Decision Market scoring rules to preserve incentives. \u00a0(See <a href=\"https:\/\/www.researchgate.net\/publication\/220853491_Decision_Markets_With_Good_Incentives\" target=\"_blank\" rel=\"noopener\">&#8220;Decision Markets with Good Incentives&#8221;<\/a>.)<\/span><\/li><\/ul><p><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\">Once running, we expect to work in monthly batches of about 300 claims. Specifics will be described in the Informed Consent statement and related materials. <\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-df62275 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"df62275\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-7527489\" data-id=\"7527489\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-d7d5cca elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"d7d5cca\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-8006e13\" data-id=\"8006e13\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-fbf6258 elementor-widget elementor-widget-text-editor\" data-id=\"fbf6258\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<h4>\u00a0Follow Us<\/h4><p><span style=\"font-family: Montserrat, sans-serif; font-size: medium;\"><span style=\"white-space: pre-wrap;\">Although final results will not be available until mid-2020, you may follow the project to receive updates and relevant posts.<\/span><\/span><\/p><p><span style=\"font-size: 12pt; font-family: Montserrat, sans-serif; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;\"><strong>Replication Markets will be launched in August 2019 pending IRB approval; you can <a href=\"https:\/\/replicationmarkets.com\/index.php\/sign-up\/\" target=\"_blank\" rel=\"noopener\">sign up here<\/a> to be notified. <\/strong><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a71a71d elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a71a71d\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-c28c05a\" data-id=\"c28c05a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-bd10ebb elementor-widget elementor-widget-post-navigation\" data-id=\"bd10ebb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"post-navigation.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t \t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Replication Markets is a government funded research project to forecast the outcomes of thousands of potential social science replications to be done from mid-2019 to mid-2020.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[18],"tags":[61,41,28,27,36,63,31],"class_list":["post-892","post","type-post","status-publish","format-standard","hentry","category-announcements","tag-crowd-sourcing","tag-markets","tag-prediction-market","tag-replication","tag-results","tag-score","tag-survey"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/posts\/892","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/comments?post=892"}],"version-history":[{"count":20,"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/posts\/892\/revisions"}],"predecessor-version":[{"id":2155,"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/posts\/892\/revisions\/2155"}],"wp:attachment":[{"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/media?parent=892"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/categories?post=892"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/replicationmarkets.org\/index.php\/wp-json\/wp\/v2\/tags?post=892"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}