Restoring News Readers can Trust
BestWorld is reimagining both traditional and social media-based news. Our goal: a scalable demonstration of a news ecosystem that is truthful, believable, impactful, beneficial, and widely multilingual — and from the technical standpoint, immediately scalable worldwide. We plan to take this experiment live with a small group of users by the end of May, 2024.
LEARN MORE
Top 10 Research Findings Supporting BestWorld
Reduce polarized attitudes via forecasting tournaments
"Drawing on the literature about the effects of accountability, the authors suggest that tournaments may hold even greater potential as tools for depolarizing political debates and resolving policy disputes."
“Political debates often suffer from vague-verbiage predictions that make it difficult to assess accuracy and improve policy. A tournament sponsored by the U.S. intelligence community revealed ways in which forecasters can better use probability estimates to make predictions—even for seemingly “unique” events—and showed that tournaments are a useful tool for generating knowledge."
Learn MoreForecasting geopolitical topics causes people toi become less polarized on unrelated topics.
"People often express political opinions in starkly dichotomous terms, such as “Trump will either trigger a ruinous trade war or save U.S. factory workers from disaster.” This mode of communication promotes polarization into ideological in-groups and out-groups. We explore the power of an emerging methodology, forecasting tournaments, to encourage clashing factions to do something odd: to translate their beliefs into nuanced probability judgments and track accuracy over time and questions."
“In a two-year forecasting tournament, participants who actively engaged in predicting US domestic events were less polarized in their policy preferences than were non-forecasters. Self-reported political attitudes were more moderate among those who forecasted than those who did not. We also found evidence that forecasters attributed more moderate political attitudes to the opposing side.”
Learn MoreFour Traits of Superforecasters [tm] of Geopolitical Events
"Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters."
We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created.”
Learn MoreWhat do forecasting rationales reveal about thinking patterns of top geopolitical forecasters?
If you want more accurate probability judgments in geopolitical tournaments, you should look for... forecasters who are exceptions to two of the more robust generalizations of 20th century experimental psychology. Look for forecasters who don’t rush to reduce cognitive dissonance (Festingher, 1957) and who don’t jump to conclusions from vivid case-specific events (Kahneman, 2011).
“(a) top forecasters show higher dialectical complexity in their rationales and use more comparison classes; (b) experimental interventions, like training and teaming, that boost accuracy also influence NLP profiles of rationales, nudging them in a 'superforecaster' direction.”
Learn More“Open-minded thinkers are more likely than others to make accurate forecasts.”
"We focus on actively open-minded thinking (AOT), need for cognition, grit, and the tendency to maximize or satisfice when making decisions. In three studies, participants made estimates and predictions of uncertain quantities, with varying levels of control over the amount of information they could collect before estimating. Only AOT predicted performance."
“This relationship was mediated by information acquisition: AOT predicted the tendency to collect information, and information acquisition predicted performance. To the extent that available information is predictive of future outcomes, actively open-minded thinkers are more likely than others to make accurate forecasts.”
Learn MoreOpen-minded Thinking and Gaining Knowledge
Errors in estimating and forecasting often result from the failure to collect and consider enough relevant information. We examine whether attributes associated with persistence in information acquisition can predict performance in an estimation task. We focus on actively open-minded thinking (AOT), need for cognition, grit, and the tendency to maximize or satisfice when making decisions.
“In three studies, participants made estimates and predictions of uncertain quantities, with varying levels of control over the amount of information they could collect before estimating. Only AOT [open-minded thinking] predicted performance. This relationship was mediated by information acquisition: AOT predicted the tendency to collect information, and information acquisition predicted performance.”
Learn MoreOpening Closed Minds with Tournaments
“Tournaments have the potential to open closed minds and increase assertion-to-evidence ratios in polarized scientific and policy debates.”
"Forecasting tournaments are level-playing-field competitions that reveal which individuals, teams, or algorithms generate more accurate probability estimates on which topics. This article describes a massive geopolitical tournament that tested clashing views on the feasibility of improving judgmental accuracy and on the best methods of doing so. The tournament's winner, the Good Judgment Project, outperformed the simple average of the crowd by (a) designing new forms of cognitive-debiasing t...
Learn MoreShifting attention to accuracy increases the quality of people’s news-sharing decisions.
"The spread of misinformation is a pressing societal challenge. Prior work shows that shifting attention to accuracy increases the quality of people’s news-sharing decisions.... We observed significant partisan moderation for single-headline 'evaluation' treatments.... Overall, we observed significant partisan moderation ... "
"[After evaluating] “a single neutral (nonpolitical, non-COVID-19) headline… [these] accuracy prompts had a significant positive effect on sharing discernment for Republicans/conservatives …We also found that accuracy prompts had a significant positive effect on sharing discernment for Democrats/liberals.”
Learn MoreAdversarial Collaborations: a way to settle differences over facts
As originally conceived by Economics Nobel Prize Laureate, Daniel Kahneman, adversarial collaborations call on scholars to: (1) make good faith efforts to articulate each other’s positions (so that each side feels fairly characterized, not caricatured); (2) work together to design methods that both sides agree constitute a fair test and that they agree, ex ante, have the potential to change their minds; (3) jointly publish the results, regardless of “who wins, loses or draws” on which topics.
"Each collaborator serves as a check on their adversary to confirm that the hypotheses are falsifiable, the scientific tests are fair, and the interpretations accurately characterize the findings. Because adversarial collaborations restrict scholars’ abilities to rig methods in favor of their own hypothesis and to dismiss unexpected results, adversarial collaborations are likely to advance debates faster and generate more reliable knowledge than traditional approaches."
Learn MoreCombatting the problem of erroneous research results through adversarial collaborations
"Some highly publicized findings have failed to replicate, and many highly touted 'science-based' interventions have failed to produce promised positive social change.... the ideological homogeneity of the social sciences has entrenched certain scientific orthodoxies and taboos that have protected weak ideas...."
"Adversarial collaborations, a methodological procedure in which disagreeing scholars work together to resolve their empirical disputes, are the next necessary science reform for addressing lingering weaknesses in social scientific norms. Adversarial collaborations will further minimize false positives, expedite scientific corrections, stimulate progress for stalemated scientific debates, and ultimately improve the quality of social scientific outputs."
Learn MoreIntense dialogues with an AI chatbot reduces beliefs in false conspiracy stories
"ChatGPT to the rescue?" " New research indicates that finding the right facts can instill skepticism in conspiracy believers, and artificial intelligence (AI) is just the tool to help. After an evidence-based dialogue with GPT-4 Turbo, an AI model, conspiracy theorists substantially reduced their belief in favored conspiracies, and this newfound skepticism remained for months. Unraveling webs of conspiracy beliefs at scale has not previously been possible."
"For years... high-speed sharing and the relentless activity of social media influencers have made the [problem of disinformation] even worse. Dangerous skepticism about the COVID-19 vaccine, for example, continues to stoke wider hesitancy to other vaccines, a disastrous consequence for public health. Solutions to blunt scientific misinformation have been elusive. Now a new study, reported in this issue of Science, suggests that artificial intelligence (AI) could provide a means to dispel mis...
Learn MoreDurably reducing conspiracy beliefs through dialogues with AI
"Conspiracy theories are... beliefs that, once adopted, are extremely difficult to dispel. Influential psychological theories propose that conspiracy beliefs are uniquely resistant to counterevidence because they satisfy important needs and motivations. Here, we raise the possibility that previous attempts to correct conspiracy beliefs have been unsuccessful merely because they failed to deliver counterevidence that was sufficiently compelling and tailored to each believer’s specific conspira...
"...we find robust evidence that the debunking conversation with the AI reduced belief in conspiracy theories by roughly 20%. This effect did not decay over 2 months time, was consistently observed across a wide range of different conspiracy theories, and occurred even for participants whose conspiracy beliefs were deeply entrenched and of great importance to their identities. Furthermore,... the intervention spilled over to reduce beliefs in unrelated conspiracies, indicating a general decre...
Learn More.
Sources
[1]
“Bringing probability judgments into policy debates via forecasting tournaments,” Philip E. Tetlock, Barbara A. Mellers and J. Peter Scoblic. SCIENCE 3 Feb 2017 Vol 355, Issue 6324 pp. 481-483. DOI: 10.1126/science.aal31
[2] “Forecasting tournaments, epistemic humility and attitude depolarization,” Barbara Mellers, Phil Tetlock and Hal Arkes, Cognition Volume 188, July 2019, Pages 19-26. DOI: https://doi.org/10.1016/j.cognition.2018.10.021
[3]
“Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions,” Barbara Mellers, Eric Stone, Terry Murray, Angela Minster, Nick Rohrbaugh, Michael Bishop, Eva Chen, Joshua Baker, Yuan Hou, Michael Horowitz, Lyle Ungar, and Philip Tetlock. Perspectives on Psychological Science, 2015, Vol. 10(3) 267–281. DOI: https://doi.org/10.1177/1745691615577
[4]
“What do forecasting rationales reveal about thinking patterns of top geopolitical forecasters?” Christopher W. Karvetski, Carolyn Meinel, Daniel T. Maxwell, Yunzi Lu, Barbara A. Mellers, Philip E. Tetlock. International Journal of Forecasting, Volume 38, Issue 2, April–June 2022, Pages 688-704. DOI: ttps://doi.org/10.1016/j.ijforecast.2021.09.003
[5]
“The role of actively open-minded thinking in information acquisition, accuracy, and calibration,” Uriel Haran, Ilana Ritov and Barbara A. Mellers. Judgment and Decision Making , Volume 8 , Issue 3 , May 2013 , pp. 188 – 201 DOI: https://doi.org/10.1017/S1930297500005921
[6] “The Role of Actively Open-Minded Thinking in Information Acquisition, Accuracy, and Calibration,” Uriel Haran, Ilana Ritov, and Barbara A. Mellers. May 2013Judgment and Decision Making 8(3):188-201, DOI: 10.1017/S1930297500005921
[7] “Forecasting Tournaments: Tools for Increasing Transparency and Improving the Quality of Debate,” Philip E. Tetlock, Barbara A. Mellers, Nick Rohrbaugh, and Eva Chen. Current Directions in Psychological Science, Volume 23, Issue 4 https://doi.org/10.1177/0963721414534257
[8]
“On the Efficacy of Accuracy Prompts Across Partisan Lines: An Adversarial Collaboration,” Cameron Martel, Steve Rathje, Cory J. Clark, Gordon Pennycook, Jay J. Van Bavel, David G. Rand, and Sander van der Linden. https://www.psychologicalscience.org. First published online March 20, 2024. https://doi.org/10.1177/09567976241232905
[9]
Adversarial Collaboration Project https://web.sas.upenn.edu/adcollabproject/
[10] “Adversarial Collaboration: The Next Science Reform,” Cory Clark and Philip E. Tetlock, a chapter in the book Political Bias in Psychology: Nature, Scope, and Solutions. C. L. Frisby, R. E. Redding, W. T. O’Donohue, & S. O. Lilienfeld (Eds.), New York: Springer 2022.
[11] https://www.science.org/toc/science/385/6714
[12]”Durably reducing conspiracy beliefs through dialogues with AI,” Thomas H Costello, Gordon Pennycook, David G Rand,
Science. 2024 Sep 13;385(6714):eadq1814. doi: 10.1126/science.adq1814. Epub 2024 Sep 133
