The Multi-AI Oracle and the bestworldbot Experiments
Currently, Jeremy is running the Multi-AI Oracle vs Staff forecasting competition on the question: What is the probability of the US Steel/Nippon Steel merger being officially announced before January 21, 2025?
Today (Oct. 11, 2024) the Multi-AI Oracle is forecasting 15%, far below Brian’s latest forecast of 66%, Carolyn’s 70%.
Below: What is the probability of the US Steel/Nippon Steel merger being officially announced before January 21, 2025? (all forecasts)
Oct. 11, 2024
Oct. 10, 2024
Oct. 9, 2024
Oct. 8, 2024
Oct. 7, 2024
Oct. 5, 2024
Oct. 3, 2024
Oct. 2, 2024
Oct. 1, 2024
Sept. 30, 2024
Sept. 27, 2024
Sept. 26, 2024
Sept. 25, 2024
Sept. 24, 2024
Sept. 23, 2024
Jeremy’s bestworldbot:
Jeremy plans on continuing to field his evolving creation, bestworldbot, on the next quarter that began Oct. 8, 2024, of the Q4 AI Forecasting Benchmark Tournament.Currently it is in test question mode. So far, bestworldbot’s practice question responses are looking good.
Another indicator of our chances in the Q4 competition is that the rules have been tightened up to prevent cheating by including human forecasting inputs.
Below is the final Q3 leaderboard, published Oct. 8, 2024. Bestworldbot finished #53 out of 55 competitors. That is down from #17 on Sept. 10. A likely explanation for bestworldbot’s collapse on the leaderboard is that in early September we began extremizing its forecasts. With most human forecasters, extremizing is, on average, effective. However, we now believe that extremizing made bestworldbot worse.
Below, the final leaderboard.
The Human Team
BestWorld’s staff of Brian LaBatte and Carolyn Meinel now are forecasting the US Steel/Nippon Steel Merger. Their version has two sub-questions:
(1) Will the US Steel / Nippon Steel merger be officially announced before Jan. 21, 2025?” Brian’s latest forecast is 78%, Carolyn’s, 70%.
(2) If yes, when will US Steel investors relations announce the final outcome of the acquisition with Nippon Steel?
a. Before November 5, 2024
b. Between November 6 and December 31, 2024
c. January 1 to 20, 2024
d. After January 21, 2024
Brian’s latest forecasts are 0%, 50%, 28%, and 20%.
Carolyn’s are 1%, 20%, 20%, and 60%.
They previously had been competing with Botmaster Jeremy Lichtman’s Mult-AI Panel on our forecasting question of “What is the probability that the US Federal Reserve Board will cut interest rates in September 2024?”
Summary of all our FED rate forecasts: Sept. 16, 2024, we were unable to update the Multi-AI Panel due to technical problems. Two days later, the FED raised the rate by half a point, the first rise since 2020. The previous answer, Sept. 13, was 85%, up from the previous day’s 75%. Old Bot remained at 85%. Our staffers remained at 84% median, 95% with extremizing via one of Prof. Barbara Meller’s algorithms.
Our staff forecasts are below. Note that our spreadsheet also includes some of our bots.
The Q4 AI Forecasting Benchmark Tournament
In this tournament, Metaculus is fielding a team of its best human forecasters competing against the bots, at last count, 53 of them, including Jeremy’s bestworldbot.
Now retired:
The Multi-AI Panel bot Jeremy used until Sept. 16, 2024 began with four generative AIs, later five: Perplexity, Claude, Mistral, Cohere, and OpenAI. They forecasted just one question through September 16, 2024: “What is the probability that the US Federal Reserve Board will cut interest rates in September 2024?”
The Bots vs Staff forecasts on the FED rate rise question:
Sept. 16, 2024
Sept. 13, 2024
Sept. 12, 2024
Sept. 11, 2024
Sept. 10, 2024
Sept. 9, 2024
Sept. 6, 2024
Sept. 5, 2024
Sept. 4, 2024
Sept. 3, 2024
Sept. 2, 2024
Aug. 30, 2024
Aug. 29, 2024
Aug. 28, 2024
Aug. 27, 2024
Aug. 26, 2024
Aug. 23, 2024
Aug. 22, 2024
Aug. 21, 2024
Aug. 20, 2024
Aug. 19, 2024
Aug. 16, 2024
Aug. 15, 2024
Aug. 14, 2024
Aug. 13, 2024
Aug. 12, 2024
Aug. 9, 2024
Aug. 8, 2024
Aug. 7, 2024
Aug. 6, 2024
Aug. 5. 2024
Aug. 2, 2024
Aug. 1, 2024
July 31, 2024
July 30, 2024
July 29, 2024
July 26, 2024
July 25, 2024
