The Experiment
Jeremy Lichtman had been coding updates to the Multi-AI Panel as he explored ways to teach AIs to become better forecasters, perhaps someday, better than our human team of Brian LaBatte and Carolyn Meinel, both of whom are “superforecasters™” which means that they were in the top 2% in IARPA’s Good Judgment Project.
We now have completed that first Multi-AI experiment and for now, are retiring the panel format.
See our current Gen AI forecasting experiments here.
The Human Team
BestWorld’s staff of Brian LaBatte and Carolyn Meinel had been competing with Botmaster Jeremy Lichtman’s Mult-AI Panel on our forecasting question of “What is the probability that the US Federal Reserve Board will cut interest rates in September 2024?” We now have the answer: Sept. 18, 2024, the US FED lowered its key overnight borrowing rate by a half percentage point, or 50 basis points, a huge cut compared to the usual quarter point changes, and the first cut since 2020. Later we’ll post our scores. The final scores will not be what we forecasted just before the answer was known, because being more correct — or incorrect — earlier will count toward the final scores.
Summary of all our FED rate forecasts: We stopped reporting the Multi-AI Panel on Sept. 13. Its final answer was 85%, up from the previous day’s 75%. As of Sept. 18, Old Bot had dropped from 85% to just 55%. Our staffers remained at 84% median, 95% with extremizing via one of Prof. Barbara Meller’s algorithms. This includes the effect of our earlier, lower forecasts. Our fianl spot forecasts were on Sept. 12th and were 98% for Brian, 99% for Carolyn.
The record of our staff forecasts is below. Note that our spreadsheet also includes some of our bots and that in this visualization the
The Multi-AI Panel Team
There were five generative AIs on the Multi-AI Panel: Perplexity, Claude, Mistral, Cohere, and OpenAI. They forecasted just one question: “What is the probability that the US Federal Reserve Board will cut interest rates in September 2024?”
.
Below, the Multi-AI Panet vs Old Bot vs Staff forecasts on the question “What is the probability that the US Federal Reserve Board will cut interest rates in September 2024?”
Sept. 18, 2024
Sept. 17, 2024
Sept. 16, 2024
Sept. 13, 2024
Sept. 12, 2024
Sept. 11, 2024
Sept. 10, 2024
Sept. 9, 2024
Sept. 6, 2024
Sept. 5, 2024
Sept. 4, 2024
Sept. 3, 2024
Sept. 2, 2024
Aug. 30, 2024
Aug. 29, 2024
Aug. 28, 2024
Aug. 27, 2024
Aug. 26, 2024
Aug. 23, 2024
Aug. 22, 2024
Aug. 21, 2024
Aug. 20, 2024
Aug. 19, 2024
Aug. 16, 2024
Aug. 15, 2024
Aug. 14, 2024
Aug. 13, 2024
Aug. 12, 2024
Aug. 9, 2024
Aug. 8, 2024
Aug. 7, 2024
Aug. 6, 2024
Aug. 5. 2024
Aug. 2, 2024
Aug. 1, 2024
July 31, 2024
July 30, 2024
July 29, 2024
July 26, 2024
July 25, 2024
