Updates from BestWorld’s AI Botmaster Jeremy Lichtman of Oct. 17, 2024

What’s the probability of the US Steel/Nippon Steel merger being officially announced before January 21, 2025? (scroll down to read after the Guide below)

Guide to Bestworldbot’s Reports

Median from the bestworldbots: We collect the output values from the LLMs in an array. We then take the median value. This is between 0 and 1 (i.e. 0.15 = 15%).
Base rate: Currently we take OpenAI’s value as the base rate. We are working on a better value using a different prompt that takes into account process. The base rate helps to determine if the median makes sense.
SD: The standard deviation between the median and the base rate.
Confidence: We query each of the LLMs on how confident they are of their predictions (between 0 and 10) and take a median. Because the LLMs tend to be overconfident, we take anything lower than 6 as being low confidence. This factors into the overall model.
Conf Mode: Based on the confidence value. >=9 is high confidence. Below 6 is low confidence (this is also triggered by an exceptionally high SD).
Mellers: This refers to Barbara Mellers, specifically a paper she wrote that includes a formula for moving values towards an extreme (i.e. 0 or 1).
Reverse Mellers: This uses the formula from above, but with a sub-1 coefficient to move the values closer to 50%.
Theory of Mind: We ask the LLMs what they think other LLMs would predict. We hope that this makes them consider the questions more deeply.
Beta Distribution: Currently unused, but possibly of interest. This is based on the median, the base rate and the SD.
Close Type: We noticed that the appropriate base case for some questions is closer to the extremes, while others are closer to 50%. When we have a low confidence value, this helps us to determine whether to extremize or de-extremize the value. ‘A’ implies closer to zero. ‘B’ implies closer to 50%. ‘C’ implies closer to 100%.
# LLM Responses: The count of LLMs that responded. We query 5 currently, but they have a relatively high API failure rate.
Model value: For normal confidence mode, this is the median. For high confidence, this is the Mellers value. For low confidence, this is either the Mellers or Reverse Mellers (depending on the Close Type).

===============================
Welcome to the Multi-AI Oracle.
===============================
What is your question? What’s the probability of the US Steel/Nippon Steel merger being officially announced before January 21, 2025? Do you have any other information you wish to pass along to assist in answering? (Hit Enter to skip)
ID: N/A

Obtaining outside newsfeed. 
Summarizing outside newsfeed:

“The potential merger between US Steel and Nippon Steel faces significant uncertainty, primarily due to opposition from the Biden administration and the United Steelworkers union over national security concerns. The White House is reportedly preparing to block the proposed $14.9 billion acquisition, reflecting alignment with union objections concerning potential impacts on jobs and the steel industry. The Committee on Foreign Investment in the United States (CFIUS) is currently reviewing the merger, although the timeline for a final decision remains unclear, indicating possible regulatory delays. These factors suggest a low likelihood of the merger being finalized before January 21, 2025. As the situation involves complex political, economic, and regulatory issues, the outcome remains uncertain.”

Querying Perplexity (AI predicts: 0.3 – confidence: 6)
Querying Claude (AI predicts: 0.15 – confidence: 6)
Querying Mistral (AI predicts: 0.2 – confidence: 6)
Querying Cohere (AI predicts: 0.2 – confidence: 6)
Querying OpenAI (AI predicts: 0.15 – confidence: 6)

Median from LLMs: 0.2
Base rate: 0.15 (from OpenAI)

SD: 0.05
Confidence: 6
Conf Mode: Normal
Mellers: 0.08
Reverse Mellers: 0.33
Theory of Mind: 0.3 (What did the LLMs think other LLMs predicted?)
Beta Distribution: 0.17
Close Type: A (B = cautious # closer to 50%; A/C = closer to extremes)
# LLM responses: 5
Model value (not submitted): 0.2

We queried Perplexity, Claude, Mistral, Cohere, OpenAI. A summary of what the responding AIs considered is as follows:

The prediction regarding the merger between US Steel and Nippon Steel centers on several significant factors contributing to a low likelihood of the deal being finalized before January 21, 2025. The primary obstacles include strong opposition from the Biden administration and the United Steelworkers union, largely due to national security concerns and potential job impacts. The Committee on Foreign Investment in the United States (CFIUS) is conducting an ongoing review with an unclear timeline, suggesting potential regulatory delays. Additionally, the complexity associated with the $14.9 billion acquisition adds to the uncertainty. However, the possibility of the prediction being incorrect exists if there are shifts in political stance, unexpected resolution of regulatory concerns, or strategic negotiations that effectively address the highlighted concerns, which may accelerate the process. Although the merger has secured approvals from non-US jurisdictions, US political and labor challenges remain significant hurdles.
Runtime: 104 seconds.

Bestworldbot News

This week bestworldbot continues to make practice forecasts on the Q4 AI Forecasting Benchmark Tournament. Click here to see them.

The Q3 AI Forecasting Benchmark Competition has concluded. Bestworldbot was #53 out of 55 competitors. A likely explanation for bestworldbot’s huge fall on the leaderboard from #17 on Sept. 10 is that we tried extremizing its forecasts. With most human forecasters, extremizing is usually effective. However, we now believe that extremizing made bestworldbot worse. Oh, well, that’s research, some things work, some don’t.

Given the agreement of the Dock Workers to salary increases, what’s the probability of a strike in Q1 2025?

===============================
Welcome to the Multi-AI Oracle.
===============================
What is your question? Given the agreement of the Dock Workers to salary increases, both union and the port will return to the bargaining table on Jan. 15, 2025 to discuss automation and other issues. As the dock workers have only walked out once since 1977, what’s the probability of a strike in Q1 2025.

Do you have any other information you wish to pass along to assist in answering? (Hit Enter to skip)
ID: N/A

Obtaining outside newsfeed. Summarizing outside newsfeed:

“As of October 16, 2024, a tentative agreement has been reached between the International Longshoremen’s Association (ILA) and the United States Maritime Alliance (USMX), granting dock workers a 62% salary increase over six years, thus averting an immediate strike. However, discussions are planned for January 15, 2025, to address unresolved issues such as automation and job security, which remain potential strike triggers if not adequately resolved. The probability of a strike occurring in early 2025 will depend significantly on the success of these future negotiations, especially given historical tensions related to automation in the industry. The newsfeed is a summary and includes sources that may contain unverified information.”

Querying Perplexity (AI predicts: 0.4 – confidence: 6)
Querying Claude (AI predicts: 0.4 – confidence: 5)
Querying Mistral (AI predicts: 0.65 – confidence: 7)
Querying Cohere(Error: Cohere is not returning an array)
Querying OpenAI (AI predicts: 0.3 – confidence: 5)

Median from LLMs: 0.4
Base rate: 0.3 (from OpenAI)

SD: 0.13
Confidence: 5.5
Conf Mode: Low
Mellers: 0.33
Reverse Mellers: 0.45
Theory of Mind: 0.5 (What did the LLMs think other LLMs predicted?)
Beta Distribution: 0.2
Close Type: B (B = cautious # closer to 50%; A/C = closer to extremes)
# LLM responses: 4
Model value (not submitted): 0.45

We queried Perplexity, Claude, Mistral, OpenAI. A summary of what the responding AIs considered is as follows:

The predictions around the likelihood of a strike by dock workers arise from a combination of historical context, recent agreements, and future negotiation dynamics. While a significant salary increase has been achieved, unresolved critical issues like automation and job security remain a source of tension, historically leading to labor disputes. The brief negotiation period set to begin in January 2025 and potential for rapid escalation underscores concerns about a possible strike if a satisfactory agreement isn’t reached. On the other hand, the goodwill generated from the salary resolution, alongside potential external economic pressures and political interventions, might lead to a successful negotiation of these issues, reducing the risk of a strike. The possibility of innovative solutions or compromises emerging in the upcoming talks could address these concerns, potentially altering the prediction.
Runtime: 224 seconds.

Bots vs Staff Forecasts

BestWorld’s staff of Brian LaBatte and Carolyn Meinel are forecasting the US Steel/Nippon Steel Merger in competition with Jeremy’s Multri-AI Oracle. Their version has two sub-questions:

(1) Will the US Steel / Nippon Steel merger be approved by the White House before Jan. 21. 2025?” Brian’s latest forecast is 80%; Carolyn’s is 70%. Michal’s is 20%.

(2) If yes, when will US Steel investors relations announce the final outcome of the acquisition with Nippon Steel?

a. Before November 5, 2024
b. Between November 6 and December 31, 2024
c. January 1 to 20, 2024
d. After January 21, 2024
Brian’s latest forecasts are 0%, 20%, 20%, and 60%.
Carolyn’s are 0%, 20%, 20%, and 60%.
Below are their forecasting spreadsheets.

If yes, when will US Steel’s investors relations announce the results?


Previous bots vs human forecasts:

Oct. 16, 2024
Oct. 15, 2024
Oct. 14, 2024
Oct. 11, 2024
Oct. 10, 2024
Oct. 9, 2024
Oct. 8, 2024
Oct. 7, 2024
Oct. 3, 2024
Oct. 2, 2024
Oct. 1, 2024
Sept. 30, 2024
Sept. 27, 2024
Sept. 26, 2024
Sept. 25, 2024
Sept. 24, 2024
Sept. 23, 2024
Sept. 18, 2024
Sept. 17, 2024
Sept. 16, 2024
Sept. 13, 2024
Sept. 12, 2024
Sept. 11, 2024
Sept. 10, 2024
Sept. 9, 2024
Sept. 6, 2024
Sept. 5, 2024
Sept. 4, 2024
Sept. 3, 2024
Sept. 2, 2024
Aug. 30, 2024
Aug. 29, 2024
Aug. 28, 2024
Aug. 27, 2024
Aug. 26, 2024
Aug. 23, 2024
Aug. 22, 2024
Aug. 21, 2024
Aug. 20, 2024
Aug. 19, 2024
Aug. 16, 2024
Aug. 15, 2024
Aug. 14, 2024
Aug. 13, 2024
Aug. 12, 2024
Aug. 9, 2024
Aug. 8, 2024
Aug. 7, 2024
Aug. 6, 2024
Aug. 5. 2024
Aug. 2, 2024
Aug. 1, 2024
July 31, 2024
July 30, 2024
July 29, 2024
July 26, 2024
July 25, 2024

More about our previous staff vs bot competition here.

Our bots vs humans experiments

Our concluded FED rate increase competition.

All our current bot projects.

Author

  • She has written many books, magazine articles, and research papers. She currently is President and CEO of the nonprofit BestWorld, Inc. and is a professional forecaster of geopolitical events at the RAND Forecasting Initiative.

    View all posts