What is the probability of the US Steel/Nippon Steel merger being officially announced before January 21, 2025?
Welcome to the Multi-AI Oracle. Today, 5%. [Carolyn‘s note: Silly bot! This should be 0%. Only three days remain for President Biden to reverse his decision and approve the merger. Incoming President Trump also is against the merger.]
Obtaining our outside newsfeed from various sources. Here’s a summary of the recent developments regarding the US Steel-Nippon Steel merger situation:
President Joe Biden recently blocked Nippon Steel’s proposed $14.9 billion acquisition of US Steel, citing national security concerns. In response, both US Steel and Nippon Steel jointly filed lawsuits against the Biden administration, the United Steelworkers union, and Cleveland-Cliffs on January 6, 2024. The lawsuit alleges that Biden’s executive order blocking the merger was unconstitutional and politically motivated rather than based on legitimate national security concerns.
The most recent development (as of January 13, 2024) is that US authorities have extended the deadline for Nippon Steel to abandon the acquisition, suggesting some flexibility in the implementation of Biden’s order. This extension comes as Japan’s Prime Minister has urged Washington to address concerns about the blocked merger, highlighting the international diplomatic implications of the decision.
The legal challenge filed by the companies claims that Biden violated the Constitution by blocking the deal through what they describe as a “sham” process. This has created tension between the US and Japan, with potential implications for their broader economic and strategic relationship. The outcome of this legal battle could have significant implications for future foreign investments in US strategic industries and international business relations between the two countries.
Summarizing outside newsfeed:
“Recent reports indicate that President Joe Biden blocked Nippon Steel’s proposed $14.9 billion acquisition of US Steel, citing national security concerns. In response, US Steel and Nippon Steel have filed a lawsuit against the Biden administration, challenging the decision as unconstitutional and politically motivated. This legal challenge, which also involves the United Steelworkers union and Cleveland-Cliffs, has added tension to U.S.-Japan relations, with Japan’s Prime Minister advocating for a resolution. While the U.S. has extended the deadline for Nippon Steel to withdraw the acquisition, signaling potential flexibility, the situation underscores broader implications for foreign investments in U.S. strategic industries and international business relations.”
Fact-checking news summary:
Of course. Let’s break down the summary into specific facts and evaluate each:
1. **Fact**: President Joe Biden blocked Nippon Steel’s proposed $14.9 billion acquisition of US Steel, citing national security concerns.
– **Correct**: Let’s assume this is true as per the summary, but it would need verification from external, reliable sources.
– **Importance to Resolution**: Critical. If true, this is a significant barrier to the merger being announced by the stated date.
– **Actual Relevance**: True. Directly impacts the probability of the merger being announced.
2. **Fact**: US Steel and Nippon Steel have filed a lawsuit against the Biden administration, challenging the decision as unconstitutional and politically motivated.
– **Correct**: Assuming it is accurately reported, but would need external confirmation.
– **Importance to Resolution**: Important. Legal actions can delay processes and influence potential outcomes.
– **Actual Relevance**: True. This lawsuit challenges the blocking and could affect the timing of any potential announcement.
3. **Fact**: The legal challenge involves the United Steelworkers union and Cleveland-Cliffs.
– **Correct**: Assuming accurate reporting, though external verification is needed.
– **Importance to Resolution**: Important. It suggests additional complexity and third-party influences that can impact the outcome.
– **Actual Relevance**: True. Third-party involvement can play a role in the outcome.
4. **Fact**: The lawsuit has added tension to U.S.-Japan relations.
– **Correct**: Assuming reliable reporting, but subjective and requires context for full verification.
– **Importance to Resolution**: Lesser importance. It affects the broader geopolitical context but less directly impacts the announcement timing.
– **Actual Relevance**: True. While it provides background, it doesn’t directly affect the legal or procedural processes.
5. **Fact**: Japan’s Prime Minister is advocating for a resolution.
– **Correct**: Assuming it’s a true statement, but requires verification.
– **Importance to Resolution**: Lesser importance. Shows political will but doesn’t change legal constraints directly.
– **Actual Relevance**: False. It is supportive context rather than a determinative factor in timing.
6. **Fact**: The U.S. has extended the deadline for Nippon Steel to withdraw the acquisition.
– **Correct**: Assuming true for consistency, though it needs checking.
– **Importance to Resolution**: Important. Indicates potential for ongoing negotiations that could influence timing.
– **Actual Relevance**: True. Extending deadlines can directly alter the potential for timely resolution or announcement.
7. **Fact**: The situation underscores broader implications for foreign investments in U.S. strategic industries and international business relations.
– **Correct**: This is a broader inference rather than a direct fact, typically true in context but requires context verification.
– **Importance to Resolution**: Unimportant. Provides background and implications but doesn’t affect specific timing or litigation outcome.
– **Actual Relevance**: False. It frames a background scenario but has no direct impact on the specific question.
Each of these facts, when verified, contributes a layer of understanding regarding the potential timing of the merger announcement, with a varied degree of influence on the resolution of the original question.
Fact-checked summary:
The probability of the US Steel/Nippon Steel merger being officially announced before January 21, 2025, faces significant challenges due to critical barriers. President Joe Biden’s decision to block the $14.9 billion acquisition citing national security concerns is a major obstacle. The subsequent lawsuit filed by US Steel and Nippon Steel against the Biden administration, claiming the decision was unconstitutional and politically motivated, further complicates the situation. This legal challenge involves additional parties, such as the United Steelworkers union and Cleveland-Cliffs, adding complexity and potential delays. Although the U.S. has extended the deadline for Nippon Steel to withdraw the acquisition, allowing for ongoing negotiations, these proceedings underscore the broader conflicts impacting the merger’s timeline. Each of these elements affects the probability of an announcement occurring by the specified date.
OpenAI gives a base rate of 0.05.
The following were considered in order to produce this base rate:
The base rate is determined by evaluating the significant obstacles to the merger, particularly the national security concerns raised by the Biden administration. Historically, mergers facing national security blocks have a low probability of overcoming such hurdles quickly, especially when legal challenges and multiple stakeholders are involved. Given the lawsuit and the involvement of powerful unions and competing companies, the likelihood of a resolution by January 21, 2025, is low. Typically, mergers with ongoing legal disputes and federal objections face lengthy negotiations and delays.
Some potential divergent considerations that might affect the base rate:
While the odds are low, sudden developments, such as a court ruling that expedites the process or a shift in the political landscape leading to a more favorable attitude toward the merger, could drastically change the outcome. Additionally, if the involved parties reach a rapid agreement that resolves the national security issues to the satisfaction of the US government, this could increase the probability of an announcement before the deadline.
The following chain of events are necessary for the question to resolve positively:
– The lawsuit filed by US Steel and Nippon Steel against the Biden administration is resolved quickly and in favor of the merger. (Improbable)
– National security concerns are adequately addressed to the satisfaction of US government agencies. (Unlikely)
– The United Steelworkers union and Cleveland-Cliffs either withdraw their objections or their objections are overruled. (Unlikely)
– Continued negotiations between the involved parties (US Steel, Nippon Steel, US Government) lead to a compromise or settlement that allows the merger to proceed. (Unlikely)
Querying Claude (AI predicts: 0.05 – confidence: 8)
Querying Mistral (AI predicts: 0.03 – confidence: 7)
Querying OpenAI (AI predicts: 0.05 – confidence: 3)
Question Type: Binary
Median from LLMs: 0.05
Base rate: 0.05 (from OpenAI)
SD: 0.01
Confidence: 7
Conf Mode: Normal
Mellers: 0.01
Reverse Mellers: 0.13
Theory of Mind: 0.1 (What did the LLMs think other LLMs predicted?)
Beta Distribution: 0.48
Close Type: A (B = cautious # closer to 50%; A/C = closer to extremes)
# LLM responses: 3
Model value: 0.05
We queried Claude, Mistral, OpenAI
A summary of what the responding AIs considered is as follows:
The merger faces several notable hurdles that make an announcement by January 21, 2025, appear unlikely. Key obstacles include active opposition from the Biden administration due to national security concerns, an ongoing lawsuit, objections from both the United Steelworkers union and Cleveland-Cliffs, and a very short timeline to address these complex issues. Historical precedents suggest that such mergers typically require extensive time to resolve, especially when legal and federal challenges are present, resulting in a low base probability for the announcement. However, there remains the possibility of unexpected developments such as a favorable court ruling or a significant political shift that could rapidly resolve outstanding issues, thereby increasing the probability of an announcement within the desired timeframea
Given the agreement of the Dock Workers to salary increases, both union and the port will return to the bargaining table on Jan. 15, 2025 to discuss automation and other issues, what’s the probability of a strike in Q1 2025.
Welcome to the Multi-AI Oracle. Today, 15%.
Obtaining our outside newsfeed from various sources. Based on the recent news articles, here’s a summary of the port workers’ strike situation:
In January 2025, a potential major port strike was averted when the International Longshoremen’s Association (ILA), representing 45,000 dock workers on the U.S. East and Gulf Coasts, reached a tentative agreement with the United States Maritime Alliance (USMX). This came after concerns about a possible strike that could have disrupted supply chains and potentially led to increased inflation and shortages.
The key issue in the negotiations was automation at the ports, which had been a central point of contention. The new agreement is a six-year master contract, though specific details of the deal were not immediately disclosed. This agreement came just months after a previous short strike in October 2024 that had caused shipping price increases and cargo backlogs at 36 affected ports.
The successful negotiation prevented what could have been a significant disruption to the U.S. supply chain, as the affected ports handle a substantial portion of U.S. maritime trade. The agreement helps maintain stability at major ports along the East Coast and Gulf Coast, avoiding potential economic impacts that could have affected both businesses and consumers.
Summarizing outside newsfeed:
“In January 2025, the International Longshoremen’s Association (ILA) and the United States Maritime Alliance (USMX) reached a tentative agreement, averting a potential major strike by 45,000 dock workers on the U.S. East and Gulf Coasts. The threat of a strike had raised concerns about disruptions to supply chains, inflation, and shortages. A key issue in the negotiations was automation at the ports. The new agreement, a six-year master contract, follows a short strike in October 2024 that led to increased shipping prices and cargo backlogs. This resolution is seen as crucial for maintaining stability at these ports, which are vital to U.S. maritime trade, thus preventing potential economic impacts on businesses and consumers. Specific details of the agreement, however, remain undisclosed.”
Fact-checking news summary:
Here is a list of specific facts extracted from the summary, along with assessments of their correctness, importance, and relevance to the question about the probability of a strike in Q1 2025:
1. **Fact**: In January 2025, the International Longshoremen’s Association (ILA) and the United States Maritime Alliance (USMX) reached a tentative agreement.
– **Correctness**: True (assuming the summary is internally consistent and accurate).
– **Importance**: Critical, as the agreement directly impacts the probability of a strike.
– **Relevance**: True, because a tentative agreement reduces the likelihood of a strike.
2. **Fact**: A potential major strike by 45,000 dock workers on the U.S. East and Gulf Coasts was averted.
– **Correctness**: True (matches the context of how agreements typically avert strikes).
– **Importance**: Critical, as averting a strike is directly relevant to the question.
– **Relevance**: True, as it confirms the likelihood of a strike has decreased.
3. **Fact**: The threat of a strike had raised concerns about disruptions to supply chains, inflation, and shortages.
– **Correctness**: True (consistent with expectations of strikes causing such concerns).
– **Importance**: Important, provides context to the potential impact of the strike.
– **Relevance**: True, shows why avoiding the strike was significant.
4. **Fact**: A key issue in the negotiations was automation at the ports.
– **Correctness**: True (matches typical contentious issues in port negotiations).
– **Importance**: Important, as unresolved automation issues may influence future disputes.
– **Relevance**: True, relevant since unresolved issues could still lead to disputes.
5. **Fact**: The new agreement is a six-year master contract.
– **Correctness**: True (assuming the detail is factually and internally consistent).
– **Importance**: Important, as the six-year term suggests long-term stability.
– **Relevance**: True, a longer contract can indicate a lower probability of near-term strikes.
6. **Fact**: The agreement follows a short strike in October 2024 that led to increased shipping prices and cargo backlogs.
– **Correctness**: True (internally consistent, assuming a previous short strike occurred).
– **Importance**: Important, provides context for urgency/need of the agreement.
– **Relevance**: True, relevant as it shows a precedent for strike action.
7. **Fact**: Stability at the ports is seen as crucial for preventing economic impacts.
– **Correctness**: True (consistent with the role of ports in trade).
– **Importance**: Important, emphasizes the consequences of port instability.
– **Relevance**: True, relevant as it highlights the broader implications of the agreement.
8. **Fact**: Specific details of the agreement remain undisclosed.
– **Correctness**: True (assuming the information was not available at the summary’s time).
– **Importance**: Lesser, as it doesn’t directly inform the likelihood of a strike.
– **Relevance**: False, since undisclosed details do not directly relate to the strike probability.
Each fact’s analysis helps gauge how the situation mentioned in the summary might influence the probability of a strike in Q1 2025.
Fact-checked summary:
In January 2025, the International Longshoremen’s Association (ILA) and the United States Maritime Alliance (USMX) reached a tentative agreement that directly impacts the probability of a strike, reducing its likelihood. This agreement averted a major strike by 45,000 dock workers on the U.S. East and Gulf Coasts, significantly decreasing concerns about disruptions to supply chains, inflation, and shortages. A key issue in the negotiations was port automation, and while unresolved, it remains relevant to future disputes. The new agreement extends over six years, suggesting long-term stability and a lower probability of near-term strikes. Stability at the ports is crucial for preventing significant economic impacts, emphasizing the importance of the agreement in maintaining economic operations. Additionally, the backdrop of a short strike in October 2024, which caused increased shipping prices and cargo backlogs, underscores the urgency and necessity of reaching the current agreement.
OpenAI gives a base rate of 0.1
The following were considered in order to produce this base rate:
The base rate is primarily influenced by the recent successful negotiation of a tentative agreement and its likely ratification, which significantly reduces the probability of a strike. Historically, once agreements are reached, especially ones that span multiple years (in this case six years), the chance of an immediate breakdown is low. The past short strike indicates some potential for disruption, but the urgency to resolve disputes quickly diminishes the likelihood of a future strike occurring. Additionally, the fact that port automation, while unresolved, did not impede the preliminary agreement, suggests that stakeholders are willing to compromise to avoid strikes.
Some potential divergent considerations that might affect the base rate:
Potential divergent considerations include the unresolved status of port automation, which could become a more contentious issue if either side revises its stance significantly. Furthermore, sudden economic changes, such as severe inflation or recession, might pressure either side to renegotiate terms more aggressively. Internal pressure from union members or corporate shareholders may also play a divergent role if they are dissatisfied with the agreement’s terms. However, given the recent agreement, these factors seem less likely to trigger immediate unrest.
The following chain of events are necessary for the question to resolve positively:
– The tentative agreement between the ILA and USMX must not be ratified by the members. Unlikely
– Significant changes in external economic conditions must occur, prompting a reevaluation of terms. Unlikely
– The key issue of port automation must become irresolvable within the current framework, leading to further disputes. Possible
– Strong opposition from a significant faction within either the ILA or USMX must arise, leading to a breakdown in negotiations. Unlikely.
Querying Claude (AI predicts: 0.15 – confidence: 7)
Querying Mistral (AI predicts: 0.15 – confidence: 7)
Querying OpenAI (AI predicts: 0.15 – confidence: 6)
Question Type: Binary
Median from LLMs: 0.15
Base rate: 0.1 (from OpenAI)
SD: 0
Confidence: 7
Conf Mode: Normal
Mellers: 0.07
Reverse Mellers: 0.24
Theory of Mind: 0.3 (What did the LLMs think other LLMs predicted?)
Beta Distribution: 0.001
Close Type: A (B = cautious # closer to 50%; A/C = closer to extremes)
# LLM responses: 3
Model value: 0.15
We queried Claude, Mistral, OpenAI
A summary of what the responding AIs considered is as follows:
The recent tentative agreement between the International Longshoremen’s Association (ILA) and the United States Maritime Alliance (USMX) has significantly decreased the probability of a strike in the near term, particularly over the agreement’s six-year duration. Such agreements typically lead to long-term stability in labor relations, especially when key issues like salary increases are settled. However, port automation remains a significant unresolved issue that could pose challenges. Historically, unresolved contentious issues like automation have had the potential to disrupt deals, but the willingness to reach a preliminary agreement suggests both parties prefer negotiation over confrontation to avoid economic disruptions similar to past strikes. Nonetheless, factors such as internal union dissent, potential shifts in economic environments, or opposition within either party on unresolved issues might increase the risk of labor unrest. These circumstances, though considered improbable, could necessitate renegotiations and elevate the risk of strikes if not managed appropriately.
How many seats will the Conservative Party win in Canada’s next federal parliamentary election?
Welcome to the Multi-AI Oracle. Today:
* less than 172: 26.32%
* between 172 and 205: 42.11%
* between 206 and 240: 21.05%
* more than 240: 10.52%
Obtaining outside newsfeed. News from various sources. Based on the most recent news articles, here’s a summary of the key developments in Canadian politics:
Most Recent Developments:
Justin Trudeau has announced he will not contest the upcoming federal election in 2025 and may exit politics altogether. This significant announcement comes amid increasing political pressure and declining popularity. Trudeau has served as Prime Minister since November 2015, making him one of the longest-serving leaders of a major Western democracy.
Key Electoral Updates:
– The Leaders’ Debates Commission has established new rules requiring parties to meet 2 of 3 requirements to participate in federal leaders’ debates for the next election
– Former central banker Mark Carney is expected to launch a bid for Liberal Party leadership at an upcoming event in Edmonton
– The Conservative Party, led by Pierre Poilievre, is showing strong momentum in polls as Trudeau’s Liberal Party faces declining support
Political Context:
The upcoming 2025 election is shaping up to be a significant transition point in Canadian politics. There’s speculation about a potential leadership race between Deputy Prime Minister Chrystia Freeland and Mark Carney for the Liberal Party leadership. Meanwhile, the Conservative Party has been gaining support, particularly as they campaign on issues like defunding the CBC and addressing economic concerns.
The news suggests Canada is entering a period of political transition, with the Liberal Party preparing for post-Trudeau leadership while the Conservatives aim to capitalize on growing public support for change.
Summarizing outside newsfeed:
“Recent reports indicate that Canadian politics is poised for a significant transition as Prime Minister Justin Trudeau has announced he will not run in the 2025 federal election and may retire from politics, amid increasing political pressure and waning popularity. The potential shift in leadership has sparked speculation about a race between Deputy Prime Minister Chrystia Freeland and former central banker Mark Carney to lead the Liberal Party, particularly as Carney is rumored to soon announce his candidacy. The Conservative Party, under Pierre Poilievre, is reportedly gaining momentum in polls by advocating for policies such as defunding the CBC and addressing economic issues, as the Liberal Party experiences declining support. Additionally, new rules have been set for party eligibility in future federal leaders’ debates. It is important to approach these developments with caution, as the information may include unsubstantiated rumors.”
Fact-checking news summary:
### Fact List, Evaluations, and Analysis
1. **Fact**: Prime Minister Justin Trudeau has announced he will not run in the 2025 federal election and may retire from politics.
– **Correctness**: False. As of the information up to October 2023, there has been no official announcement from Justin Trudeau stating he will not run in the 2025 election or that he may retire from politics.
– **Importance**: Important. Any decision by Trudeau not to run could significantly impact the political landscape and the outcome of the election, affecting how many seats each party, including the Conservatives, might win.
– **Relevance**: True. This directly affects the dynamics of the election and therefore is relevant to predicting seat outcomes.
2. **Fact**: There is speculation about a race between Chrystia Freeland and Mark Carney to lead the Liberal Party.
– **Correctness**: True, up to a point. Speculation about potential leadership contests often occurs amid uncertainty about a sitting leader’s future, but there is no formal race or candidacy as of October 2023.
– **Importance**: Important. The leadership of the Liberal Party is significant in shaping party policies and voter appeal, impacting election results.
– **Relevance**: True. Leadership changes could influence election dynamics.
3. **Fact**: Mark Carney is rumored to soon announce his candidacy for leadership of the Liberal Party.
– **Correctness**: Potentially true as a rumor. It has been rumored but not confirmed as of October 2023.
– **Importance**: Less important. While interesting, rumors of candidacy announcements alone don’t alter political strategies until confirmed.
– **Relevance**: True. If Carney runs, it could affect Liberal support and thus seat calculations.
4. **Fact**: The Conservative Party, under Pierre Poilievre, is reportedly gaining momentum in polls by advocating for policies such as defunding the CBC and addressing economic issues.
– **Correctness**: True. Pierre Poilievre has been known to focus on such policies, and reports of Conservative momentum have been noted.
– **Importance**: Critical. Polling momentum and policy popularity are key indicators of potential seat wins in an election.
– **Relevance**: True. Polling and policy influence voter behavior and are directly relevant to election outcomes.
5. **Fact**: The Liberal Party is experiencing declining support.
– **Correctness**: True, according to various reports and analyses up to October 2023. The Liberal Party has faced challenges maintaining its previous levels of support.
– **Importance**: Critical. Declining support for the Liberals directly impacts their seat projections and the Conservative Party’s chances.
– **Relevance**: True. Changes in Liberal support levels affect overall election dynamics.
6. **Fact**: New rules have been set for party eligibility in future federal leaders’ debates.
– **Correctness**: True. It is common for debate rules to be updated ahead of election cycles, though specifics would need verification.
– **Importance**: Unimportant. While debate rules can impact public perceptions of the parties, they don’t directly predict seat outcomes.
– **Relevance**: False. Eligibility rules for debates don’t necessarily influence seat count predictions.
7. **Fact**: The information may include unsubstantiated rumors.
– **Correctness**: True. Reports and summaries often contain a mix of verified facts and speculative rumors.
– **Importance**: Important. Awareness of rumor versus fact is necessary for accurate analysis.
– **Relevance**: True.
Acknowledging rumors ensures more critical consumption of information, which is crucial for accurate election forecasting.
This analysis highlights which factors are significant and relevant to predicting the number of seats the Conservative Party might win in Canada’s next federal election, based on the provided summary.
Fact-checked summary:
In evaluating the factors relevant to predicting the number of seats the Conservative Party might win in Canada’s next federal election, several key points emerge. First, under Pierre Poilievre’s leadership, the Conservative Party is reportedly gaining momentum in polls by advocating policies such as defunding the CBC and addressing economic issues. This momentum is considered critical since polling trends and policy popularity heavily influence potential seat outcomes. Additionally, the Liberal Party is experiencing declining support according to various analyses up to October 2023, which is also critical as shifts in Liberal support directly affect their seat projections and impact the Conservative Party’s chances. These elements are directly relevant to understanding the election dynamics and predicting the possible distribution of parliamentary seats.
Querying Claude (AI predicts: [“less than 172”: 0.35, “between 172 and 205”: 0.4, “between 206 and 240”: 0.2, “more than 240”: 0.05] – confidence: 5)
Querying Mistral (AI predicts: [“less than 172”: 0.25, “between 172 and 205”: 0.45, “between 206 and 240”: 0.2, “more than 240”: 0.1] – confidence: 6)
Querying OpenAI (AI predicts: [“less than 172”: 0.15, “between 172 and 205”: 0.3, “between 206 and 240”: 0.4, “more than 240”: 0.15] – confidence: 6)
Question Type: Multiple Choice
Confidence: 6
# LLM responses: 3
Model value:
* less than 172: 0.2632
* between 172 and 205: 0.4211
* between 206 and 240: 0.2105
* more than 240: 0.1052
We queried Claude, Mistral, OpenAI
A summary of what the responding AIs considered is as follows:
The AIs suggest that the Conservative Party, led by Pierre Poilievre, is gaining momentum in polls due to key policy stances and declining support for the Liberal Party as of late 2023. Despite this momentum, historical trends show that Conservative seat counts rarely exceed 200, suggesting a moderate victory is more likely. Key factors influencing this prediction include the political climate, economic issues resonating with voters, and the challenges posed by Canada’s electoral system and regional voting patterns. Nonetheless, the prediction may not hold due to the potential unreliability of polls, possible shifts in regional voting behavior, unforeseen political events, and the chance of a Liberal Party recovery before the election. This leads to a potential outcome where the Conservatives secure between 172 and 205 seats.
Runtime: 185 seconds fo rall forecasts on this page.
Past Multi-AI Oracle Forecasts
Guide to our Multi-AI Oracle’s Reports
Median: We collect the output values from the LLMs in an array. We then take the median value. This is between 0 and 1 (i.e. 0.15 = 15%).
Base rate: Currently we take OpenAI’s value as the base rate. We are working on a better value using a different prompt that takes into account process. The base rate helps to determine if the median makes sense.
SD: The standard deviation between the median and the base rate.
Confidence: We query each of the LLMs on how confident they are of their predictions (between 0 and 10) and take a median. Because the LLMs tend to be overconfident, we take anything lower than 6 as being low confidence. This factors into the overall model.
Conf Mode: Based on the confidence value. >=9 is high confidence. Below 6 is low confidence (this is also triggered by an exceptionally high SD).
Mellers: This refers to Barbara Mellers, specifically a paper she wrote that includes a formula for moving values towards an extreme (i.e. 0 or 1).
Reverse Mellers: This uses the formula from above, but with a sub-1 coefficient to move the values closer to 50%.
Theory of Mind: We ask the LLMs what they think other LLMs would predict. We hope that this makes them consider the questions more deeply.
Beta Distribution: Currently unused, but possibly of interest. This is based on the median, the base rate and the SD.
Close Type: We noticed that the appropriate base case for some questions is closer to the extremes, while others are closer to 50%. When we have a low confidence value, this helps us to determine whether to extremize or de-extremize the value. ‘A’ implies closer to zero. ‘B’ implies closer to 50%. ‘C’ implies closer to 100%.
# LLM Responses: The count of LLMs that responded. We query 5 currently, but they have a relatively high API failure rate.
Model value: For normal confidence mode, this is the median. For high confidence, this is the Mellers value. For low confidence, this is either the Mellers or Reverse Mellers (depending on the Close Type).