Writing a quality rationale–a clear explanation that gives context behind your probabilistic forecast–can be challenging. In this post, we explore how you can use an AI tool like ChatGPT or Claude to gain a quick understanding of a complex forecast topic as well as improve the logical support behind your rationale.
AI tools can be valuable references in your initial research before coming up with your first forecast. They can help to summarize underlying context around an issue, give meaning to technical terms you may be unfamiliar with, or provide data on previous similar cases (base rates). Since you can ask follow up questions to the AI, they are often more useful for gaining a deeper understanding of a new topic compared to search engines, which might only direct you to sources that are either overly technical or insufficiently informative.
AI tools can also summarize lengthy, complex source materials. If you provide the AI with reports, academic studies, or news articles relevant to the forecast, then the AI can extract the most salient points and write them in an easily understandable format. This allows you to quickly incorporate key facts and data as evidence to formulate your forecast, without having to pore through dozens of pages of reports yourself.
Perhaps the greatest value an AI tool provides is in improving the written logic, or argumentation, in your rationale. Argumentation, in general, relies on proving a claim by using reasoning which proves that the claim is true. This reasoning, especially in forecasting, often incorporates evidence from data, news reporting, etc. You can probe the AI about these concepts to improve the argumentative strength and rigor of your rationale.
Here’s an example of using an AI (Claude, by Anthropic) for rationale feedback.
Author: Below is a forecast value and forecast rationale I will submit in response to the question "Will Vladimir Putin cease to be the president of the Russian Federation before 1 March 2023?" The forecast will be submitted to a crowdsourced forecasting site on May 16, 2022. List the logical gaps in the rationale that I may have overlooked.
Rationale for why 11%:
These outcomes are not all mutually exclusive, so I’m adding up the first 2 and adding half of each of the last 2 (as a rough estimate) to generate the forecast probability.
Claude: Here are some potential logical gaps I see in the rationale provided:
Overall the rationale provides a good starting point covering multiple angles, but could be strengthened by citing more evidence, avoiding arbitrary percentages, and mapping out dependencies between scenarios. Explaining the logic behind combining probabilities would also help justify the final 11% forecast.
In this example, Claude provided some actionable feedback, such as saying I should articulate reasons for the probabilities of the various scenarios I thought would be most likely to trigger a positive question resolution, which probably would have improved my model of the question topic. On the other hand, it also provided some weaker feedback; while Claude said that Putin might face electoral defeat, the next presidential election is not scheduled until after the end of the question period. It also included obvious or generic advice that wasn’t usable, like the last paragraph of the output.
Another area where the AI can be useful is in scanning your rationale and helping you to incorporate an argumentative writing technique called "signposting". Functioning like road signs for your writing, these are explicit signals that guide the reader through the structure and progression of the argument. You'll often see top forecasters use signposts like "When looking at base rates…" or "My forecast may be wrong due to…" This helps a reader, and the rationale writer themselves, see how the different pieces of the rationale fit together. An AI could review a draft of the forecaster’s rationale and suggest where signposts might be added to improve the clarity of the argument. By identifying these potential weak spots, the AI allows you to shore up your rationale preemptively.
As you work with AI to improve your rationales, be sure to refer to your forecasting site’s policy on using AI tools. We encourage the use of these tools responsibly, making sure you don’t take all of its information as fact without checking evidence, giving credit appropriately to the AI model used, and they should in no way replace your own unique perspective and authorship. AI tools can be of value as illustrated in this post – both for personal development and for the benefit of the broader forecasting community.