In many of the forecasting tournaments that we run, we receive some form of the question: I made a forecast in a question -- why didn't I receive a score? Typically, forecasters in this situation will see something like:
The scoring system in Cultivate Forecasts utilizes a variant of a Brier score as the measurement of accuracy. When a question is resolved/judged, the scoring system will generate a score for each day that the question was open. For an in depth explanation of the scoring system, you can read What is a Brier Score and How is it Calculated? and What are Net Brier Points and how are they calculated?
Because the basic, underlying unit of our accuracy score is a daily Brier score, the system only generates scores for full days that the question was open for forecasting. This can result in scenarios such as:
To avoid scenarios like this, make sure to make your forecasts early and often! In addition to ensuring your forecast is received before the scoring cutoff, research has shown that forecasters who update their forecasts regularly tend to be more accurate.
|More support articles|