Introducing RSO-F

We are building RSO-F to forecast real-world events. The model estimates how likely something is to happen within a defined period, across subjects ranging from customer demand to economic change and world affairs.

A probability gives someone a way to express an expectation before the outcome is known. It can be compared with another estimate, reconsidered as information changes and evaluated once the event resolves. RSO-F is being developed to make those estimates useful across different subjects and time horizons.

Defining the question

Forecasting starts with a question precise enough to answer. An expectation that demand will rise needs a period and a basis for comparison. A forecast of disruption needs to specify what would count as disruption and when the outcome will be checked.

Consider a business preparing for a seasonal increase in orders. Whether orders will exceed its existing capacity is a different question from whether sales will grow at all. Both concern demand, but they could lead to different operational decisions. The forecast has to match the question the business needs to resolve.

Measuring accuracy

We are evaluating RSO-F on ForecastBench, where forecasts are submitted before the events resolve. As outcomes become known, those predictions can be scored and compared with other models. This lets us examine estimates made while the answer was still uncertain.

We look at performance across questions and time horizons, as well as calibration. Calibration concerns whether events assigned a given probability occur about that often across enough forecasts. It helps us examine the confidence of the model alongside its ability to distinguish more likely outcomes from less likely ones.

A result needs its context. The subjects covered, the period evaluated and the questions that have resolved all affect what a comparison can tell us. We will publish results with those details and the relevant comparisons. Applications outside that evaluation will need evidence appropriate to their own use.

Using a forecast

A forecast informs a decision together with its costs and consequences. In the capacity example, one business may be able to accommodate extra orders at short notice. Another may need to commit to equipment well in advance. The same estimate can support different choices because the businesses face different constraints.

We are developing RSO-F for use through an API, so forecasts can appear within the products where those choices are made. Potential applications include planning tools and financial analysis, where an estimate can be considered alongside the information specific to that decision.

Working with predictions

Our interface work focuses on what people can do with an estimate once they have it. A dynamic document could carry a forecast into a calculation and show how changing an assumption affects the result. A conversational interface could help someone examine alternatives or clarify the question they want to ask.

We are developing those interfaces alongside the model. For a planning decision, the useful view might include the forecast, the capacity already available and the cost of changing it. Keeping those relationships visible would let someone examine the basis for a choice before committing to it.