Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

Nowcasting

Nowcasting is a method of time series modeling that predicts the current value of a target based on past and present data. Technically, it is a forecast window in which the start and end times are 0 (now). Nowcasting builds an explanatory model that can describe latent present conditions and factors that contribute to a particular on-going behavior. In other words, based on the current input values and recent history, what is the target right now? For example, in an anomaly detection project you may want to answer the question, "is the observation I see right now an anomaly?"

Forecasting, by contrast, is the practice of predicting future values based on past and present data. In other words, by using information at a given time, you can predict values in a future row. Target values from later rows are aligned with feature values from past rows.

Some sample uses for nowcasting:

  • Manufacturing and financial markets perform “fair value” modeling. For example, the Federal Reserve builds and publishes nowcasts that drive short-to-medium term policies and market changes.
  • Explain natural gas prices under various conditions.
  • Estimate an economic indicator before it is reported.
  • Understand the drivers of something when time dynamics matter.
  • Forecast a fixed point from multiple past points (for example, daily EOM-only forecast).
  • Generate more timely current condition estimates when data readings or traditional computations are delayed (useful in weather modeling).

Any time series forecasting models that follow the forecast distance framework can be used for nowcasting. The standard time series insights also apply to nowcasting. Feature Impact is especially useful for nowcasting as it can provide good insight into the important features that may explain observed current values. See the sections below for more information.

Use nowcasting

The nowcasting workflow follows the same steps as the forecasting method documented in the time series workflow overview (set target, date/time feature, and, as applicable, series ID). The following describes the process once you have selected nowcasting.

FDW settings

Nowcasting applies forecast window (FW) settings of [0, 0] for the forecast start and end times. Additionally, the Feature Derivation Window (FDW) end is set at a single time step prior to the current time step, allowing DataRobot to derive additional features for the target, such as rolling statistics (lags), without risking target leakage.

Notice that in contrast to the forecasting image representation that illustrates the past and future date range selected, nowcasting shows only the past rolling window.

Features known in advance

By default, DataRobot marks all covariate (non-target) features as known in advance (KA). This helps build better models because it provides guardrails for accuracy. Most commonly, all values in a nowcasting project are known in advance (because you are predicting "now", not "tomorrow"). It would be an exception to the rule to have data that you don't know now.

DataRobot reports the number of features that are KA:

Click the pencil icon () to open the Time Series > Add features as known in advance advanced option and adjust the list of features.

Derive features from target

With nowcasting, DataRobot derives features from the target by default, enabling automatic time-based feature engineering. While this is a technique used with covariates to prevent the risk of target leakage, disabling target-derived features with nowcasting potentially limits the performance and selection of available blueprints. Additionally, for multiseries projects, target derivation supports calculating features from other series.

When checked, features are derived from the target. This setting is linked to the Time Series > Exclude features from derivation advanced option setting. If you deselect the box, the target feature name is added to the feature list in that field:

More info...

The following sections provide background information to help understand the application of nowcasting.

Addition of target-derived features

The nowcasting capability, in comparison to simply setting the FW start and end times to [0, 0], provides a variety of benefits:

  • More features lists created.
  • More blueprints available for selection.
  • Large increase in derived features.
  • Target-derived features are available.

Additionally, with cross series enabled, the set of derived features is richer still, and cross-series blueprints become available.

How nowcasting works

When nowcasting is selected as the time-aware modeling method and EDA2 begins, DataRobot automatically marks non-target features (covariates) as known in advance (KA). This allows real-time features (for example, latest transaction volume of stock) to predict the target (latest known price index). You can choose your desired FDW setting and mark covariates as known in advance.

DataRobot provides guardrails to prevent the automatically derived features from resulting in target leakage. Specifically, target-derived feature lags are inferred from the FDW end, which is prior to the present point in time. This ensures that the most recent derived rolling statistics do not result in target leakage (otherwise, trained models would suggest unrealistic performance).

Nowcasting framework

The standard time series framework (forecasting) is described in the time series section and, once gaps inherent to time series problems are added, can be illustrated like this:

With nowcasting, that illustration changes a bit:

Given a time series dataset:

Time Inputs Target
2009 1.23 9.9
2010 1.41 10.0
2011 2.09 9.82
2012 1.31 7.99
2013 0.31 8.54
2014 3.09 7.42
2015 4.12 4.01
2016 5.91 6.73

For forecasting, DataRobot creates derived time series features and a forecast target as follows:

Time Forecast point Distance Target
2010 2009 +1 year 10.0
2011 2009 +2 year 9.82
-- -- --
2011 2010 +1 year 9.82
2012 2010 +2 year 7.99
-- -- --
2012 2011 +1 year 7.99
2013 2011 +2 year 8.54

For nowcasting, DataRobot creates derived time series features and a forecast target as follows:

Time Forecast point Distance Target
2009 2009 +0 year 9.9
2010 2010 +0 year 10.0
2011 2011 +0 year 9.82
2012 2012 +0 year 7.99
2013 2013 +0 year 8.54
2014 2014 +0 year 7.42
2015 2015 +0 year 4.01
2016 2016 +0 year 6.73

Updated September 20, 2022