# Segmented modeling
URL: https://docs.datarobot.com/en/docs/classic-ui/modeling/time/ts-segmented.html

> Describes segmented modeling for multiseries projects in DataRobot.

# Segmented modeling for multiseries

Complex and accurate demand forecasting typically requires deep statistical know-how and lengthy development projects around big data architectures. DataRobot's multiseries with segmented modeling automates this requirement by creating multiple projects—"under the hood." Once the segments are identified and built, they are merged to make a single-object—the Combined Model. This leads to improved model performance and decreased time to deployment.

When using segmented modeling, DataRobot creates a full project for each segment—running Autopilot (full or quick) then selecting (and preparing) a recommended model for deployment. (See the note on using segmented modeling in [Manual](https://docs.datarobot.com/en/docs/classic-ui/modeling/time/ts-segmented.html#manual-mode-in-segmented-modeling) mode.) DataRobot also marks the recommended model as "segment champion," although you can [reassign the champion](https://docs.datarobot.com/en/docs/classic-ui/modeling/time/ts-segmented.html#reassign-the-champion-model) at any time.

> [!NOTE] Note
> Although DataRobot creates a project for each segment, these projects are not available from the [Project Control Center](https://docs.datarobot.com/en/docs/classic-ui/modeling/manage-projects.html). Instead, they are investigated and managed from within the Combined Model, which is available in the Project Control Center.

DataRobot incorporates each segment champion to create the Combined Model —a main umbrella project that acts as a collection point for all the segments. The result is a “one-model" deployment—the segmented project—while each segment has its own model running in the deployment behind the scenes.

It is important to remember that while segmented modeling solves some problems (model factories, multiple deployments), it cannot know which segments you care most about or which have the highest ROI. To be successful, you must correctly define the use case, set up the dataset, and define the segments.

See the [segmented modeling FAQ](https://docs.datarobot.com/en/docs/reference/pred-ai-ref/ts-reference/segmented-faq.html) for more detailed information. See the [visual overview](https://docs.datarobot.com/en/docs/reference/pred-ai-ref/ts-reference/segmented-qs.html) for a quick representation of why to use segmented modeling.

Modeling with segmentation is available for multiseries regression projects.

## Segmented modeling workflow

Time series segmented modeling requires, first, defining segments that divide the dataset. To define segments, you can allow DataRobot to:

- Discover clusters in your data and then use those clusters as segments.
- Assign segments for you based on the configured segment ID.

To build a segmented modeling project:

1. Follow the standardtime series workflow—set the target and turn on time-aware modeling. ChooseAutomated time series forecastingas the modeling method.
2. Enable multiseries modeling bysetting the series identifier.
3. Set the segmentation method by clicking the pencil:
4. Set whether to enable segmented modeling:
5. Set how segments are defined. The table below describes each option: OptionDescriptionID columnSelect a column from the training dataset that DataRobot will use as the segment ID. Start to type a column name and see matching auto-complete selections or select from the identifiers that DataRobot identified. Segment ID must be different than series ID (see note below).Existing clustering modelUse aclustering model previously savedto the Model Registry.New clustering modelStart a new clustering project, with results later applied via theExisting clustering modeloption, by clickingtime series clusteringlink in the help text. What if I want to have one series per segment?The columns specified for segment ID and series ID cannot be the same; however, you can duplicate the series ID column and give it a new name. Then, set the segment ID to the new column name (using theHow are the segments definedsection). DataRobot will generate the segments using the series ID.
6. Once the method is selected—either the ID is set or an existing clustering model is selected—clickSet segmentation method. TheTime Series Forecastingwindow returns, where you can thencontinue the configuration—training windows, duration,KA, and calendar selection—including changing the selected series and segment. How are the training periods determined if clustering was used?When building a segmented model usingfound clustersto split the dataset into the child projects (segments), DataRobot applies the training window settings from the clustering project to the segmented modeling project. This protects the holdout in segmented modeling and prevents data leakage from the clustering model when splitting the segmented dataset into child projects. Using the start and end dates of each series, the general scenarios that affect the methodology:If the series data contains the time window needed (as defined in the clustering project), DataRobot simply passes the series data along.Series data before the clustering training end: If there is a series that is shorter than the full training window and extends past holdout, DataRobot only uses data points before the clustering end that is the size of the training duration (only the portions that exist within the training boundary).Series data has only data older in time than the clustering training end: If there is a legacy series in which its data does not fall into the training window, DataRobot "slides back" and gathers data for thedurationof the training window so it can be used in segmentation and not lost.Series data only exists "newer" in time than the clustering training end: If a series only exists in holdout, DataRobot slides the window forward but does not select any data that was used in training. In this way, the data is not dropped, but it is only used for examining the holdout of a child project.
7. When the configuration is ready, select Quick or full Autopilot, orManualmode, and clickStart. DataRobot prompts to remind you that because it builds a complete project for each segment, the time required to finish modeling could be quite long. Confirm you want to proceed by clickingStart modeling. (You can set DataRobot to proceed without approval for future segmented projects.)
8. AfterEDA2completes, DataRobot immediately creates the Combined Model. Because the "child" models (the independent segment models) are still building, the Combined Model is not complete. However, you cancontrol building and worker assignmentfrom the Combined Model.
9. When modeling completes, use the Combined Model to explore segments.

## Explore results

Once modeling has finished, the Model tab indicates that one model has built. (See the [note](https://docs.datarobot.com/en/docs/classic-ui/modeling/time/ts-segmented.html#manual-mode-in-segmented-modeling) regarding outcome when using Manual mode.) This is the completed Combined Model.

The charts and graphs available for segmented modeling are dependent on the model type:

- For the Combined Model, you can access theSegmentation, a model blueprint, the modeling log,Make Predictions, andComments.
- For the models available in the individual segments, the visualizations and modeling tabs (Repository, Compare Models, etc.) appropriate to a multiseries regression project are available.

### Segmentation tab

Click to expand the Combined Model and expose the Segmentation tab.

The following table describes components of the Segmentation tab:

| Component | Description |
| --- | --- |
| Search | Use search to change the display so that it only includes segments that match the entered string. |
| Download CSV | Download a spreadsheet containing the metadata associated with the Combined Model project, including metric scores, champion history, IDs, and project history. |
| Segment | Lists the segment values, found in the training data by DataRobot, in the specified segment ID. |
| Rows | Displays segment statistics from the training data—the raw number of rows and the percentage of the dataset that those rows represent. |
| Total models | Indicates the number of models DataRobot built for that segment during the build process. |
| Champion last updated | Indicates the time and the responsible party for the last segment champion assignment. The entry also provides an icon indicating the champion model type. Initially, all rows will list by DataRobot. Segments are listed by the "All backtests" scores; click the column header to re-sort. |
| Backtest 1 | Indicates the champion model's Backtest 1 score for the selected metric. |
| All backtests | Indicates the average score for all backtests run for the champion model. |
| Holdout | Provides an icon that indicates whether Holdout has been unlocked. |

## Explore segments

The Combined Model is comprised of one model per segment—the segment champion. The individual segments, on the other hand, comprise a complete project. You can investigate the project from the segment's Leaderboard and even deploy a segment model, independent of the Combined Model.

### Access a segment's Leaderboard

There are multiple ways to access a segment's Leaderboard.

#### From the Combined Model

Expand the Combined Model and click the segment name in the Segmentation tab list.

Once clicked, the segment's Leaderboard opens. Notice that:

| Indicator | Description |
| --- | --- |
| (1) | A full set of models has been built. |
| (2) | DataRobot has recommended a model for deployment and marked a model as champion. |
| (3) | Regular Worker Queue controls are available. |

#### From the Segment dropdown

Use the Segment dropdown to change your view.

- From a segment:
- From the Combined Model, select a segment to open the segment's Leaderboard.

### Reassign the champion model

While DataRobot initially assigns a segment champion, you may want to change the designation. This could be the case, for example, if it were important to you that all segments provide the same model type to the Combined Model. Identify the segment champion from a segment's Leaderboard, where it is marked with the champion badge:

To reassign the champion, from the segment Leaderboard, select the model you want as champion. Then, from the menu select Leaderboard options > Mark model as champion.

The badge moves to the new model:

And the Combined Model's Segmentation tab shows when the champion was last updated and who assigned the new champion.

## Control across projects

Because DataRobot treats each segment as an individual project, completing the Combined Model can take significantly longer than a regular multiseries project. The exact time is dependent on the number of segments and size of your dataset. You can use the controls described below to set workers (1) and to stop and start modeling (2). All actions are performed from the Worker Queue of the Combined Model and apply to all segment projects. You can also use it to [unlock Holdout](https://docs.datarobot.com/en/docs/classic-ui/modeling/time/ts-segmented.html#global-holdout-unlock) (3).

### Worker control

From the Combined Model, you can control the number of modeling workers across all segment projects. DataRobot automatically re-balances workers between segments, distributing available workers between running segments as each segment completes modeling. When changing the worker count, DataRobot ignores any projects not in the modeling stage.

### Pause/Start/Stop child modeling

From the Worker Queue of the parent segmented project, you can control modeling actions of the child projects. Use the stop/start/cancel buttons in the sidebar, and the selected action is applied to all child projects simultaneously. Specifically:

- At the start of a segmented project, no queue actions are available.
- When all segments have reached theEDA2stage, the Pause and Start buttons become available.
- The Cancel button becomes available when child projects are in the modeling stage and have at least one job running.

### Unlock Holdout

You can [unlock Holdout](https://docs.datarobot.com/en/docs/classic-ui/modeling/build-models/build-basic/unlocking-holdout.html) for an entire project or for each segment.

- To unlock the entire project—all models in all segments—chooseUnlock Holdoutfrom the Combined Model's Worker Queue.
- To unlock Holdout for all models in a segment, (open the segment's Leaderboard) and chooseUnlock project Holdout for all modelsin the Worker Queue.

### Leaderboard model scores

Scores for the Combined Model are updated when a model completes building (champions are assigned in all segments). Scores are recalculated any time a segment's champion is replaced. For efficiency, it is calculated by aggregating individual champion scores. Metrics that support this method of calculation are: MAD, MAE, MAPE, MASE, RMSE, RMSLE, SMAPE, and Theil’s U.

When one or more champions are prepared for deployment, the scores shown reflect the parent scores. (The parent of the champion/recommended model is the model the champion is trained from.)

When looking at the Combined Model on the Leaderboard, you may notice that the model has no score (only N/A), which indicates:

- The model is not yet complete.
- All backtests have not yet completed for one or more segment champions (the All Backtests score is N/A).
- The selected metric does not support score aggregation (for example, FVE Poisson, Gamma Deviance, etc.). For example: Change the metric to a supported metric, for example MAE, and an aggregated score from the Combined Model becomes available.

To see the individual champion scores, expand the Combined Model to display the Segmentation tab.

Individual champion scores are reported there. Note that:

- Both the Backtest 1 and All Backtests scores are N/A if the champion is not assigned.
- An asterisk indicates that the champion model has been prepared for deployment (retrained as a start/end model into the most recent data) and thus the scores of it's parent model are used.

If you change the champion, DataRobot passes the scores from the new champion (or its parent) to the Combined Model.

## Manual mode in segmented modeling

When using Manual mode with segmented modeling, DataRobot creates individual projects per segment and completes preparation as far as the modeling stage. However, DataRobot does not create per-project models. It does create the Combined Model (as a placeholder), but does not select a champion. Using Manual mode is a technique you can use to have full manual control over which models are trained in each segment and selected as champions, without taking the time to build the models.

## Deploy a Combined Model

To fully leverage the value of segmented modeling, you can deploy Combined Models like any other time series model. After selecting the champion model for each included project, you can deploy the Combined Model to create a "one-model" deployment for multiple segments; however, the individual segments in the deployed Combined Model still have their own segment champion models running in the deployment behind the scenes. Creating a deployment allows you to use [DataRobot MLOps](https://docs.datarobot.com/en/docs/api/dev-learning/python/mlops/index.html) for accuracy monitoring, prediction intervals, challenger models, and retraining.

When segmented modeling completes, you can deploy the resulting Combined Model:

1. Once Autopilot has finished, theModeltab contains one model. This model is the completed Combined Model.
2. Click theCombined Model, and then clickPredict > Deploy.
3. On theDeploytab, clickRegister to deploy.
4. Configure themodel registration settingsand then clickAdd to registry. The model opens on theModel Registry > Registered Modelstab.
5. While the registered model builds, clickDeployand thenconfigure the deployment settings.
6. Monitor, manage, and govern the deployed model inDataRobot MLOps. Set upretraining policiesto maintain model performance post-deployment.

### Combined Model deployment considerations

Consider the following when working with segmented modeling deployments:

- Time series segmented modeling deployments do not support data drift monitoring.
- Automatic retraining for segmented deployments that use clustering models is disabled; retraining must be done manually.
- Retraining can be triggered by accuracy drift in a Combined Model; however, it doesn't support monitoring accuracy in individual segments or retraining individual segments.
- Combined model deployments can include standard model challengers.

## Modify and clone a deployed Combined Model

After deploying a Combined Model, you can change the segment champion for a segment by cloning the deployed Combined Model and modifying the cloned model. This process is automatic and occurs when you attempt to change a segment's champion within a deployed Combined Model. The cloned model you can modify becomes the Active Combined Model. This process ensures stability in the deployed model while allowing you to test changes within the same segmented project.

> [!NOTE] Note
> Only one Combined Model on a project's Leaderboard can be the Active Combined Model (marked with a badge).

To modify and clone a deployed Combined Model, take the following steps:

1. Once a Combined Model is deployed, it is labeledPrediction API Enabled.
2. Click the active and deployedCombined Model, and then in theSegmentstab, click the segment you want to modify.
3. Reassign the segment champion.
4. In the dialog box that appears, clickYes, create new combined model.
5. On the project'sLeaderboard, you can access and modify theActive Combined Model. TipWhile theCombined Model updatednotification is visible, you can clickGo to Combined Modelto return to the segment's Combined Model in theLeaderboard.

---
