Supervised predictive modeling¶

Supervised learning uses the other features of your dataset to make predictions. Unsupervised learning uses unlabeled data to surface insights about patterns in your data. The supervised learning setup is described below.

Create basic¶

Follow the steps below to create a new experiment from within a Use Case.

Create a feature list¶

Before modeling, you can create a custom feature list from the data explore page. You can then select that list during modeling setup to create the modeling data using only the features in that list. Learn more about feature lists post-modeling here.

Add experiment¶

From within a Use Case, click Add and select Experiment. The Set up new experiment page opens, which lists all data previously loaded to the Use Case.

Add data¶

Add data to the experiment, either by adding new data (1) or selecting a dataset that has already been loaded to the Use Case (2).

Once the data is loaded to the Use Case (option 2 above), click to select the dataset you want to use in the experiment. Workbench opens a preview of the data.

From here, you can:

Start modeling setup¶

Once you have proceeded, Workbench prepares the dataset for modeling (EDA 1).

Set learning type¶

When EDA1 finishes, Workbench progresses to the modeling setup. First, set the learning type.

Set target¶

If using supervised mode, set the target, either by:

Depending on the number of values for a given target feature, DataRobot automatically determines the experiment type—either regression or classification. Classification experiments can be either binary (binary classification) or more than two classes (multiclass). The following table describes how DataRobot assigns a default problem type for numeric and non-numeric target data types:

With a target selected, Workbench displays a histogram providing information about the target feature's distribution and, in the right pane, a summary of the experiment settings.

From here you can:

• Change a regression experiment to a multiclass experiment.

• Click Next to view Additional settings, where you can build models with the default settings or modify those settings.

• For multiclass classification experiments, click Show more classification settings to further configure aggregation settings.

If using the default settings, click Start modeling to begin the Quick mode Autopilot modeling process.

Regression targets¶

Regression experiments are those where the target is a numeric value. The regression prediction problem predicts continuous values (e.g., 1.7, 6, 9.8...) given a list of input variables (features). Examples of regression problems include financial forecasting, time series forecasting, maintenance scheduling, and weather analysis.

Regression experiments can also be handled as classification by changing the target type from Numeric to Classification.

To change a regression problem (numeric target) to classification, change the radio button identifying the target type:

Changing the target type enables the multiclass configuration options. If there are more than 1000 numeric values (classes) for the target, the Aggregate low-frequency classes option, described below, is enabled by default.

Classification targets¶

In a classification experiment, the model groups observations into categories by identifying shared characteristics of certain classes. It compares those characteristics to the data you're classifying and estimates how likely it is that the observation belongs to a particular class. Classification projects can be binary (two classes) or multiclass (three or more classes).

Configuration of a classification experiment depends on the type (number of classes), which is reported under the target feature entry as either Target type: Binary classification or Target type: Classification, in which case the number of classes is also reported:

A multiclass confusion matrix helps to visualize where a model is, perhaps, mislabeling one class as another.

Customize basic settings¶

Changing experiment parameters is a good way to iterate on a Use Case. Before starting to model, you can change a variety of settings:

Change modeling mode¶

By default, DataRobot builds experiments using Quick Autopilot. However, you can change the modeling mode to train specific blueprints or all applicable repository blueprints.

The following table describes each of the modeling modes:

Change optimization metric¶

The optimization metric defines how DataRobot scores your models. After you choose a target feature, DataRobot selects an optimization metric based on the modeling task. Typically, the metric DataRobot chooses for scoring models is the best selection for your experiment. To build models using a different metric, overriding the recommended metric, use the Optimization metric dropdown:

See the reference material for a complete list and descriptions of available metrics.

Change feature list (pre-modeling)¶

Feature lists control the subset of features that DataRobot uses to build models. Workbench defaults to the Informative Features list, but you can modify that before modeling. To change the feature list, click the Feature list dropdown and select a different list:

You can also change the selected list on a per-model basis once the experiment finishes building.

Set additional automation¶

Before moving to advanced settings or beginning modeling, you can configure other automation settings.

After the target is set and the basic settings display, expand Show additional automation settings to see additional options.

Train on GPUs¶

For datasets that include text and/or images and require deep learning models, you can select to train on GPUs to speed up training time. While some of these models can be run on CPUs, others require GPUs to achieve reasonable response time. When Allow training on GPUs is selected, DataRobot detects blueprints that contain certain tasks and includes GPU-supported blueprints in the Autopilot run. Both GPU and CPU variants are available in the repository, allowing a choice of which worker type to train on; GPU variant blueprints are optimized to train faster on GPU workers. Notes about working with GPUs:

• Once the Leaderboard populates, you can easily identify GPU-based models using filtering.
• When retraining models, the resulting model is also trained using GPUs.
• When using Manual mode, you can identify GPU-supported blueprints by filtering in the blueprint repository.
• If you did not initially select to train with GPUs, you can add GPU-supported blueprints via the repository or by rerunning modeling.

• Models trained on GPUs are marked with a badge on the Leaderboard:

  

Start modeling¶

Once all settings are applied, you can begin model training. To do so, click Next and either:

• Click Start modeling to begin the Quick mode predictive modeling process.

  

• Customize more advanced settings.

• Change the project's target or feature list before modeling by returning to the Target page. To return, click the target icon, the Back button, or the Target field in the summary:

  

When training begins, DataRobot runs EDA2 and further evaluates the data. Use the queue in the far-right panel to:

• Control the number of workers applied to the experiment. Increase or decrease workers for your experiment as needed. (See troubleshooting tips, if needed.)

  

• View the jobs that are running, queued, and failed.

Expand the queue, if necessary, to list the running jobs and assigned workers.

What's next?¶

After you start modeling, DataRobot populates the Leaderboard with models as they complete. You can:

• Use the View experiment info option to view a variety of information about the experiment.
• Begin model evaluation on any available model.