View model insights¶

Feature Impact, available for DataRobot and custom models (not agent-monitored external models), provides a high-level visualization that identifies which features are most strongly driving model decisions. It informs:

• Which features are the most important—is it demographic data, transaction data, or something else driving model results? Does it align with the knowledge of industry experts? By understanding which features are important to model outcomes, you can more easily validate if the model complies with business rules.

• Are there opportunities to improve the model? For example, there may be features with negative accuracy. Dropping them by creating a new feature list might increase model accuracy and speed. Some features may have unexpectedly low importance, which may be worth investigating. Is there a problem in the data? Were data types defined incorrectly?

To access the Feature Impact insight, in the Registry > Model directory, open a registered model version and click the Insights tab. Then, open the Feature Impact panel and click Compute:

For custom models, the registered model version must include training data, assigned during model assembly. If training data is not assigned to the registered model version of a custom model, a notification appears, directing you to go to the model workshop, create a new custom model version, assign a training dataset, and create a new registered model version.

Control the Feature Impact chart¶

After you compute Feature Impact for the registered model version, use the provided controls to change the display:

Select a compute method¶

You can select either SHAP or permutation impact as the computation methodology. To see SHAP—or either method—for any model, you must recompute for each.

• SHAP-based shows how much, on average, each feature affects training data prediction values. For supervised projects, SHAP is available for AutoML projects only. See also the SHAP reference and SHAP considerations.

• Permutation-based shows how much the error of a model would increase, based on a sample of the training data, if values in the column are shuffled.

Some notable characteristics of the methodologies:

• SHAP- and permutation-based impact offers a model-agnostic approach that works for all modeling techniques.

• SHAP Feature Impact is faster and more robust on a smaller sample size than permutation-based Feature Impact.

Use quick-compute¶

When working with Feature Impact, the Use quick-compute option controls the sample size used in the visualization. The row count used to build the visualization is based on the toggle setting:

• On: DataRobot uses 2500 rows or the number of rows in the model training sample size, whichever is smaller.

• Off: DataRobot uses 100,000 rows or the number of rows in the model training sample size, whichever is smaller.

You may want to use this option, for example, to train Feature Impact at a sample size higher than the default 2500 rows (or less, if downsampled) in order to get more accurate and stable results.

Deep dive¶

Feature Impact is an on-demand feature, meaning that you must initiate a calculation for each model to see the results. It is calculated using training data, sorted from most to least important by default, and the accuracy of the most important model is always normalized to 1.

Method calculations¶

This section contains technical details on computation for each of the two available methodologies:

• Permutation-based Feature Impact
• SHAP-based Feature Impact

Permutation-based Feature Impact¶

Permutation-based Feature Impact measures a drop in model accuracy when feature values are shuffled. To compute values, DataRobot:

1. Makes predictions on a sample of training records—2500 rows by default, maximum 100,000 rows.
2. Alters the training data (shuffles values in a column).
3. Makes predictions on the new (shuffled) training data and computes a drop in accuracy that resulted from shuffling.
4. Computes the average drop.
5. Repeats steps 2-4 for each feature.
6. Normalizes the results (i.e., the top feature has an impact of 100%).

The sampling process corresponds to one of the following criteria:

• For balanced data, random sampling is used.
• For imbalanced binary data, smart downsampling is used; DataRobot attempts to make the distribution for imbalanced binary targets closer to 50/50 and adjusts the sample weights used for scoring.
• For zero-inflated regression data, smart downsampling is used; DataRobot groups the non-zero elements into the minority class.
• For imbalanced multiclass data, random sampling is used.

SHAP-based Feature Impact¶

SHAP-based Feature Impact measures how much, on average, each feature affects training data prediction value. To compute values, DataRobot:

1. Takes a sample of records from the training data (5000 rows by default, with a maximum of 100,000 rows).
2. Computes SHAP values for each record in the sample, generating the local importance of each feature in each record.
3. Computes global importance by taking the average of abs(SHAP values) for each feature in the sample.
4. Normalizes the results (i.e., the top feature has an impact of 100%).

Feature considerations¶

Consider the following when evaluating Feature Impact:

• Feature Impact is calculated using a sample of the model's training data. Because sample size can affect results, you may want to recompute the values on a larger sample size.

• Occasionally, due to random noise in the data, there may be features that have negative feature impact scores. In extremely unbalanced data, they may be largely negative. Consider removing these features.

• Under some conditions, Feature Impact results can vary due to the function of the algorithm used for modeling. This could happen, for example, in the case of multicollinearity. In this case, for algorithms using L1 penalty—such as some linear models—the impact will be concentrated on one signal only, while for trees, the impact will be spread uniformly over the correlated signals.