Composable ML considerations¶

Consider the following when working with Composable ML.

Environment support¶

Custom tasks are supported on DataRobot’s managed AI Platform (US, EU). Composable ML blueprints with DataRobot tasks are available for all users.

DRUM supported OSs¶

• DRUM works on Mac OS, Linux, and Windows 10. Testing is on Linux only and therefore compatibility issues may occur when running on other platforms.

Custom task languages¶

• Python and R are supported.

• SAS is not supported.

Task limits¶

Modeling support¶

The following describes modeling-specific capabilities.

Modeling specifics¶

The following are supported:

• Predictive ML, including time-aware but not time series, and Feature Discovery.

• Estimators, both built-in and custom, are available for binary classification, regression, multiclass, anomaly detection, and clustering.

• Preprocessing, both built-in and custom.

API¶

• Python SDK for blueprint generation.
• Python client for custom task generation.

Modeling options¶

The following describes support for advanced modeling options with custom blueprints:

• Metrics and loss function, where “loss function” is used to train a model and “metric” is used to evaluate models and for accuracy monitoring. Typically, the same measure is used for both. Support includes:

  • Built-in metrics.
  • Custom loss functions (as a part of your custom estimator)
  • Custom metrics are not yet supported.

• Exposure, Weight, Counts of Events, Offset

  • For custom estimators, weight is supported (others are not).
  • For built-in estimators, all options are supported.

  Note

  DataRobot does not indicate whether a task takes into account any of these options. As a result, using Composable ML on a project that uses those options is not recommended. (For example, if you train a custom blueprint on a project that uses exposure, there is no guarantee whether the model will use y or y/exposure as the target.) On the other hand, when using blueprints generated by Autopilot, these options are taken into account correctly, in line with the project settings.

• Monotonic constraints are not supported.

• Hyperparameter tuning is supported for custom and built-in tasks.

• Blenders are supported with the exception of custom estimators. You can however manually create a blueprint that uses multiple estimators.

Insights and compliance documentation¶

• Model-agnostic insights are supported:

  • All model evaluation insights are supported (including Evaluate tab, Model Comparison, Learning Curve, Speed vs Accuracy, etc.).
  • Permutation-based Feature Impact, Feature Effects, and XEMP Prediction Explanations are supported (including for Anomaly Detection models).

• Model-specific insights offer limited support:

  • SHAP-based Feature Impact, Prediction Explanations, Hotspots, and Variable Effects are not supported.
  • Coefficients and Tree-based variable importance are supported for built-in estimators. Word Cloud is supported for built-in estimators when the parameter WordCloud is set to True.
  • Rating tables are not supported when there are custom tasks.

• Compliance documentation is supported.

Deployment options¶

• All MLOps model monitoring and management features are supported.

• Deployment of a custom blueprint inside the DataRobot platform is supported.

• Deployment outside of the DataRobot platform (using Scoring Code or Portable Prediction Server) is only supported for blueprints without custom tasks.