Composable ML considerations¶
Consider the following when working with Composable ML.
Composable ML is supported on DataRobot’s Managed AI Cloud (US, EU).
DRUM supported OSs¶
- DRUM works on Mac OS, Linux, and Windows 10. Testing is on Linux only and therefore compatibility issues may occur when running on other platforms.
Custom task languages¶
Python and R are supported.
SAS is not supported.
|RAM||60GB training / 4GB scoring|
|Artifact (Serialized trained model/transformer size)||10GB|
|Timeout||72 hrs (for fit)|
|Max custom tasks per blueprint||3|
The following describes modeling-specific capabilities.
The following are supported:
AutoML, including OTV (not time series) and Feature Discovery.
Estimators, both built-in and custom, are available for binary classification, regression, multiclass, and anomaly detection.
Preprocessing, both built-in and custom.
- Python SDK for blueprint generation.
- Python client for custom task generation.
The following describes support for advanced modeling options with custom blueprints:
Metrics and loss function, where “loss function” is used to train a model and “metric” is used to evaluate models and for accuracy monitoring. Typically, the same measure is used for both. Support includes:
- Built-in metrics.
- Custom loss functions (as a part of your custom estimator)
- Custom metrics are not yet supported.
Exposure, Weight, Counts of Events, Offset
- For custom estimators, weight is supported (others are not).
- For built-in estimators, all options are supported.
DataRobot does not indicate whether a task takes into account any of these options. As a result, using Composable ML on a project that uses those options is not recommended. (For example, if you train a custom blueprint on a project that uses exposure, there is no guarantee whether the model will use
y/exposureas the target.) On the other hand, when using blueprints generated by Autopilot, these options are taken into account correctly, in line with the project settings.
Monotonic constraints are not supported.
Hyperparameter tuning is supported for built-in tasks, but is not yet supported for custom tasks (although it can be embedded inside a task).
Blenders are supported with the exception of custom estimators. You can however manually create a blueprint that uses multiple estimators.
Insights and compliance documentation¶
Model-agnostic insights are supported:
- All model evaluation insights are supported (including Evaluate tab, Model Comparison, Learning Curve, Speed vs Accuracy, etc.).
- Permutation-based Feature Impact, Feature Effects, and XEMP Prediction Explanations are supported (including for Anomaly Detection models).
Model-specific insights offer limited support:
- SHAP-based Feature Impact, Prediction Explanations, Hotspots, and Variable Effects are not supported.
- Coefficients and Tree-based variable importance are supported for built-in estimators. Word Cloud is supported for built-in estimators when the parameter
WordCloudis set to
- Rating tables are not supported when there are custom tasks.
Compliance documentation is supported.
All MLOps model monitoring and management features are supported.
Deployment of a custom blueprint inside the DataRobot platform is supported.
Deployment outside of the DataRobot platform (using Scoring Code or Portable Prediction Server) is only supported for blueprints without custom tasks.