Composable ML considerations¶
Consider the following when working with Composable ML.
Environment support¶
Composable ML is supported on DataRobot’s managed AI Platform (US, EU).
DRUM supported OSs¶
- DRUM works on Mac OS, Linux, and Windows 10. Testing is on Linux only and therefore compatibility issues may occur when running on other platforms.
Custom task languages¶
-
Python and R are supported.
-
SAS is not supported.
Task limits¶
Component | Size |
---|---|
RAM | 60GB training / 4GB scoring |
CPU Cores | 4 |
GPU | not supported |
Storage | 350GB |
Artifact (Serialized trained model/transformer size) | 10GB |
Timeout | 72 hrs (for fit) |
Max custom tasks per blueprint | 3 |
Modeling support¶
The following describes modeling-specific capabilities.
Modeling specifics¶
The following are supported:
-
AutoML, including OTV (not time series) and Feature Discovery.
-
Estimators, both built-in and custom, are available for binary classification, regression, multiclass, and anomaly detection.
-
Preprocessing, both built-in and custom.
API¶
- Python SDK for blueprint generation.
- Python client for custom task generation.
Modeling Options¶
The following describes support for advanced modeling options with custom blueprints:
-
Metrics and loss function, where “loss function” is used to train a model and “metric” is used to evaluate models and for accuracy monitoring. Typically, the same measure is used for both. Support includes:
- Built-in metrics.
- Custom loss functions (as a part of your custom estimator)
- Custom metrics are not yet supported.
-
Exposure, Weight, Counts of Events, Offset
- For custom estimators, weight is supported (others are not).
- For built-in estimators, all options are supported.
Note
DataRobot does not indicate whether a task takes into account any of these options. As a result, using Composable ML on a project that uses those options is not recommended. (For example, if you train a custom blueprint on a project that uses exposure, there is no guarantee whether the model will use
y
ory/exposure
as the target.) On the other hand, when using blueprints generated by Autopilot, these options are taken into account correctly, in line with the project settings. -
Monotonic constraints are not supported.
-
Hyperparameter tuning is supported for built-in tasks, but is not yet supported for custom tasks (although it can be embedded inside a task).
-
Blenders are supported with the exception of custom estimators. You can however manually create a blueprint that uses multiple estimators.
Insights and compliance documentation¶
-
Model-agnostic insights are supported:
- All model evaluation insights are supported (including Evaluate tab, Model Comparison, Learning Curve, Speed vs Accuracy, etc.).
- Permutation-based Feature Impact, Feature Effects, and XEMP Prediction Explanations are supported (including for Anomaly Detection models).
-
Model-specific insights offer limited support:
- SHAP-based Feature Impact, Prediction Explanations, Hotspots, and Variable Effects are not supported.
- Coefficients and Tree-based variable importance are supported for built-in estimators. Word Cloud is supported for built-in estimators when the parameter
WordCloud
is set toTrue
. - Rating tables are not supported when there are custom tasks.
-
Compliance documentation is supported.
Deployment options¶
-
All MLOps model monitoring and management features are supported.
-
Deployment of a custom blueprint inside the DataRobot platform is supported.
-
Deployment outside of the DataRobot platform (using Scoring Code or Portable Prediction Server) is only supported for blueprints without custom tasks.