Composable ML considerations¶

Consider the following when working with Composable ML.

Environment support¶

Composable ML is supported on DataRobot’s managed AI Platform (US, EU).

DRUM supported OSs¶

• DRUM works on Mac OS, Linux, and Windows 10. Testing is on Linux only and therefore compatibility issues may occur when running on other platforms.

Custom task languages¶

• Python and R are supported.

• SAS is not supported.

Task limits¶

Modeling support¶

The following describes modeling-specific capabilities.

Modeling specifics¶

The following are supported:

• AutoML, including OTV (not time series) and Feature Discovery.

• Estimators, both built-in and custom, are available for binary classification, regression, multiclass, and anomaly detection.

• Preprocessing, both built-in and custom.

API¶

• Python SDK for blueprint generation.
• Python client for custom task generation.

Modeling Options¶

The following describes support for advanced modeling options with custom blueprints:

• Metrics and loss function, where “loss function” is used to train a model and “metric” is used to evaluate models and for accuracy monitoring. Typically, the same measure is used for both. Support includes:

  • Built-in metrics.
  • Custom loss functions (as a part of your custom estimator)
  • Custom metrics are not yet supported.

• Exposure, Weight, Counts of Events, Offset

  • For custom estimators, weight is supported (others are not).
  • For built-in estimators, all options are supported.

  Note

  DataRobot does not indicate whether a task takes into account any of these options. As a result, using Composable ML on a project that uses those options is not recommended. (For example, if you train a custom blueprint on a project that uses exposure, there is no guarantee whether the model will use y or y/exposure as the target.) On the other hand, when using blueprints generated by Autopilot, these options are taken into account correctly, in line with the project settings.

• Monotonic constraints are not supported.

• Hyperparameter tuning is supported for built-in tasks, but is not yet supported for custom tasks (although it can be embedded inside a task).

• Blenders are supported with the exception of custom estimators. You can however manually create a blueprint that uses multiple estimators.

Insights and compliance documentation¶

• Model-agnostic insights are supported:

  • All model evaluation insights are supported (including Evaluate tab, Model Comparison, Learning Curve, Speed vs Accuracy, etc.).
  • Permutation-based Feature Impact, Feature Effects, and XEMP Prediction Explanations are supported (including for Anomaly Detection models).

• Model-specific insights offer limited support:

  • SHAP-based Feature Impact, Prediction Explanations, Hotspots, and Variable Effects are not supported.
  • Coefficients and Tree-based variable importance are supported for built-in estimators. Word Cloud is supported for built-in estimators when the parameter WordCloud is set to True.
  • Rating tables are not supported when there are custom tasks.

• Compliance documentation is supported.

Deployment options¶

• All MLOps model monitoring and management features are supported.

• Deployment of a custom blueprint inside the DataRobot platform is supported.

• Deployment outside of the DataRobot platform (using Scoring Code or Portable Prediction Server) is only supported for blueprints without custom tasks.