Experiment reference¶
What types of experiments are supported in Workbench?
Because development is ongoing, see the capability matrix for a full list of supported capabilities.
Are experiments I create in Workbench available in DataRobot Classic?
When you create experiments in Workbench, you can access them in DataRobot Classic via the project management center. When you select an experiment from Classic that was created in Workbench, you will see the Use Case name above the dataset name, indicating it is available in Workbench.
Can an experiment be linked to more than one use case?
An experiment can only be a part of a single Use Case. The only asset that can be in multiple Use Cases is a dataset. The reason for this is because a Use Case is intended to represent a specific business problem and experiments within the Use Case are typically directed at solving that problem. If an experiment is relevant for more than one Use Case, consider consolidating the two Use Cases.
Why isn't the prepared for deployment model at the top of the Leaderboard?
When Workbench prepares a model for deployment, it trains the model on 100% of the data. While the most accurate was selected to be prepared, it was selected based on a 64% sample size. As a part of preparing the most accurate model for deployment, Workbench unlocks Holdout, resulting in the prepared model being trained on different data from the original. If you do not change the Leaderboard to sort by Holdout, the validation score in the left bar can make it appear as if the prepared model is not the most accurate.
Does Workbench create blender models?
You cannot currently create a blender model in Workbench; however, there is a workaround. To add a blender to your use case:
-
Create an experiment and build models.
-
Open the project in DataRobot Classic and blend models.
-
Reopen the experiment in Workbench.
The blender will be present in the Leaderboard.