Select models using custom metrics¶

Access this AI accelerator on GitHub

When it comes to evaluating model performance, DataRobot provides many of the standard metrics out-of-the box, either on the Leaderboard or as part of a model insight.

However, depending on the industry, you may need to sort your DataRobot leaderboard by a specific metric not natively supported by DataRobot. This AI Accelerator demonstrates how one can leverage DataRobot's Python client to extract predictions, compute custom metrics, and sort their DataRobot models accordingly. The topics covered are as follows:

• Setup: import libraries and connect to DataRobot
• Build models with Autopilot
• Retrieve predictions and actuals
• Sort models by Brier Skill Score (BSS)
• Sort models by Rate@Top1%
• Sort models by return-on-investment (ROI)