When a model displays the rating table icon on the Leaderboard, you can export the model's complete, validated parameters. Validation assures that the downloaded parameters are correct and that you can reproduce the model's performance outside of DataRobot. For organizations that have the capability enabled, you can modify the table coefficients and apply the new table to the original (parent) model, resulting in a new "child" model available on the Leaderboard.
Note that, for GA2M models, you can specify the pairwise interactions included in the model's output.
Download rating tables¶
To export rating table coefficients:
From the Leaderboard, identify a model with this icon, indicating that it produced a rating table.
Expand the model and click the Rating Table tab. (The screen may appear different, depending on your permissions.)
Click the Download Table link to save the CSV file. See this additional information for help interpreting the rating table output.
Modify your rating table in a text editor or spreadsheet application. If applicable, you can next upload the modified table to the parent and create a new child model with the table.
Modify rating tables¶
When you modify a rating table and upload it to the original parent model (and then run the model), DataRobot creates a child model with the modified version of the original parent model's rating table. Available from the Leaderboard, the new model has access to the same features as the parent (with these exceptions).
The following, briefly and then in detail, describes the workflow for creating a new child model.
The following outlines the steps to iterate on building models with modified rating tables:
- Download the rating table from the parent.
- Modify the rating table outside of DataRobot using an appropriate editor.
- Upload the modified table to the parent model.
- Score the new model, adding it to the Leaderboard.
- Click Open Child Model to view the new model.
- To iterate on rating table changes, download the child's rating table.
- Modify the child's rating table outside of DataRobot.
- Upload the newly modified table to the parent model.
- Return to step 4 and repeat as necessary.
The following describes, in more detail, the steps for working with rating tables:
Select a model from the Leaderboard that displays the rating table icon. This is the parent model.
Download the parent model's rating table.
Edit the coefficients in the rating table CSV file using an appropriate editor.
Once you have completed modifications to the exported rating table, drag-and-drop or browse to upload the new rating table:
All available (newly and previously uploaded) ratings tables are listed under Uploaded Tables.
If desired, and only before you run the model, you can click the pencil icon to rename the uploaded table, up to 50 characters. Note that the child model's name is based on the name of the rating table it was created from. You can also rename the table outside of the application. If you specify an existing name, DataRobot appends a numeric to the table name.
Click the Add to Leaderboard link to create and score the new model. DataRobot first validates the new rating table and, after building completes, the new child model is available on the Leaderboard. A green check indicates a successfully validated and uploaded table; otherwise, DataRobot displays an error message indicating the issue. (You can monitor build status in the Worker Queue.)
Once the build completes, click the Open Child Model link corresponding to the child model/rating table pair you would like to view. DataRobot opens (and places you in) the Rating Tables tab of the child model. The child model name is
Modified Rating Table: <rating_table_name>.csvand is visible and accessible from the Leaderboard.
From the child model, you can do the following:
|Download Table||Download the rating table of the child model. To iterate on coefficient changes in a table, download the child's rating table, upload the modified child rating table to the parent, compare scores, and continue the process as necessary.|
|Open Parent Model||Move back to the Rating Tables tab of the parent (original) model. From there you can upload new tables, build new models, or open any built child models.|
You cannot upload a new rating table to the child model. You can only upload rating tables to the parent model.
Rating table validation¶
When DataRobot builds a model that produces rating tables (for example, GA2M), it runs validation on the model before making it available from the Leaderboard. For validation, DataRobot compares predictions made by Java rating table Scoring Code (the same predictions to produce that specific Rating Table) against predictions made by a Python code model in the DataRobot application that is independent from the rating table CSV file. If the predictions are different, the rating table fails to validate, and DataRobot marks the model as errored.