Generate model compliance documentation¶
After you create a registered model in the Model Registry (the inventory), you can generate automated compliance documentation for the model. The compliance documentation provides evidence that the components of the model work as intended, the model is appropriate for its intended business purpose, and the model is conceptually sound. This individualized model documentation is especially important for highly regulated industries. For the banking industry, for example, the report can help complete the Federal Reserve System's SR 11-7: Guidance on Model Risk Management.
Tip
You can also generate compliance documentation by selecting a model on the Leaderboard and clicking the Compliance tab.
After you generate the compliance documentation, you can view it or download it as a Microsoft Word (DOCX) file and edit it further. You can also create specialized templates for your organization.
Note
When model packages are shared with users, all generated compliance documentation for the model is also shared.
Generate compliance documentation¶
-
Create a model package if it is not yet in the inventory (Model Registry). To create a model package, you can:
-
Click Model Registry > Registered Models and select a registered model.
-
In the Versions list, to open the registered model version, do either of the following:
-
To open the version in the current tab, click the row for the version you want to access.
-
To open the version in a new tab, click the open icon () next to the Type column for the version you want to access.
-
-
Click the Compliance Documentation tab, select a Report template, and then click Create Report.
The default template is the Automated Compliance Document template. You can instead create a custom report template and select that template.
Compliance documentation for custom models without null imputation support
To generate the Sensitivity Analysis section of the default Automated Compliance Document template, your custom model must support null imputation (the imputation of NaN values), or compliance documentation generation will fail. If the custom model doesn't support null imputation, you can use a specialized template to generate compliance documentation. In the Report template drop-down list, select Automated Compliance Document (for models that do not impute null values). This template excludes the Sensitivity Analysis report and is only available for custom models. If this template option is not available for your version of DataRobot, you can download the custom template for regression models or the custom template for binary classification models.
Generating Compliance Documentation requires DataRobot to execute many dependent insight tasks. This can take several minutes to complete. The documentation appears below when complete:
-
After the compliance documentation is generated, you can:
- Preview the report by clicking the eye () icon.
- Download a Microsoft Word (DOCX) version of the report by clicking the download () icon.
- Delete the report by clicking the trash can () icon.
Feature considerations¶
Consider the following when generating compliance documentation:
-
Compliance documentation, in most cases, is not available for managed AI Platform users. Interested users should contact their DataRobot representative for additional information.
-
Compliance documentation is available for the following project types:
- Binary
- Multiclass
- Regression
- Anomaly detection for time series projects with DataRobot models, but not for non-time series unsupervised mode.