The Model Deployment Approval Workflow is only available for DataRobot MLOps users. Contact your DataRobot representative for more information about enabling this feature.
Note that the approval workflow for deployment events is only prompted when the Model Deployment Approval Workflow is enabled. When you create a new or change an existing deployment, if the approval workflow is enabled, an MLOps administrator within your organization must approve your changes.
Approval policies affect the users who have permissions to review deployments, and provide automated actions when reviews time out. Approval policies also affect users whose deployment events are governed by a configured policy (e.g., new deployment creation, model replacement).
Be sure to review the deployment approval workflow considerations before proceeding.
Deployment importance levels¶
Before you deploy your model, you are prompted to assign an importance level to it: Critical, High, Moderate, or Low. Importance represents an aggregate of factors relevant to your organization such as the prediction volume of the deployment, level of exposure, potential financial impact, and more.
Deployment creation approval¶
Once the deployment is created, MLOps Admins are alerted via email that the deployment requires review. While awaiting review the deployment is flagged as "NEEDS APPROVAL" in the deployments inventory.
While a deployment with "NEEDS APPROVAL" status can still make predictions, DataRobot recommends contacting your MLOps administrator before using it to do so. Once the deployment is approved, the flag updates to "APPROVED" in the inventory. Additionally, predictions made after the deployment is approved are marked as “APPROVED” in the prediction server response metadata.
Deployment event approval¶
As a deployment owner, you can make changes to an existing deployment and include comments to detail the reason for the change. You also choose whether, after the change request has been approved, you want the change manually or automatically applied.
You are always notified via email when your change request has been approved or requires changes. After approval, you can apply the changes; if changes are set to automatic, DataRobot applies changes immediately after approval.
MLOps Admin: Approve deployment creation¶
The MLOps administrator role offers access and governance to those within organizations who oversee deployments. Administrators are often responsible for monitoring deployments that make prediction requests, ensuring the quality of deployment performance, assisting with debugging, and reporting on deployment usage and activity.
The MLOps administrator role, assigned by the system administrator, has User role permissions for all existing and newly created deployments within their organization. They receive email notifications to approve deployment events such as creation, model replacement, changes to importance levels, and deletion.
These elevated permissions only apply to deployments.
Your primary function as an MLOps administrator is to review and approve deployment events within your organization. When a user within your organization creates a deployment, you are alerted via email that the deployment requires review. When you access a deployment that needs approval, DataRobot presents a notification banner prompting you to begin the deployment review process. Select Add Review:
A dialog box populates to submit approval or request updates for the deployment. Review the deployment and its importance level (chosen by the deployment creator). Optionally, include any comments with your decision. If approved, DataRobot removes the “NEEDS APPROVAL” flag from the deployments inventory listing. If changes were requested, the flag remains until the changes are addressed and the deployment is approved.
MLOps Admin: Approve changes to existing deployments¶
In addition to reviewing deployment creations, MLOps administrators review and approve deployment events such as a model replacement, changes to importance levels, and deployment deletion. You will receive email notifications for these triggering events that require approval. Deployment owners are notified via email when you approve or request changes.
As an MLOps administrator accessing a deployment that needs approval for a change, DataRobot presents a notification banner prompting you to begin the deployment review process. You can review the deployment's history of changes in the Overview tab under the Governance header. The Governance header details the history of changes made to a deployment, including the importance levels, the changes made, who made them, and who reviewed them.
To approve a deployment change, select Add Review from the notification banner (also available under the Governance header):
A dialog box populates including a summary of the changes and any comments provided by the deployment owner making the change. Review the desired changes, include any comments with your decision, and submit approve or request updates for the deployment.
After submitting your review, the notification banner updates if the changes were approved. If the deployment owner chose to have the changes applied manually, the owner can click Apply Changes to do so.
A deployment Owner can choose to share a deployment with MLOps administrators and grant either User or Owner permissions. When explicitly shared, Owner rights are the default.
Although MLOps administrators can, by default, monitor deployments in the deployments inventory at the User level, they do not have the ability to make predictions on those deployments unless those deployments are explicitly shared with them.