Skip to content

Restoring custom models

DataRobot Custom Models are executed on a Kubernetes cluster. Every custom model is backed by DataRobot's S3-compatible Object Storage and MongoDB data. この意味を以下に示します。

  • A backup of all custom models is represented by the backed-up state of the S3-compatible Object Storage and MongoDB.
  • You can fully restore Custom Models to a running state using this data.

To recover all custom models data to a running state on a target installation, DataRobot provides an additional Python script. You should run this script after the following conditions are met:

  • Filestore (S3-compatible Object Storage) and MongoDB have been successfully restored.
  • The DataRobot application is in a running state.
  • An Installation Admin API token is available (this is required for script execution).

The custom model images will be rebuilt as needed and pushed to the new repository as part of the restore process.

前提条件

Ensure the following tools are installed on the host from which you will execute commands:

  • kubectl: Version 1.23 or later.

    • See Kubernetes Tools Documentation.
    • kubectl must be configured to access the Kubernetes cluster where the DataRobot application is running. Verify this configuration with the kubectl cluster-info command.

Running the restore script

The restore script must be started from within a DataRobot application container (specifically, a pod running the mmapp-app deployment).

  1. Set the DR_CORE_NAMESPACE environment variable to your DataRobot application's Kubernetes namespace. Replace <your-datarobot-namespace> with the actual namespace.

    export DR_CORE_NAMESPACE=<your-datarobot-namespace> 
    
  2. Run the restore script. Replace $EXTERNAL_WEB_SERVER_URL with your DataRobot application's URL and <ADMIN_API_TOKEN> with your Installation Admin API token.

    kubectl -n $DR_CORE_NAMESPACE exec deploy/mmapp-app -it -- \
      /entrypoint tools/custom_model/restore_custom_models.py \
      --webserver-address "$EXTERNAL_WEB_SERVER_URL" \
      --admin-api-token <ADMIN_API_TOKEN> 
    

    A similar output will be presented, where each eligible custom model will be restored and re-enabled one by one:

    2021-11-16 00:40:24 INFO     Custom Model LRS list to start (1): ['6192d2abdb1a2688cb94b968']
    2021-11-16 00:40:24 INFO     Starting Custom Model LRS 6192d2abdb1a2688cb94b968
    2021-11-16 00:40:35 INFO     Started Custom Model LRS 6192d2abdb1a2688cb94b968
    2021-11-16 00:40:35 INFO     Started: 1 models. Failed: 0 models 
    

The script is idempotent and will not alter custom models that were already running at the time of execution.