Skip to content

Scale down DataRobot application services

概要

Taking periodic backups does not require stopping the DataRobot application. However, stopping the application during the backup and restoration process is recommended in scenarios where absolute data consistency is crucial (for example, during a migration). This helps prevent any conflicting transactions while the database is being restored.

Consider the following when planning to stop the application:

  • Graceful Shutdown: Coordinate with stakeholders to schedule a maintenance window. Notify users in advance to minimize disruptions.
  • Application Connection Termination: Ensure that all active connections to the PostgreSQL database from the application are terminated before initiating the restoration or backup.
  • Database Locks: Consider applying locks or restrictions to prevent any write operations on the database during the process.

前提条件

These prerequisites are particularly important if you are scaling the application down to ensure absolute data consistency.

  • The kubectl utility (version 1.23 or compatible) must be installed on the host where you will perform these steps. See the kubectl documentation.
  • kubectl must be configured to access the Kubernetes cluster where the DataRobot application is running. Verify this with the kubectl cluster-info command.

手順

  1. Export the name of the DataRobot application Kubernetes namespace to the DR_CORE_NAMESPACE variable. Replace <namespace> with your actual namespace:

    export DR_CORE_NAMESPACE=<namespace> 
    
  2. Before scaling down, annotate each application deployment with its current number of replicas. This allows you to easily scale back up to the same state later.

    for d_obj in $(kubectl -n $DR_CORE_NAMESPACE get deploy -o name -l release=dr); do
        r_count=$(kubectl -n $DR_CORE_NAMESPACE get $d_obj -o jsonpath='{.spec.replicas}')
        kubectl -n $DR_CORE_NAMESPACE annotate --overwrite $d_obj replicas=$r_count
    done 
    
  3. Scale down the DataRobot application services and pcs-pgpool to zero replicas:

    kubectl -n $DR_CORE_NAMESPACE scale deploy -l release=dr --replicas=0
    kubectl -n $DR_CORE_NAMESPACE scale deployments/pcs-pgpool --replicas=0