LRS Auto-scaling with KEDA¶

Overview¶

DataRobot's Long Running Service (LRS) auto-scaling feature provides dynamic horizontal scaling capabilities for custom model deployments. This feature leverages KEDA (Kubernetes Event-Driven Autoscaling) and the KEDA HTTP Add-on to automatically scale LRS instances based on HTTP workload and CPU usage, including the ability to scale to zero for cost optimization.

The auto-scaling functionality provides a standardized scaling approach that enables:

• CPU-based scaling: Scale based on CPU utilization thresholds
• HTTP-based scaling: Scale based on HTTP request metrics
• Scale-to-zero: Automatically scale down to zero replicas during periods of inactivity
• Cost optimization: Reduce infrastructure costs by only running resources when needed

Note: This feature is initially available for deployed custom models on LRS and not for other LRS-based workloads.

Prerequisites¶

LRS auto-scaling depends on KEDA and the KEDA HTTP Add-on, which are shared cluster amenities. Before enabling LRS auto-scaling, install and configure them as described in KEDA event-driven autoscaling, including the prerequisites, HTTP Add-on configuration values, network requirements, and version requirements.

Custom model (LRS) deployments use path-based HTTPScaledObject routing and have no KEDA HTTP Add-on version restriction.

LRS configuration¶

The following LRS-specific values control how the LRS controller connects to the cluster-wide KEDA and KEDA HTTP Add-on installations.

Configuration values for LRS¶

These are the default values set in the LRS chart:

lrs-operator:
   keda:
      enabled: false
      kedaNamespace: "keda-operator"
      kedaHTTPNamespace: "keda-http-addon"
      proxyServiceName: "keda-add-ons-http-interceptor-proxy"
      proxyServicePort: "8080"
      externalScalerName: "keda-add-ons-http-external-scaler"
      externalScalerPort: "9090"
   operator:
      ...
      config:
         scaledObjectCooldownSeconds: 86400 #  24 hours
      ...

To enable scaling, set lrs-operator.keda.enabled to true. Make sure to update the rest of the values to match your installation of KEDA and the KEDA HTTP Add-on so the LRS controller knows which namespace the components are installed in and what they're called. Set scaledObjectCooldownSeconds: 604800 which translates to scaling to zero after 7 days of idle time.