GCP With cloud monitoring¶

This section shows how to configure the chart and provision the infrastructure to observe DataRobot on GCP managed services.

要件¶

OIDC provider must be configured. Refer to the Google - Google Kubernetes Engine (GKE) documentation in the installation guide.

The created Google service account must: * Get assigned the logging.logWriter, monitoring.metricWriter and cloudtrace.agent roles * Be bound to each of the Kubernetes service accounts that are bound to the observability subchart pods

The Kubernetes service accounts used by the pods are the following (unless explicitly named otherwise in the subchart):

• observability-v2-otel-deployment
• observability-v2-otel-daemonset
• observability-v2-otel-statsd
• observability-v2-otel-scraper
• observability-v2-otel-scraper-static

This is explained below.

Google service account¶

First, the Google Service Account needs to be created for the project:

GOOGLE_SERVICE_ACCOUNT_ID="<GOOGLE_SERVICE_ACCOUNT_ID>"
PROJECT_NAME="<PROJECT_NAME>"
GOOGLE_SERVICE_ACCOUNT_DISPLAY_NAME="<GOOGLE_SERVICE_ACCOUNT_DISPLAY_NAME>"

gcloud iam service-accounts create "$GOOGLE_SERVICE_ACCOUNT_ID" \
    --project="$PROJECT_NAME" \
    --display-name="$GOOGLE_SERVICE_ACCOUNT_DISPLAY_NAME"

# Get service account email {: #get-service-account-email }
GOOGLE_SERVICE_ACCOUNT_EMAIL="$GOOGLE_SERVICE_ACCOUNT_ID@$PROJECT_NAME.iam.gserviceaccount.com" 

The GOOGLE_SERVICE_ACCOUNT_EMAIL is needed later to configure the chart. The value is <GOOGLE_SERVICE_ACCOUNT_ID>@<PROJECT_NAME>.iam.gserviceaccount.com, where GOOGLE_SERVICE_ACCOUNT and PROJECT_NAME were manually specified above at the time of account creation.

Role binding to the Google service account¶

The previously mentioned roles need to be bound to this account:

declare -a PROJECT_ROLES=(
    "roles/logging.logWriter"
    "roles/monitoring.metricWriter"
    "roles/cloudtrace.agent"
)

for ROLE in "${PROJECT_ROLES[@]}"; do
    echo "  -> Granting role: $ROLE"
    gcloud projects add-iam-policy-binding "$PROJECT_NAME" \
        --member="serviceAccount:$GOOGLE_SERVICE_ACCOUNT_EMAIL" \
        --role="$ROLE" \
        --no-user-output-enabled
done 

Workload identity binding to Kubernetes service accounts¶

Finally, bind the Kubernetes service accounts mentioned earlier to the Google service account:

NAMESPACE="<KUBERNETES_NAMESPACE>"

declare -a KUBERNETES_SERVICE_ACCOUNT_NAMES=(
    "observability-v2-otel-deployment"
    "observability-v2-otel-daemonset"
    "observability-v2-otel-statsd"
    "observability-v2-otel-scraper"
    "observability-v2-otel-scraper-static"
)

for KSA in "${KUBERNETES_SERVICE_ACCOUNT_NAMES[@]}"; do
    MEMBER="serviceAccount:${PROJECT_NAME}.svc.id.goog[${NAMESPACE}/${KSA}]"

    gcloud iam service-accounts add-iam-policy-binding "$GOOGLE_SERVICE_ACCOUNT_EMAIL" \
        --project="$PROJECT_NAME" \
        --role="$WORKLOAD_IDENTITY_BINDING_ROLE" \
        --member="$MEMBER" \
        --no-user-output-enabled
done 

Full chart configuration¶

The following configuration is added to the datarobot-prime chart values. Replace the placeholder values with the actual values obtained in the previous sections.

For additional exporter configuration options, refer to the upstream OpenTelemetry documentation for the Google Cloud Exporter.

global:
  observability:
    auth:
      gcp:
        enabled: true
        serviceAccount: <GOOGLE_SERVICE_ACCOUNT_EMAIL>
        project: <PROJECT_NAME>

    exporters:
      googlecloud:
        project: <PROJECT_NAME>
        log:
          default_log_name: <LOG_NAME>

    signals:
      logs:
        exporters: [googlecloud]
      metrics:
        exporters: [googlecloud]
      traces:
        exporters: [googlecloud] 

各パラメーターについて説明します。

• <GOOGLE_SERVICE_ACCOUNT_EMAIL>: see Google service account
• <PROJECT_NAME>: the project name where DataRobot is deployed (same as the one used in Google service account)
• <LOG_NAME>: a default name for log entries (log.default_log_name in the googlecloud exporter)

Setting auth.gcp.enabled: true automatically:

• Adds the iam.gke.io/gcp-service-account annotation with the provided serviceAccount to all collector serviceAccounts