Install guide complete

Self-managed installation and maintenance > Installation and configuration guide > DataRobot advanced configuration section > AWS Elasticsearch offering > Install guide complete

title: Notebooks has a bug to default migration collection to admin description: notebooks:

Installing external PCS with latest DataRobot version¶

We support configuring external PCS components such as Mongo Atlas, Postgres and Redis managed services that customer manages

Following versions of managed database engine are recommended to ensure compatibility

Database	DR Version	Version Recommended
Mongo Atlas or Self Hosted Mongo	11.x	7.x
Postgres RDS or Self Hosted Postgres	11.x	14.x
Redis Elasticache or Self Hosted Redis	11.x	8.0.x,8.2.x

Pre-requisites¶

Redis¶

Redis needs to be non-cluster mode as we need master endpoint specifically.
Set Default AUTH access token as same as pcs-redis k8s secrets value
Starting 11.0.x we support custom username and password for redis authentication
DR versions older than 11.0.x needs to use default user, if using latest version of DR then we support custom user name. User needs to provisioned with following access string on ~* +@all to grant permissions to set, read , del keys for DR
Ensure security group has inbound rule set to allow vpc cidr range on custom tcp port 6379 if default.

The above pre-requisite applies for on-prem redis if they want to bring their own and do not have cloud-managed instance. Only difference is user management, they can add user in redis through ACL utility for on-prem.

Mongo¶

We support both standard and srv connection strings for Mongo
DR versions older than 11.0.x needs to have this user created as entire DR application uses pcs-mongodb to connect to mongo

> use admin
switched to admin
> db.createUser({
  user: "pcs-mongodb",
  pwd: "<PASSWORD_FROM_K8S_SECRETS>",
  roles: [
    { role: "readWriteAnyDatabase", db: "admin" },
    { role: "userAdminAnyDatabase", db: "admin" },
    { role: "dbAdminAnyDatabase", db: "admin" }
  ]
})

Latest versions of DR support custom user to be configured for Mongo so customer can create any custom user according to their compliance
Latest version of DR support custom replicaset name in latest versions of DR
We support custom ports in case Mongo is self hosted/provisioned instance
Ensure security group has inbound rule set to allow vpc cidr range on port range 1000-1800 for private link setup when using Mongo Atlas, 27017 for standard connection or if custom port then inbound rule needs to be configured respectively. Along with all ICMP pings and 443 for https access.

The above pre-requisite applies for on-prem mongo if they want to bring their own and do not have cloud-managed instance. The only change is for the security group where they wouldn’t need port range 1000-1800 if on-prem and not cloud managed.

Postgres¶

We support self provisioned Postgres as well as cloud managed Postgres.

User provisioned needs to have a super user created and credentials needs to be same as pcs-postgresql kubernetes secret
If user has limited privileges and customer cannot provider admin user then please refer to this doc
Starting from 11.1.1 of DR we support custom superuser that can be configured.
Ensure Postgres has security group to allow inbound rules on cidr and port from kubernetes cluster
Ensure Postgres has password auth mechanism both md5 and scram as some of the services need scram-sha authentication methods in DR versions 9.x,10.x
Starting 11.x password auth method can be set only to scram-sha-256

In RDS the parameter is as follows

rds.accepted_password_auth_method  md5+scram

The above pre-requisite applies for on-prem postgres deployment if they want to bring their own and do not have cloud-managed instance.

Chart values to upgrade to latest DR version¶

We have added a lot of additional capabilities for custom PCS configurations, this adds flexilbility for customer compliance requests.

Mongo¶

All services rely on global mongo configuration if the parameters are standard, but there might be cases where customer has provisioned mongo with ldap or custom args. In this case, we can pass MONGO_URI that will override individual parameters

Values file needs the following block to get external Mongo to work

global:
  mongodb:
    auth:
      username: <CUSTOM_USERNAME>
      password: <CUSTOM_PASSWORD>
    connect_method: mongodb # this can be mongodb or mongodb+srv
    default_database: MMApp
    database: MMApp
    hosts:
    -  <MONGO_ENDPOINT>
    port: <CUSTOM_PORT>
    internal: false
    replicaset_name: <CUSTOM_REPLICASET> # if its not default rs0
    uri_template: '%(MONGO_CONNECT_METHOD)s://%(MONGO_USER)s:%(MONGO_PASSWORD)s@%(MONGO_HOST)s/MMApp?replicaSet=%(MONGO_REPLICASET)s&authMechanism=PLAIN&authSource=%%24external' # any addtional parameters can be added to template example: authMechanism=PLAIN&authSource=%%24external
    use_tls: <TLS_OPTION_TRUE_OR_FALSE>
    #ca_cert_path: /datarobot/tls/ca-cert-bundle.crt # set this if use_tls is true with the same path mentioned here
core:
  config_env_vars:
    MONGO_CONNECT_METHOD: mongodb # this can be mongodb or mongodb+srv
    MONGO_URI: <CUSTOM_URI>
# Notebooks has a bug to default migration collection to admin
# Following env_vars are needed if customer's user provisioned do not have access to admin database
notebooks:
  migrations:
    job:
      env_vars:
        common:
        - name: DB_NAME
          value: "notebooks"
        - name: MIGRATION_DATABASE_NAME
          value: "notebooks"
        - name: MIGRATION_COLLECTION_NAME
          value: "nbx_migrations"
  global:
    deployment:
      common:
        envVars:
          MONGO_URI: <CUSTOM_URI>
buzok-onprem:
  buzok-worker:
    services:
      mongo:
        uri: <CUSTOM_URI>
        authMechanism: "PLAIN" # for LDAP configurations, overridden when uri is passed that includes this param
        authSource: '$external' # for LDAP configurations, overridden when uri is passed that includes this param
blob-view-service:
  env:
    MONGO_CONNECTION_STRING:
      value: <CUSTOM_URI>

Postgres¶

We have updated postgres templates to support additional customer configurations and there are additonal blocks in values file that needs postgres config

global:
  postgresql:
    internal: false
    hostname: <CUSTOM_PGSQL_ENDPOINT>
    port: <CUSTOM_PGSQL_PORT>
    tls: <TLS_OPTION_TRUE_OR_FALSE>
    ca_cert_path: "/datarobot/tls/ca-cert-bundle.crt" # cert path for private/custom tls if configured
    auth:
      username: <CUSTOM_PGSQL_ADMIN_USER>
      password: <CUSTOM_PGSQL_PASSWORD>
# Password needs to be set again in the following block
postgresql-ha:
  postgresql:
    postgresPassword: <CUSTOM_PGSQL_PASSWORD>
# Compute spark needs postgres vars set
compute-spark:
  services:
    postgresql:
      database: cspspark
      enabled: true
      externalNodePort: true
      hostname: <CUSTOM_PGSQL_ENDPOINT>
      username: <CUSTOM_PGSQL_ADMIN_USER>
      port: <CUSTOM_PGSQL_PORT>

Redis¶

External redis configurations in latest DR adds additional parameter support, here are blocks in values file that needs to be added to get redis to work

global:
  redis:
    auth:
      username: <CUSTOM_REDIS_USERNAME> # if not specified it is set to default username
      password: <CUSTOM_REDIS_PASSWORD>
    hostname: <CUSTOM_REDIS_ENDPOINT>
    internal: false
    port: <CUSTOM_REDIS_PORT>
    tls: true
    ca_cert_path: "/datarobot/tls/ca-cert-bundle.crt" # in case of bundle defined
    sentinel:
      enabled: false
core:
  config_env_vars:
    REDISPROXY_PORT: <CUSTOM_REDIS_PORT>
# Notebooks block
notebooks:
  global:
    deployment:
      common:
        envVars:
          WEBSOCKET_BROKER_HOST: <CUSTOM_REDIS_ENDPOINT>
          WEBSOCKET_BROKER_PASSWORD: <CUSTOM_REDIS_PASSWORD>
          WEBSOCKET_BROKER_PORT: <CUSTOM_REDIS_PORT>
          WEBSOCKET_BROKER_SSL: true
          WEBSOCKET_BROKER_CA_CERT_PATH: "/datarobot/tls/ca-cert-bundle.crt" # in case of bundle defined
          WEBSOCKET_BROKER_TYPE: redis
nbx-websockets:
  configs:
    data:
      WEBSOCKET_BROKER_HOST: <CUSTOM_REDIS_ENDPOINT>
      WEBSOCKET_BROKER_PORT: <CUSTOM_REDIS_PORT>
      WEBSOCKET_BROKER_SSL: true
      WEBSOCKET_BROKER_TYPE: redis
      WEBSOCKET_BROKER_PASSWORD: <CUSTOM_REDIS_PASSWORD>
      WEBSOCKET_BROKER_CA_CERT_PATH: /datarobot/tls/ca-cert-bundle.crt # in case of bundle defined
  microservice:
    secretsAsFiles:
    - items:
      - fileName: broker_password
        key: redis-password
      secret: pcs-redis

For older versions please refer confluence doc