Datarobot Services Overview¶
This section has a short overview of the internal services that are used to build a Datarobot 10.0 cluster.
Persistent Critical Services¶
| コンポーネント | グループ | 説明 | Networking |
|---|---|---|---|
| mongodb | ストレージ | MongoDB is a document-oriented NoSQL database used to store application data. | port: 27017 |
| postgresql-ha | ストレージ | PostgreSQL is a relational SQL database used to store application data. | port: 5432 |
| rabbitmq | WorkerQueue | RabbitMQ is a message broker used for communication within the application, in particular for coordinating the worker queue. | port: 5672, 15672 (http) |
| redis | Caching | redis is used for distributed caching within the platform. | port: 6379, 26379 (sentinel) |
Main Application Services¶
These are the components provided by the main Datarobot helm charts:
Unless otherwise noted, all ports are internal to the k8s cluster and not exposed outside the cluster
| コンポーネント | グループ | 説明 | Networking |
|---|---|---|---|
| analyticsbroker | CorePlatform | Collects, enriches and stores audit logs and app usage event data. | api: 8101 (TCP), ingest: 8018 (UDP), publisher: 8101 (TCP) |
| apigateway* | CorePlatform | API Gateway used by Apps Builder UI | port 8024 |
| apps-builder | アプリ | Application to build No Code Apps. | ports: 8083, 8084 |
| auth-server | CorePlatform | OAuth2 authorization server | port 8008 |
| build-service | CustomModels | Service for building container images within the DataRobot Platform. | (ClusterIP) port 80 → 8080 |
| cns* | CorePlatform | Provides Ory Hydra for OAuth. | hydra public: 4444, hydra admin: 4445 |
| common | CorePlatform | This is a library chart that defines base templates for each type of Kubernetes resource, but does not actually create any objects. | |
| core | CorePlatform | Contains common application configuration settings. | ingress port 80 to nginx** |
| core-integration-tasks | Jobs for performing various maintenance tasks on the cluster | ||
| datasets-service | DataManagement | Performs ETL tasks and is responsible for EDA analysis | port 8100 |
| internal-api | CorePlatform | An internal API used for coordination, storage and retrieval. | port 8001 |
| kavmon | CorePlatform | Kubernetes Availability Monitor | port 8000 |
| kubeworkers | WorkerQueue | Execution layer services for worker jobs | |
| lrs-operator | CustomModels | Manages Long Running Service (LRS) custom resources | |
| mmapp | WebTier | Web server for the frontend UI. | port 8000 |
| mmqueue | WebTier | Websocket server for event notifications to the frontend UI. | port 8011 |
| modmon | MLOps | Model Monitoring services | ports 7005, 7105 |
| notebooks* | Notebooks | Notebook services to manage user notebooks and codespaces. | |
| nbx-ingress | Notebooks | Notebook requests proxy service to stand beween api-gateway and notebooks services. | |
| nbx-websockets | Notebooks | Notebook websockets service to handle all notebooks websocket connections. | |
| nbx-operator | Notebooks | Notebook Operator to manage lifecycle of Notebooks resources. | |
| cfx-session-port-proxy | Notebooks | Notebook requests proxy service to target specific opened by user ports on the notebook. | |
| notifications-broker | CorePlatform | Listens on RabbitMQ for notification events | port 9104 |
| pngexport | MLDev | Renders insights charts as PNG | port 9494 |
| prediction-server | MLOps | API server for prediction requests against model deployments. | Port 80** |
| public-api | WebTier | A web service providing the Public API | port 8004 |
| queue-exec-manager | WorkerQueue | Manages worker execution (job submission, tracking status, etc.) | |
| tableau-extension | インテグレーション | A Tableau dashboard extension | port 80 |
| task-manager | WorkerQueue | Orchestrates jobs submitted to the worker queue | |
| tileservergl | MLDev | Geospatial tile server for map-based charts in the frontend UI. | port 8051 |
| upload | DataManagement | Upload service for dataset ingest | port 8023 |
* depends on notebooks.enabled condition
** exposed via ingress
Pods per service¶
| コンポーネント | Pods |
|---|---|
| analyticsbroker | analyticsbroker-api |
| analyticsbroker | analyticsbroker-ingest |
| analyticsbroker | analyticsbroker-publisher |
| apigateway | apigateway-apigateway |
| apps-builder | apps-builder-api |
| apps-builder | apps-builder-internal-api |
| apps-builder | apps-builder-worker |
| auth-server | auth-server-app |
| build-service | build-service |
| cns | dr-hydra |
| cns | hydra-maester |
| core | core-nginx |
| core-integration-tasks | core-integration-tasks-setup |
| datasets-service | datasets-service-api |
| datasets-service | datasets-service-quickworker |
| datasets-service | datasets-service-worker |
| internal-api | internal-api-app |
| kavmon | kavmon-check-datarobotqueue-group |
| kavmon | kavmon-check-dbhealth-group |
| kavmon | kavmon-check-info-group |
| kavmon | kavmon-check-testjobs-group |
| kubeworkers | kubeworkers-compute-services-k8s |
| kubeworkers | kubeworkers-low-latency |
| kubeworkers | kubeworkers-queue-exec-manager |
| kubeworkers | kubeworkers-qw |
| lrs-operator | dr-lrs-operator |
| mmapp | mmapp-app |
| mmqueue | mmqueue-app |
| modmon | modmon-access-processor |
| modmon | modmon-actuals-processor |
| modmon | modmon-ustom-metrics-processor |
| modmon | modmon-scheduler |
| modmon | modmon-worker-predictions-data |
| modmon | modmon-worker-scheduled-job |
| notebooks | notebooks-code-nuggets-notebooks-code-nuggets-deployment |
| notebooks | notebooks-data-retention-notebooks-data-retention-deployment |
| notebooks | notebooks-environment-va-notebooks-env-vars-deployment |
| notebooks | notebooks-environments-notebooks-environments-deployment |
| notebooks | notebooks-execution-envi-notebooks-exec-envs-deployment |
| notebooks | notebooks-filesystems-notebooks-filesystems-deployment |
| notebooks | notebooks-import-notebooks-import-deployment |
| notebooks | nbx-ingress-notebooks-ingress-deployment |
| notebooks | nbx-notebook-opera-notebooks-notebook-operator |
| notebooks | notebooks-notebooks-notebooks-notebooks-deployment |
| notebooks | notebooks-orchestrator-notebooks-orchestrator-deployment |
| notebooks | notebooks-permissions-notebooks-permissions-deployment |
| notebooks | notebooks-revisions-notebooks-revisions-deployment |
| notebooks | notebooks-terminals-notebooks-terminals-deployment |
| notebooks | notebooks-usage-tracking-notebooks-usage-tracking-deployment |
| notebooks | notebooks-users-notebooks-users-deployment |
| notebooks | nbx-websockets-notebooks-websockets-deployment |
| notifications-broker | notifications-broker-app |
| pngexport | pngexport-app |
| prediction-server | prediction-server-app |
| prediction-server | prediction-server-pps-upload |
| public-api | public-api-app |
| queue-exec-manager | queue-exec-manager-build-service |
| queue-exec-manager | queue-exec-manager-customtask |
| queue-exec-manager | queue-exec-manager-lrs |
| tableau-extension | tableau-extension-app |
| task-manager | task-manager-app |
| tileservergl | tileservergl-app |
| tileservergl | tileservergl-manager |
| upload | upload-app |