Monitoring job definitions¶

To integrate more closely with external data sources, monitoring job definitions allow DataRobot to monitor deployments running and storing feature data and predictions outside of DataRobot. たとえば、Snowflakeに接続して、関連するSnowflakeテーブルから元データを取得し、監視目的でDataRobotにデータを送信する監視ジョブを作成することができます。

監視ジョブAPI¶

この連携により、batchPredictionJobDefinitionsとbatchPredictionsの既存の予測APIルートの機能が拡張され、batch_job_type:monitoringプロパティが追加されました。 この新しいプロパティによって、監視ジョブを作成できます。

Job definition routes¶

Batch prediction routes¶

Monitoring job properties¶

{
  "batchJobType": "monitoring",
  "deploymentId": "<deployment_id>",
  "intakeSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "credentialId": "<credential_id>",
      "table": "lending_club_regresion",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "catalog": "SANDBOX"
  },
  "outputSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "table": "lending_club_regresion_out",
      "catalog": "SANDBOX",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "statementType": "insert",
      "createTableIfNotExists": true,
      "credentialId": "<credential_id>",
      "commitInterval": 10,
      "whereColumns": [],
      "updateColumns": []
  },
  "passthroughColumns": [],
  "monitoringColumns": {
      "predictionsColumns": "PREDICTION",
      "uniqueRowIdentifierColumns": ["id"],
      "monitoredStatusColumn": "monitored",
      "associationIdColumn": "id",
      "actualsValueColumn": "loan_amnt"
  }
  "schedule": {
      "minute": [ 0  ],
      "hour": [ 17   ],
      "dayOfWeek": ["*" ],
      "dayOfMonth": ["*" ],
      "month": [ "*” ]
  },
  "enabled": true
} 

{
  "batchJobType": "monitoring",
  "deploymentId": "<deployment_id>",
  "intakeSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "credentialId": "<credential_id>",
      "table": "lending_club_regresion",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "catalog": "SANDBOX"
  },
  "outputSettings": {
      "type": "jdbc",
      "dataStoreId": "<data_store_id>",
      "table": "lending_club_regresion_out",
      "catalog": "SANDBOX",
      "schema": "SCORING_CODE_UDF_SCHEMA",
      "statementType": "insert",
      "createTableIfNotExists": true,
      "credentialId": "<credential_id>",
      "commitInterval": 10,
      "whereColumns": [],
      "updateColumns": []
  },
  "monitoringColumns": {
  "predictionsColumns": [
              {
                "className": "True",
                "columnName": "readmitted_True_PREDICTION"
              },
              {
                "className": "False",
                "columnName": "readmitted_False_PREDICTION"
              }
          ],
      "uniqueRowIdentifierColumns": ["id"],
      "monitoredStatusColumn": "monitored",
      "associationIdColumn": "id",
      "actualsValueColumn": "loan_amnt"
  }
  "schedule": {
      "minute": [ 0  ],
      "hour": [ 17   ],
      "dayOfWeek": ["*" ],
      "dayOfMonth": ["*" ],
      "month": [ "*” ]
  },
  "enabled": true
} 

監視ジョブUI¶

予測APIに加え、DataRobot UIを通じて、監視ジョブの定義を作成することができます。 You can then view and manage monitoring job definitions as you would any other job definition.

To create the monitoring jobs in DataRobot:

1. Click Deployments and select a deployment from the inventory.

2. On the selected deployment's Overview, click Job Definitions.

3. On the Job Definitions page, click Monitoring Jobs, and then click Add Job Definition.

4. On the New Monitoring Job Definition page, configure the following options:

   

   	フィールド名	説明
   	監視ジョブの定義名	Enter the name of the monitoring job that you are creating for the deployment.
   	監視データの送信元	ソースのタイプを設定し、スコアリングするデータの接続を定義します。
   	監視オプション	Configure the monitoring options.
   	データ送信先	(Optional) Configure the data destination options if you enable output monitoring.
   	ジョブスケジュール	Configure whether to run the job immediately and whether to schedule the job.
   	監視ジョブの定義を保存	このボタンをクリックして、ジョブ定義を保存します。 このジョブをすぐに実行トグルがオンになっている場合、このボタンは予測ジョブ定義を保存して実行に変わります。 検定エラーがある場合、このボタンは無効になっています。

Set monitoring data source¶

Select a monitoring source, called an [intake adapter](intake-options, and complete the appropriate authentication workflow for the source type. 以下の接続タイプを選択すると、フィールドの説明が表示されます。

データベース接続

• JDBC

クラウドストレージ接続

• Azure
• GCP (Google Cloud Platform Storage)
• S3

データウェアハウス接続

• BigQuery
• Snowflake
• Synapse

その他

• AIカタログ

After you set your monitoring source, DataRobot validates that the data is applicable to the deployed model.

Set monitoring options¶

When setting the monitoring options, the options available depend on the model type: regression or classification.

Set output monitoring and data destination options¶

After setting the prediction and actuals monitoring options, you can choose to enable Output monitoring status and configure the following options:

With Output monitoring status enabled, you must also configure the Data destination options to specify where the monitored data results should be stored. Select a monitoring data destination, called an output adapter, and complete the appropriate authentication workflow for the destination type. 以下の接続タイプを選択すると、フィールドの説明が表示されます。

データベース接続

• JDBC

クラウドストレージ接続

• Azure
• GCP (Google Cloud Platform Storage)
• S3

データウェアハウス接続

• BigQuery
• Snowflake
• Synapse

その他

• Tableau

Schedule monitoring jobs¶

You can schedule monitoring jobs to run automatically on a schedule. When outlining a monitoring job definition, enable Run this job automatically on a schedule, then specify the frequency (daily, hourly, monthly, etc.) and time of day to define the schedule on which the job runs.

さらに詳細な指定をする場合は、高度なスケジューラーを使用するを選択します。 You can set the exact time (to the minute) you want to run the monitoring job.

After setting all applicable options, click Save monitoring job definition.