Prediction output options¶

You can configure a prediction destination using the Predictions > Job Definitions tab or the Batch Prediction API. This topic describes both the UI and API output options.

If you are using a custom CSV format, any output option dealing with CSV will adhere to that format. The columns that appear in the output are documented in the section on output format.

Local file streaming¶

If your job is configured with local file streaming as the output option, you can start downloading the scored data as soon as the job moves to a RUNNING state. In the example job data JSON below, the URL needed to make the local file streaming request is available in the download key of the links object:

{
  "elapsedTimeSec": 97,
  "failedRows": 0,
  "jobIntakeSize": 1150602342,
  "jobOutputSize": 107791140,
  "jobSpec": {
    "deploymentId": "5dc1a6a9865d6c004dd881ef",
    "maxExplanations": 0,
    "numConcurrent": 4,
    "passthroughColumns": null,
    "passthroughColumnsSet": null,
    "predictionWarningEnabled": null,
    "thresholdHigh": null,
    "thresholdLow": null
  },
  "links": {
    "download": "https://app.datarobot.com/api/v2/batchPredictions/5dc45e583c36a100e45276da/download/",
    "self": "https://app.datarobot.com/api/v2/batchPredictions/5dc45e583c36a100e45276da/"
  },
  "logs": [
    "Job created by user@example.org from 203.0.113.42 at 2019-11-07 18:11:36.870000",
    "Job started processing at 2019-11-07 18:11:49.781000",
    "Job done processing at 2019-11-07 18:13:14.533000"
  ],
  "percentageCompleted": 0.0,
  "scoredRows": 3000000,
  "status": "COMPLETED",
  "statusDetails": "Job done processing at 2019-11-07 18:13:14.533000"
}

If you download faster than DataRobot can ingest and score your data, the download may appear sluggish because DataRobot streams the scored data as soon as it arrives (in chunks).

Refer to the this sample use case for a complete example.

HTTP write¶

You can point Batch Predictions at a regular URL, and DataRobot streams the data:

The URL can optionally contain a username and password such as: https://username:password@example.com/datasets/scoring.csv.

The http adapter can be used for writing data to pre-signed URLs from either S3, Azure, or GCP.

JDBC write¶

DataRobot supports writing prediction results back to a JDBC data source. For this, the Batch Prediction API integrates with external data sources using securely stored credentials.

Supply data destination details using the Predictions > Job Definitions tab or the Batch Prediction API (outputSettings) as described in the table below.

Statement types¶

When dealing with Write strategy options, you can use the following statement types to write data, depending on the situation:

Allowed source IP addresses¶

Any connection initiated from DataRobot originates from one of the following IP addresses:

SAP Datasphere write¶

To use SAP Datasphere, supply data destination details using the Predictions > Job Definitions tab or the Batch Prediction API (outputSettings) as described in the table below.

Azure Blob Storage write¶

Azure Blob Storage is an option for writing large files. To save a dataset to Azure Blob Storage, you must set up a credential with DataRobot consisting of an Azure Connection String.

Azure credentials are encrypted and only decrypted when used to set up the client for communication with Azure during scoring.

Google Cloud Storage write¶

DataRobot supports the Google Cloud Storage adapter. To save a dataset to Google Cloud Storage, you must set up a credential with DataRobot consisting of a JSON-formatted account key.

GCP credentials are encrypted and are only decrypted when used to set up the client for communication with GCP during scoring.

Amazon S3 write¶

DataRobot can save scored data to both public and private buckets. To write to S3, you must set up a credential with DataRobot consisting of an access key (ID and key) and optionally a session token.

AWS credentials are encrypted and only decrypted when used to set up the client for communication with AWS during scoring.

BigQuery write¶

To use BigQuery, supply data destination details using the Predictions > Job Definitions tab or the Batch Prediction API (outputSettings) as described in the table below.

Refer to the example section for a complete API example.

Snowflake write¶

To use Snowflake, supply data destination details using the Predictions > Job Definitions tab or the Batch Prediction API (outputSettings) as described in the table below.

Refer to the example section for a complete API example.

Azure Synapse write¶

To use Azure Synapse, supply data destination details using the Predictions > Job Definitions tab or the Batch Prediction API (outputSettings) as described in the table below.

Refer to the example section for a complete API example.