# Batch predictions
URL: https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html

# Batch predictions

### class datarobot.models.BatchPredictionJob

A Batch Prediction Job is used to score large data sets on
prediction servers using the Batch Prediction API.

- Variables: id ( str ) – the ID of the job

#### classmethod score(deployment, intake_settings=None, output_settings=None, csv_settings=None, timeseries_settings=None, num_concurrent=None, chunk_size=None, passthrough_columns=None, passthrough_columns_set=None, max_explanations=None, max_ngram_explanations=None, explanation_algorithm=None, threshold_high=None, threshold_low=None, prediction_threshold=None, prediction_warning_enabled=None, include_prediction_status=False, skip_drift_tracking=False, prediction_instance=None, abort_on_error=True, column_names_remapping=None, include_probabilities=True, include_probabilities_classes=None, download_timeout=120, download_read_timeout=660, upload_read_timeout=600, explanations_mode=None)

Create new batch prediction job, upload the scoring dataset and
return a batch prediction job.

The default intake and output options are both localFile which
requires the caller to pass the file parameter and either
download the results using the download() method afterwards or
pass a path to a file where the scored data will be downloaded to
afterwards.

- Variables:

#### classmethod apply_time_series_data_prep_and_score(deployment, intake_settings, timeseries_settings, **kwargs)

Prepare the dataset with time series data prep, create new batch prediction job,
upload the scoring dataset, and return a batch prediction job.

The supported intake_settings are of type localFile or dataset.

For timeseries_settings of type forecast the forecast_point must be specified.

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

Added in version v3.1.

- Variables:

#### classmethod score_to_file(deployment, intake_path, output_path, **kwargs)

Create new batch prediction job, upload the scoring dataset and
download the scored CSV file concurrently.

Will block until the entire file is scored.

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

- Variables:
- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

#### classmethod apply_time_series_data_prep_and_score_to_file(deployment, intake_path, output_path, timeseries_settings, **kwargs)

Prepare the input dataset with time series data prep. Then, create a new batch prediction
job using the prepared AI catalog item as input and concurrently download the scored CSV
file.

The function call will return when the entire file is scored.

For timeseries_settings of type forecast the forecast_point must be specified.

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

Added in version v3.1.

- Variables:

#### classmethod score_s3(deployment, source_url, destination_url, credential=None, endpoint_url=None, **kwargs)

Create new batch prediction job, with a scoring dataset from S3
and writing the result back to S3.

This returns immediately after the job has been created. You
must poll for job completion using get_status() or
wait_for_completion() (see datarobot.models.Job)

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

- Variables:
- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

#### classmethod score_azure(deployment, source_url, destination_url, credential=None, **kwargs)

Create new batch prediction job, with a scoring dataset from Azure blob
storage and writing the result back to Azure blob storage.

This returns immediately after the job has been created. You
must poll for job completion using get_status() or
wait_for_completion() (see datarobot.models.Job).

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

- Variables:
- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

#### classmethod score_gcp(deployment, source_url, destination_url, credential=None, **kwargs)

Create new batch prediction job, with a scoring dataset from Google Cloud Storage
and writing the result back to one.

This returns immediately after the job has been created. You
must poll for job completion using get_status() or
wait_for_completion() (see datarobot.models.Job).

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

- Variables:
- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

#### classmethod score_from_existing(batch_prediction_job_id)

Create a new batch prediction job based on the settings from a previously created one

- Variables: batch_prediction_job_id ( str ) – ID of the previous batch prediction job
- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

#### classmethod score_pandas(deployment, df, read_timeout=660, **kwargs)

Run a batch prediction job, with a scoring dataset from a
pandas dataframe. The output from the prediction will be joined
to the passed DataFrame and returned.

Use columnNamesRemapping to drop or rename columns in the
output

This method blocks until the job has completed or raises an
exception on errors.

Refer to the [datarobot.models.BatchPredictionJob.score()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJob.score) method for details on the other
kwargs parameters.

- Variables:
- Return type: Tuple [ BatchPredictionJob , DataFrame ]
- Returns:

#### classmethod score_with_leaderboard_model(model, intake_settings=None, output_settings=None, csv_settings=None, timeseries_settings=None, passthrough_columns=None, passthrough_columns_set=None, max_explanations=None, max_ngram_explanations=None, explanation_algorithm=None, threshold_high=None, threshold_low=None, prediction_threshold=None, prediction_warning_enabled=None, include_prediction_status=False, abort_on_error=True, column_names_remapping=None, include_probabilities=True, include_probabilities_classes=None, download_timeout=120, download_read_timeout=660, upload_read_timeout=600, explanations_mode=None)

Creates a new batch prediction job for a Leaderboard model by
uploading the scoring dataset. Returns a batch prediction job.

The default intake and output options are both localFile, which
requires the caller to pass the file parameter and either
download the results using the download() method afterwards or
pass a path to a file where the scored data will be downloaded to.

- Variables:

#### classmethod get(batch_prediction_job_id)

Get batch prediction job

- Variables: batch_prediction_job_id ( str ) – ID of batch prediction job
- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

#### download(fileobj, timeout=120, read_timeout=660)

Downloads the CSV result of a prediction job

- Variables:

#### delete(ignore_404_errors=False)

Cancel this job. If this job has not finished running, it will be
removed and canceled.

- Return type: None

#### get_status()

Get status of batch prediction job

- Returns: Dict with job status
- Return type: BatchPredictionJob status data

#### classmethod list_by_status(statuses=None)

Get jobs collection for specific set of statuses

- Variables: statuses – List of statuses to filter jobs ([ABORTED|COMPLETED…])
  if statuses is not provided, returns all jobs for user
- Returns: List of job statuses dicts with specific statuses
- Return type: BatchPredictionJob statuses

### class datarobot.models.BatchPredictionJobDefinition

#### classmethod get(batch_prediction_job_definition_id)

Get batch prediction job definition

- Variables: batch_prediction_job_definition_id ( str ) – ID of batch prediction job definition
- Returns: Instance of BatchPredictionJobDefinition
- Return type: BatchPredictionJobDefinition

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> definition = dr.BatchPredictionJobDefinition.get('5a8ac9ab07a57a0001be501f')
> >>> definition
> BatchPredictionJobDefinition(60912e09fd1f04e832a575c1)
> ```

#### classmethod list(search_name=None, deployment_id=None, limit=, offset=0)

Get job all definitions

- Parameters:
- Returns: List of job definitions the user has access to see
- Return type: List[BatchPredictionJobDefinition]

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> definition = dr.BatchPredictionJobDefinition.list()
> >>> definition
> [
>     BatchPredictionJobDefinition(60912e09fd1f04e832a575c1),
>     BatchPredictionJobDefinition(6086ba053f3ef731e81af3ca)
> ]
> ```

#### classmethod create(enabled, batch_prediction_job, name=None, schedule=None)

Creates a new batch prediction job definition to be run either at scheduled interval or as
a manual run.

- Variables:

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> job_spec = {
> ...    "num_concurrent": 4,
> ...    "deployment_id": "foobar",
> ...    "intake_settings": {
> ...        "url": "s3://foobar/123",
> ...        "type": "s3",
> ...        "format": "csv"
> ...    },
> ...    "output_settings": {
> ...        "url": "s3://foobar/123",
> ...        "type": "s3",
> ...        "format": "csv"
> ...    },
> ...}
> >>> schedule = {
> ...    "day_of_week": [
> ...        1
> ...    ],
> ...    "month": [
> ...        "*"
> ...    ],
> ...    "hour": [
> ...        16
> ...    ],
> ...    "minute": [
> ...        0
> ...    ],
> ...    "day_of_month": [
> ...        1
> ...    ]
> ...}
> >>> definition = BatchPredictionJobDefinition.create(
> ...    enabled=False,
> ...    batch_prediction_job=job_spec,
> ...    name="some_definition_name",
> ...    schedule=schedule
> ... )
> >>> definition
> BatchPredictionJobDefinition(60912e09fd1f04e832a575c1)
> ```

#### update(enabled, batch_prediction_job=None, name=None, schedule=None)

Updates a job definition with the changed specs.

Takes the same input as [create()](https://docs.datarobot.com/en/docs/api/reference/sdk/batch-predictions.html#datarobot.models.BatchPredictionJobDefinition.create)

- Variables:
- Returns: Instance of the updated BatchPredictionJobDefinition
- Return type: BatchPredictionJobDefinition

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> job_spec = {
> ...    "num_concurrent": 5,
> ...    "deployment_id": "foobar_new",
> ...    "intake_settings": {
> ...        "url": "s3://foobar/123",
> ...        "type": "s3",
> ...        "format": "csv"
> ...    },
> ...    "output_settings": {
> ...        "url": "s3://foobar/123",
> ...        "type": "s3",
> ...        "format": "csv"
> ...    },
> ...}
> >>> schedule = {
> ...    "day_of_week": [
> ...        1
> ...    ],
> ...    "month": [
> ...        "*"
> ...    ],
> ...    "hour": [
> ...        "*"
> ...    ],
> ...    "minute": [
> ...        30, 59
> ...    ],
> ...    "day_of_month": [
> ...        1, 2, 6
> ...    ]
> ...}
> >>> definition = BatchPredictionJobDefinition.create(
> ...    enabled=False,
> ...    batch_prediction_job=job_spec,
> ...    name="updated_definition_name",
> ...    schedule=schedule
> ... )
> >>> definition
> BatchPredictionJobDefinition(60912e09fd1f04e832a575c1)
> ```

#### run_on_schedule(schedule)

Sets the run schedule of an already created job definition.

If the job was previously not enabled, this will also set the job to enabled.

- Variables: schedule ( dict ) – Same as schedule in create() .
- Returns: Instance of the updated BatchPredictionJobDefinition with the new / updated schedule.
- Return type: BatchPredictionJobDefinition

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> definition = dr.BatchPredictionJobDefinition.create('...')
> >>> schedule = {
> ...    "day_of_week": [
> ...        1
> ...    ],
> ...    "month": [
> ...        "*"
> ...    ],
> ...    "hour": [
> ...        "*"
> ...    ],
> ...    "minute": [
> ...        30, 59
> ...    ],
> ...    "day_of_month": [
> ...        1, 2, 6
> ...    ]
> ...}
> >>> definition.run_on_schedule(schedule)
> BatchPredictionJobDefinition(60912e09fd1f04e832a575c1)
> ```

#### run_once()

Manually submits a batch prediction job to the queue, based off of an already
created job definition.

- Returns: Instance of BatchPredictionJob
- Return type: BatchPredictionJob

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> definition = dr.BatchPredictionJobDefinition.create('...')
> >>> job = definition.run_once()
> >>> job.wait_for_completion()
> ```

#### delete()

Deletes the job definition and disables any future schedules of this job if any.
If a scheduled job is currently running, this will not be cancelled.

> [!NOTE] Examples
> ```
> >>> import datarobot as dr
> >>> definition = dr.BatchPredictionJobDefinition.get('5a8ac9ab07a57a0001be501f')
> >>> definition.delete()
> ```

- Return type: None

## Batch job

### class datarobot.models.batch_job.IntakeSettings

Intake settings typed dict

### class datarobot.models.batch_job.OutputSettings

Output settings typed dict

## Predict job

### datarobot.models.predict_job.wait_for_async_predictions(project_id, predict_job_id, max_wait=600)

Given a Project id and PredictJob id poll for status of process
responsible for predictions generation until it’s finished

- Parameters:
- Returns: predictions – Generated predictions.
- Return type: pandas.DataFrame
- Raises:

### class datarobot.models.PredictJob

Tracks asynchronous work being done within a project

- Variables:

#### classmethod from_job(job)

Transforms a generic Job into a PredictJob

- Parameters: job ( Job ) – A generic job representing a PredictJob
- Returns: predict_job – A fully populated PredictJob with all the details of the job
- Return type: PredictJob
- Raises: ValueError: – If the generic Job was not a predict job, e.g., job_type != JOB_TYPE.PREDICT

#### classmethod get(project_id, predict_job_id)

Fetches one PredictJob. If the job finished, raises PendingJobFinished
exception.

- Parameters:
- Returns: predict_job – The pending PredictJob
- Return type: PredictJob
- Raises:

#### classmethod get_predictions(project_id, predict_job_id, class_prefix='class_')

Fetches finished predictions from the job used to generate them.

> [!NOTE] Notes
> The prediction API for classifications now returns an additional prediction_values
> dictionary that is converted into a series of class_prefixed columns in the final
> dataframe. For example, = 1.0 is converted to ‘class_1.0’. If you are on an
> older version of the client (prior to v2.8), you must update to v2.8 to correctly pivot
> this data.

- Parameters:
- Returns: predictions – Generated predictions
- Return type: pandas.DataFrame
- Raises:

#### cancel()

Cancel this job. If this job has not finished running, it will be
removed and canceled.

#### get_result(params=None)

- Parameters: params ( dict or None ) – Query parameters to be added to request to get results.

> [!NOTE] Notes
> For featureEffects, source param is required to define source,
> otherwise the default is training.

- Returns:result– Return type depends on the job type
: - for model jobs, a Model is returned
  - for predict jobs, a pandas.DataFrame (with predictions) is returned
  - for featureImpact jobs, a list of dicts by default (seewith_metadataparameter of theFeatureImpactJobclass and itsget()method).
  - for primeRulesets jobs, a list of Rulesets
  - for primeModel jobs, a PrimeModel
  - for primeDownloadValidation jobs, a PrimeFile
  - for predictionExplanationInitialization jobs, a PredictionExplanationsInitialization
  - for predictionExplanations jobs, a PredictionExplanations
  - for featureEffects, a FeatureEffects.
*Return type:object*Raises:*JobNotFinished– If the job is not finished, the result is not available.
*AsyncProcessUnsuccessfulError– If the job errored or was aborted

#### get_result_when_complete(max_wait=600, params=None)

- Parameters:
- Returns: result – Return type is the same as would be returned by Job.get_result.
- Return type: object
- Raises:

#### refresh()

Update this object with the latest job data from the server.

#### wait_for_completion(max_wait=600)

Waits for job to complete.

- Parameters: max_wait ( Optional[int] ) – How long to wait for the job to finish.
- Return type: None

## Prediction dataset

### class datarobot.models.PredictionDataset

A dataset uploaded to make predictions

Typically created via project.upload_dataset

- Variables:

#### classmethod get(project_id, dataset_id)

Retrieve information about a dataset uploaded for predictions

- Parameters:
- Returns: dataset – A dataset uploaded to make predictions
- Return type: PredictionDataset

#### delete()

Delete a dataset uploaded for predictions

Will also delete predictions made using this dataset and cancel any predict jobs using
this dataset.

- Return type: None

---
