Skip to content

Click in-app to access the full platform documentation for your version of DataRobot.

Make predictions

When the modeling process is complete and you have retrieved the best model, you can then test its prediction capabilities and deploy the model to a production environment to generate predictions.

Test predictions

For testing purposes, test prediction generation with the top-performing model before deploying the model to a prediction server.

Prediction Explanations

To include Prediction Explanations, run the cells below to first calculate Feature Impact and then initialize Prediction Explanations.

SEGMENT_NAME = "segment_name"

combined_models = project.get_combined_models()
current_combined_model = combined_models[0]
segments_info = current_combined_model.get_segments_info()
segment_metadata = {x.segment: x for x in segments_info}

model = dr.Model.get(segment_metadata["SEGMENT_NAME"].project_id, segment_metadata["SEGMENT_NAME"].model_id)
child_project = dr.Project.get(project_id = segment_metadata["SEGMENT_NAME"].project_id)
feature_impacts = model.get_or_request_feature_impact()
# Upload prediction dataset 
# This will be used for establishing prediction explanation
dataset = project.upload_dataset('Acme_Test_Cluster_0.csv')
# Run Feature Impact
# Need to do this to get Prediction Explanations
model.get_or_request_feature_impact()
# Compute predictions
predict_job = model.request_predictions(dataset.id)
# Initialize Prediction Explanations
pei_job = dr.PredictionExplanationsInitialization.create(PID, MID)
# Compute Prediction Explanations
pe_job = dr.PredictionExplanations.create(project.id, model.id, dataset.id)
pe = pe_job.get_result_when_complete()
# Get Prediction Explanations as a Dataframe
pred_exp = pe.get_all_as_dataframe()
pred_exp
row_id prediction explanation_0_feature explanation_0_feature_value explanation_0_label explanation_0_qualitative_strength explanation_0_strength explanation_1_feature explanation_1_feature_value explanation_1_label explanation_1_qualitative_strength explanation_1_strength explanation_2_feature explanation_2_feature_value explanation_2_label explanation_2_qualitative_strength explanation_2_strength
0 51 3.69103 SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -17.49522 SalesQty (7 day median) 0.00000 SalesQty (actual) --- -15.28814 SalesQty (naive latest value) 3.00000 SalesQty (actual) -- -6.23734
1 52 3.83318 SalesQty (7 day median) 0.00000 SalesQty (actual) --- -24.76728 SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -18.77639 SalesQty (naive latest value) 3.00000 SalesQty (actual) -- -7.17425
2 53 3.45923 SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -16.67129 SalesQty (7 day median) 0.00000 SalesQty (actual) -- -5.25844 SalesQty (14 day median) 1.00000 SalesQty (actual) -- -4.65901
3 54 2.57798 SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -17.32528 SalesQty (7 day median) 0.00000 SalesQty (actual) -- -4.84710 SalesQty (14 day median) 1.00000 SalesQty (actual) -- -4.55620
4 55 2.57798 SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -17.32528 SalesQty (7 day median) 0.00000 SalesQty (actual) -- -4.84710 SalesQty (14 day median) 1.00000 SalesQty (actual) -- -4.55620
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
832 7730 10.20752 SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.22766 SalesQty (7 day max) 25.00000 SalesQty (actual) -- -4.19796 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -4.08395
833 7731 9.44667 SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.75699 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -3.66404 SalesQty (7 day max) 25.00000 SalesQty (actual) -- -2.99822
834 7732 9.44667 SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.75699 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -3.66404 SalesQty (14 day min) 3.00000 SalesQty (actual) -- -3.56161
835 7733 7.89606 SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.23909 SalesQty (7 day max) 25.00000 SalesQty (actual) -- -3.11896 SalesQty (14 day median) 6.50000 SalesQty (actual) -- -2.58671
836 7734 9.10800 SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -7.83997 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -6.35970 SalesQty (14 day median) 6.50000 SalesQty (actual) -- -3.53018

837 rows × 17 columns

Retrieve forecasts

Use the following code for validation purposes prior to deploying the model. Before proceeding, review the guidelines for configuring a prediction dataset to make time series series predictions.

RECORD = 0

# Verify correct project
PID = best_models_features['Project_ID'].values[RECORD]
MID = best_models_features['Model_ID'].values[RECORD]

project = dr.Project.get(PID)
model   = dr.Model.get(PID, MID)
print(project, model)

You can then upload the predictions test dataset to get predictions with 85% prediction intervals and three Prediction Explanations.

%%time

dataset = project.upload_dataset('Acme_Test_Cluster_0.csv')

pred_job = model.request_predictions(dataset_id = dataset.id,
                                     include_prediction_intervals = True,
                                     prediction_intervals_size = 85
                                    )

preds = pred_job.get_result_when_complete()

preds.head(5)

Use the code below to join prediction results, prediction intervals, and Prediction Explanations to one dataframe.

fin_df = preds.merge(pred_exp, how='left', on='row_id')
fin_df.drop(labels=['prediction_y'], axis=1, inplace=True)
fin_df.rename(columns={'prediction_x':'prediction'}, inplace=True)
fin_df
series_id forecast_distance forecast_point prediction prediction_interval_lower_bound prediction_interval_upper_bound row_id timestamp explanation_0_feature explanation_0_feature_value explanation_0_label explanation_0_qualitative_strength explanation_0_strength explanation_1_feature explanation_1_feature_value explanation_1_label explanation_1_qualitative_strength explanation_1_strength explanation_2_feature explanation_2_feature_value explanation_2_label explanation_2_qualitative_strength explanation_2_strength
0 01053_Alcohol_3633522291 1 2018-11-16T00:00:00.000000Z 3.69103 0.47841 6.14793 51 2018-11-17T00:00:00.000000Z SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -17.49522 SalesQty (7 day median) 0.00000 SalesQty (actual) --- -15.28814 SalesQty (naive latest value) 3.00000 SalesQty (actual) -- -6.23734
1 01053_Alcohol_3633522291 2 2018-11-16T00:00:00.000000Z 3.83318 0.59759 6.31103 52 2018-11-18T00:00:00.000000Z SalesQty (7 day median) 0.00000 SalesQty (actual) --- -24.76728 SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -18.77639 SalesQty (naive latest value) 3.00000 SalesQty (actual) -- -7.17425
2 01053_Alcohol_3633522291 3 2018-11-16T00:00:00.000000Z 3.45923 0.21665 5.92187 53 2018-11-19T00:00:00.000000Z SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -16.67129 SalesQty (7 day median) 0.00000 SalesQty (actual) -- -5.25844 SalesQty (14 day median) 1.00000 SalesQty (actual) -- -4.65901
3 01053_Alcohol_3633522291 4 2018-11-16T00:00:00.000000Z 2.57798 -0.55875 5.00040 54 2018-11-20T00:00:00.000000Z SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -17.32528 SalesQty (7 day median) 0.00000 SalesQty (actual) -- -4.84710 SalesQty (14 day median) 1.00000 SalesQty (actual) -- -4.55620
4 01053_Alcohol_3633522291 5 2018-11-16T00:00:00.000000Z 2.57798 -0.60271 4.96715 55 2018-11-21T00:00:00.000000Z SalesQty (7 day mean) 1.28571 SalesQty (actual) --- -17.32528 SalesQty (7 day median) 0.00000 SalesQty (actual) -- -4.84710 SalesQty (14 day median) 1.00000 SalesQty (actual) -- -4.55620
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
832 01115_Pizzas_3633541599 3 2018-11-16T00:00:00.000000Z 10.20752 3.66792 18.94339 7730 2018-11-19T00:00:00.000000Z SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.22766 SalesQty (7 day max) 25.00000 SalesQty (actual) -- -4.19796 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -4.08395
833 01115_Pizzas_3633541599 4 2018-11-16T00:00:00.000000Z 9.44667 2.95212 18.08329 7731 2018-11-20T00:00:00.000000Z SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.75699 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -3.66404 SalesQty (7 day max) 25.00000 SalesQty (actual) -- -2.99822
834 01115_Pizzas_3633541599 5 2018-11-16T00:00:00.000000Z 9.44667 2.69762 17.61851 7732 2018-11-21T00:00:00.000000Z SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.75699 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -3.66404 SalesQty (14 day min) 3.00000 SalesQty (actual) -- -3.56161
835 01115_Pizzas_3633541599 6 2018-11-16T00:00:00.000000Z 7.89606 1.34399 16.73757 7733 2018-11-22T00:00:00.000000Z SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -8.23909 SalesQty (7 day max) 25.00000 SalesQty (actual) -- -3.11896 SalesQty (14 day median) 6.50000 SalesQty (actual) -- -2.58671
836 01115_Pizzas_3633541599 7 2018-11-16T00:00:00.000000Z 9.10800 2.68591 17.80487 7734 2018-11-23T00:00:00.000000Z SalesQty (7 day mean) 11.14286 SalesQty (actual) --- -7.83997 SalesQty (7 day median) 8.00000 SalesQty (actual) -- -6.35970 SalesQty (14 day median) 6.50000 SalesQty (actual) -- -3.53018

837 rows × 23 columns

Deploy a model

After testing predictions with the top-performing model, use the code below to deploy the model to a dedicated prediction server.

now = datetime.now()
current_time = now.strftime("%H:%M:%S")

# DataRobot project
project = dr.Project.get(project_id=PID)
combined_models = project.get_combined_models()
current_combined_model = combined_models[0]

# Model ID
model_id = current_combined_model.id

# Deploy the model
prediction_server = dr.PredictionServer.list()[0]
DATAROBOT_KEY = prediction_server.datarobot_key
PREDICTIONSENDPOINT = prediction_server.url
PREDICTIONSHEADERS = {'Content-Type': 'application/json', 'datarobot-key': '%s' % DATAROBOT_KEY}

deployment = dr.Deployment.create_from_learning_model(
    model_id = model_id,
    label = 'Acme Store Sales Multi Series Deployment ' + str(date.today()) + " " + str(current_time),
    description = 'Acme Sales - Example Deployment ' + str(date.today()),
    default_prediction_server_id = prediction_server.id)

DEPLOYMENT_ID = deployment.id
# Write the Deployment ID to a text file in the current working directory to reference
# later if needed
os.system("echo " + str(deployment.id) + "> ./deployment_id.txt")

print('Deployment Name is: ', deployment.label)
print('Deployment ID is:    ' + str(DEPLOYMENT_ID))
deployment

Set prediction intervals for a deployment

PREDICTION_INTERVAL = 85
dr.Deployment.update_prediction_intervals_settings(deployment, [PREDICTION_INTERVAL])
# Verify settings
dr.Deployment.get_prediction_intervals_settings(deployment)
dr.Deployment.get_prediction_warning_settings(deployment)

Use the code below to list all deployments that you can access.

dr.Deployment.list(order_by=None, search=['Acme'])

Make predictions with a deployment

# Command outline that will need to be run
# python datarobot-predict.py --forecast_point <date> <input-file.csv> <output-file.csv>

To get predictions, open the DataRobot UI and navigate to Predictions > Prediction API. Copy the script to and paste it into a .py file saved as datarobot-predict.py.

from IPython import display
display.Image("./Prediction_Script.png")

Batch predictions

Use the code below to retrieve batch predictions from the deployed model. Be sure to set the forecast point (the last date that the target is populated for).

%%time

!python datarobot-predict.py --forecast_point "2018-11-16" data/Acme_Test_Cluster_0.csv Cluster_0_predictions.csv

Next, read in the prediction file from the deployment.

preds = pd.read_csv('Cluster_0_predictions.csv', 
                    infer_datetime_format=True,
                    parse_dates=['FORECAST_POINT', 'Date']
                    )

preds.head(5)
Series FORECAST_POINT Date FORECAST_DISTANCE SalesQty (actual)_PREDICTION EXPLANATION_1_FEATURE_NAME EXPLANATION_1_STRENGTH EXPLANATION_1_ACTUAL_VALUE EXPLANATION_1_QUALITATIVE_STRENGTH EXPLANATION_2_FEATURE_NAME EXPLANATION_2_STRENGTH EXPLANATION_2_ACTUAL_VALUE EXPLANATION_2_QUALITATIVE_STRENGTH EXPLANATION_3_FEATURE_NAME EXPLANATION_3_STRENGTH EXPLANATION_3_ACTUAL_VALUE EXPLANATION_3_QUALITATIVE_STRENGTH DEPLOYMENT_APPROVAL_STATUS PREDICTION_85_PERCENTILE_LOW PREDICTION_85_PERCENTILE_HIGH
0 01053_Alcohol_3633522291 2018-11-16 2018-11-17 1 3.69103 SalesQty (7 day mean) -17.49522 1.28571 --- SalesQty (7 day median) -15.28814 0.0 --- SalesQty (naive latest value) -6.23734 3.0 -- APPROVED 0.47841 6.14793
1 01053_Alcohol_3633522291 2018-11-16 2018-11-18 2 3.83318 SalesQty (7 day median) -24.76728 0.00000 --- SalesQty (7 day mean) -18.77639 1.28571428571 --- SalesQty (naive latest value) -7.17425 3.0 -- APPROVED 0.59759 6.31103
2 01053_Alcohol_3633522291 2018-11-16 2018-11-19 3 3.45923 SalesQty (7 day mean) -16.67129 1.28571 --- SalesQty (7 day median) -5.25844 0.0 -- SalesQty (14 day median) -4.65901 1.0 -- APPROVED 0.21665 5.92187
3 01053_Alcohol_3633522291 2018-11-16 2018-11-20 4 2.57798 SalesQty (7 day mean) -17.32528 1.28571 --- SalesQty (7 day median) -4.84710 0.0 -- SalesQty (14 day median) -4.55620 1.0 -- APPROVED -0.55875 5.00040
4 01053_Alcohol_3633522291 2018-11-16 2018-11-21 5 2.57798 SalesQty (7 day mean) -17.32528 1.28571 --- SalesQty (7 day median) -4.84710 0.0 -- SalesQty (14 day median) -4.55620 1.0 -- APPROVED -0.60271 4.96715

Use the code below to disable prediction intervals and prediction explanations and then retrieve predictions. Disabling these features can accelerate prediction response time.

dr.Deployment.update_prediction_intervals_settings(deployment, enabled=False, percentiles=None)
# Verify change
dr.Deployment.get_prediction_intervals_settings(deployment)
%%time

!python datarobot-predict.py --forecast_point "2018-11-16" data/Acme_Test_Cluster_0.csv Cluster_0_predictions.csv

Deployment deletion

Before proceeding with deletion, verify it is the correct deployment as there is no way to undo this action.

deployment = Deployment.get(deployment_id=DEPLOYMENT_ID)

print('Deployment Name is: ', deployment.label)
print('Deployment ID is:    ' + str(DEPLOYMENT_ID))
# Delete the deployment
deployment.delete()
# Verify deployment was deleted
dr.Deployment.list(order_by=None, search=['Acme'])

Project deletion

Use the code below to delete all projects created in this notebook.

remove_projects = dr.Project.list(search_params={'project_name': BASE}) 
remove_projects
for p in remove_projects:
    project = Project.get(p.id)
    project.delete()
    remove_projects.remove(project)

remove_projects

You can use the code below to verify projects were successfully deleted.

dr.Project.list(search_params={'project_name': BASE}) 

Updated May 2, 2022
Back to top