# Custom metrics > Custom metrics - Delete a custom metric associated with a deployment. This Markdown file sits beside the HTML page at the same path (with a `.md` suffix). It summarizes the topic and lists links for tools and LLM context. Companion generated at `2026-05-06T18:17:09.823211+00:00` (UTC). ## Primary page - [Custom metrics](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html): Full documentation for this topic (HTML). ## Sections on this page - [classdatarobot.models.deployment.custom_metrics.CustomMetric](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric): In-page section heading. - [classmethodcreate(name, deployment_id, units, is_model_specific, aggregation_type, time_step='hour', directionality=None, description=None, baseline_value=None, value_column_name=None, sample_count_column_name=None, timestamp_column_name=None, timestamp_format=None, batch_column_name=None, categories=None, is_geospatial=None, geospatial_segment_attribute=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.create): In-page section heading. - [classmethodget(deployment_id, custom_metric_id)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.get): In-page section heading. - [classmethodlist(deployment_id)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.list): In-page section heading. - [classmethoddelete(deployment_id, custom_metric_id)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.delete): In-page section heading. - [update(name=None, units=None, aggregation_type=None, directionality=None, time_step=None, description=None, baseline_value=None, value_column_name=None, sample_count_column_name=None, timestamp_column_name=None, timestamp_format=None, batch_column_name=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.update): In-page section heading. - [unset_baseline()](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.unset_baseline): In-page section heading. - [submit_values(data, model_id=None, model_package_id=None, dry_run=False)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.submit_values): In-page section heading. - [submit_single_value(value, model_id=None, model_package_id=None, dry_run=False, segments=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.submit_single_value): In-page section heading. - [submit_values_from_catalog(dataset_id, model_id=None, model_package_id=None, batch_id=None, segments=None, geospatial=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.submit_values_from_catalog): In-page section heading. - [get_values_over_time(start, end, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None, bucket_size='P7D')](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.get_values_over_time): In-page section heading. - [get_values_over_space(start=None, end=None, model_id=None, model_package_id=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.get_values_over_space): In-page section heading. - [get_summary(start, end, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.get_summary): In-page section heading. - [get_values_over_batch(batch_ids=None, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.get_values_over_batch): In-page section heading. - [get_batch_summary(batch_ids=None, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetric.get_batch_summary): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime): In-page section heading. - [classmethodget(deployment_id, custom_metric_id, start, end, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None, bucket_size='P7D')](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime.get): In-page section heading. - [propertybucket_values: Dict\[datetime, int\]](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime.bucket_values): In-page section heading. - [propertybucket_sample_sizes: Dict\[datetime, int\]](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime.bucket_sample_sizes): In-page section heading. - [get_buckets_as_dataframe()](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime.get_buckets_as_dataframe): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.CustomMetricSummary](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricSummary): In-page section heading. - [classmethodget(deployment_id, custom_metric_id, start, end, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricSummary.get): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch): In-page section heading. - [classmethodget(deployment_id, custom_metric_id, batch_ids=None, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch.get): In-page section heading. - [propertybucket_values: Dict\[str, int\]](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch.bucket_values): In-page section heading. - [propertybucket_sample_sizes: Dict\[str, int\]](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch.bucket_sample_sizes): In-page section heading. - [get_buckets_as_dataframe()](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch.get_buckets_as_dataframe): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.CustomMetricBatchSummary](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricBatchSummary): In-page section heading. - [classmethodget(deployment_id, custom_metric_id, batch_ids=None, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricBatchSummary.get): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.HostedCustomMetricTemplate](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricTemplate): In-page section heading. - [classmethodlist(search=None, order_by=None, metric_type=None, offset=None, limit=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricTemplate.list): In-page section heading. - [classmethodget(template_id)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricTemplate.get): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.HostedCustomMetric](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetric): In-page section heading. - [classmethodlist(job_id, skip=None, limit=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetric.list): In-page section heading. - [classmethodcreate_from_template(template_id, deployment_id, job_name, custom_metric_name, job_description=None, custom_metric_description=None, sidecar_deployment_id=None, baseline_value=None, timestamp=None, value=None, sample_count=None, batch=None, schedule=None, parameter_overrides=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetric.create_from_template): In-page section heading. - [classmethodcreate_from_custom_job(custom_job_id, deployment_id, name, description=None, baseline_value=None, timestamp=None, value=None, sample_count=None, batch=None, schedule=None, parameter_overrides=None, geospatial_segment_attribute=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetric.create_from_custom_job): In-page section heading. - [update(name=None, description=None, units=None, directionality=None, aggregation_type=None, baseline_value=None, timestamp=None, value=None, sample_count=None, batch=None, schedule=None, parameter_overrides=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetric.update): In-page section heading. - [delete()](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetric.delete): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.DeploymentDetails](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.DeploymentDetails): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.MetricBaselineValue](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.MetricBaselineValue): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.SampleCountField](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.SampleCountField): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.ValueField](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.ValueField): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.MetricTimestampSpoofing](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.MetricTimestampSpoofing): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.BatchField](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.BatchField): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.HostedCustomMetricBlueprint](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricBlueprint): In-page section heading. - [classmethodget(custom_job_id)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricBlueprint.get): In-page section heading. - [classmethodcreate(custom_job_id, directionality, units, type, time_step, is_model_specific, is_geospatial=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricBlueprint.create): In-page section heading. - [update(directionality=None, units=None, type=None, time_step=None, is_model_specific=None, is_geospatial=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.HostedCustomMetricBlueprint.update): In-page section heading. - [classdatarobot.models.deployment.custom_metrics.CustomMetricValuesOverSpace](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverSpace): In-page section heading. - [classmethodget(deployment_id, custom_metric_id, start=None, end=None, model_package_id=None, model_id=None)](https://docs.datarobot.com/en/docs/api/reference/sdk/custom-metrics.html#datarobot.models.deployment.custom_metrics.CustomMetricValuesOverSpace.get): In-page section heading. ## Related documentation - [Developer documentation](https://docs.datarobot.com/en/docs/api/index.html): Linked from this page. - [API reference](https://docs.datarobot.com/en/docs/api/reference/index.html): Linked from this page. - [Python API client](https://docs.datarobot.com/en/docs/api/reference/sdk/index.html): Linked from this page. - [Monitoring](https://docs.datarobot.com/en/docs/api/reference/sdk/tag-observability.html): Linked from this page. ## Documentation content ### class datarobot.models.deployment.custom_metrics.CustomMetric A DataRobot custom metric. Added in version v3.4. - Variables: #### classmethod create(name, deployment_id, units, is_model_specific, aggregation_type, time_step='hour', directionality=None, description=None, baseline_value=None, value_column_name=None, sample_count_column_name=None, timestamp_column_name=None, timestamp_format=None, batch_column_name=None, categories=None, is_geospatial=None, geospatial_segment_attribute=None) Create a custom metric for a deployment - Parameters: - Returns: The custom metric object. - Return type: CustomMetric > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > from datarobot.enums import CustomMetricAggregationType, CustomMetricDirectionality > > custom_metric = CustomMetric.create( > deployment_id="5c939e08962d741e34f609f0", > name="Sample metric", > units="Y", > baseline_value=12, > is_model_specific=True, > aggregation_type=CustomMetricAggregationType.AVERAGE, > directionality=CustomMetricDirectionality.HIGHER_IS_BETTER > ) > ``` #### classmethod get(deployment_id, custom_metric_id) Get a custom metric for a deployment - Parameters: - Returns: The custom metric object. - Return type: CustomMetric > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > > custom_metric.id > >>>'65f17bdcd2d66683cdfc1113' > ``` #### classmethod list(deployment_id) List all custom metrics for a deployment - Parameters: deployment_id ( str ) – The ID of the deployment. - Returns: custom_metrics – A list of custom metrics objects. - Return type: list > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metrics = CustomMetric.list(deployment_id="5c939e08962d741e34f609f0") > custom_metrics[0].id > >>>'65f17bdcd2d66683cdfc1113' > ``` #### classmethod delete(deployment_id, custom_metric_id) Delete a custom metric associated with a deployment. - Parameters: - Return type: None > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > CustomMetric.delete( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > ``` #### update(name=None, units=None, aggregation_type=None, directionality=None, time_step=None, description=None, baseline_value=None, value_column_name=None, sample_count_column_name=None, timestamp_column_name=None, timestamp_format=None, batch_column_name=None) Update metadata of a custom metric - Parameters: - Returns: The custom metric object. - Return type: CustomMetric > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > from datarobot.enums import CustomMetricAggregationType, CustomMetricDirectionality > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > custom_metric = custom_metric.update( > deployment_id="5c939e08962d741e34f609f0", > name="Sample metric", > units="Y", > baseline_value=12, > is_model_specific=True, > aggregation_type=CustomMetricAggregationType.AVERAGE, > directionality=CustomMetricDirectionality.HIGHER_IS_BETTER > ) > ``` #### unset_baseline() Unset the baseline value of a custom metric - Return type: None > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > from datarobot.enums import CustomMetricAggregationType, CustomMetricDirectionality > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > custom_metric.baseline_values > >>> [{'value': 12.0}] > custom_metric.unset_baseline() > custom_metric.baseline_values > >>> [] > ``` #### submit_values(data, model_id=None, model_package_id=None, dry_run=False) Submit aggregated custom metrics values from JSON. - Parameters: - Return type: None > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > > # data for values over time > data = [{ > 'value': 12, > 'sample_size': 3, > 'timestamp': '2024-03-15T14:00:00' > }] > > # data witch association ID > data = [{ > 'value': 12, > 'sample_size': 3, > 'timestamp': '2024-03-15T14:00:00', > 'association_id': '65f44d04dbe192b552e752ed' > }] > > # data for batches > data = [{ > 'value': 12, > 'sample_size': 3, > 'batch': '65f44c93fedc5de16b673a0d' > }] > > # for deployment specific metrics > custom_metric.submit_values(data=data) > > # for model specific metrics pass model_package_id or model_id > custom_metric.submit_values(data=data, model_package_id="6421df32525c58cc6f991f25") > > # dry run > custom_metric.submit_values(data=data, model_package_id="6421df32525c58cc6f991f25", dry_run=True) > ``` #### submit_single_value(value, model_id=None, model_package_id=None, dry_run=False, segments=None) Submit a single custom metric value at the current moment. - Parameters: - Return type: None > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > > # for deployment specific metrics > custom_metric.submit_single_value(value=121) > > # for model specific metrics pass model_package_id or model_id > custom_metric.submit_single_value(value=121, model_package_id="6421df32525c58cc6f991f25") > > # dry run > custom_metric.submit_single_value(value=121, model_package_id="6421df32525c58cc6f991f25", dry_run=True) > > # for segmented analysis > segments = [{"name": "custom_seg", "value": "val_1"}] > custom_metric.submit_single_value(value=121, model_package_id="6421df32525c58cc6f991f25", segments=segments) > ``` #### submit_values_from_catalog(dataset_id, model_id=None, model_package_id=None, batch_id=None, segments=None, geospatial=None) Submit aggregated custom metrics values from dataset (AI catalog). The names of the columns in the dataset should correspond to the names of the columns that were defined in the custom metric. In addition, the format of the timestamps should also be the same as defined in the metric. - Parameters: - Return type: None > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > > # for deployment specific metrics > custom_metric.submit_values_from_catalog(dataset_id="61093144cabd630828bca321") > > # for model specific metrics pass model_package_id or model_id > custom_metric.submit_values_from_catalog( > dataset_id="61093144cabd630828bca321", > model_package_id="6421df32525c58cc6f991f25" > ) > > # for segmented analysis > segments = [{"name": "custom_seg", "column": "column_with_segment_values"}] > custom_metric.submit_values_from_catalog( > dataset_id="61093144cabd630828bca321", > model_package_id="6421df32525c58cc6f991f25", > segments=segments > ) > ``` #### get_values_over_time(start, end, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None, bucket_size='P7D') Retrieve values of a single custom metric over a time period. - Parameters: - Returns: custom_metric_over_time – The queried custom metric values over time information. - Return type: CustomMetricValuesOverTime > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > from datetime import datetime, timedelta > > now=datetime.now() > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > values_over_time = custom_metric.get_values_over_time(start=now - timedelta(days=7), end=now) > > values_over_time.bucket_values > >>> {datetime.datetime(2024, 3, 22, 14, 0, tzinfo=tzutc()): 1.0, > >>> datetime.datetime(2024, 3, 22, 15, 0, tzinfo=tzutc()): 123.0}} > > values_over_time.bucket_sample_sizes > >>> {datetime.datetime(2024, 3, 22, 14, 0, tzinfo=tzutc()): 1, > >>> datetime.datetime(2024, 3, 22, 15, 0, tzinfo=tzutc()): 1}} > > values_over_time.get_buckets_as_dataframe() > >>> start end value sample_size > >>> 0 2024-03-21 16:00:00+00:00 2024-03-21 17:00:00+00:00 NaN NaN > >>> 1 2024-03-21 17:00:00+00:00 2024-03-21 18:00:00+00:00 NaN NaN > ``` #### get_values_over_space(start=None, end=None, model_id=None, model_package_id=None) Retrieve values of a custom metric over space. - Parameters: - Returns: custom_metric_over_space – The queried custom metric values over space information. - Return type: CustomMetricValuesOverSpace > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > values_over_space = custom_metric.get_values_over_space(model_package_id='6421df32525c58cc6f991f25') > ``` #### get_summary(start, end, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None) Retrieve the summary of a custom metric over a time period. - Parameters: - Returns: custom_metric_summary – The summary of the custom metric. - Return type: CustomMetricSummary > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > from datetime import datetime, timedelta > > now=datetime.now() > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > summary = custom_metric.get_summary(start=now - timedelta(days=7), end=now) > > print(summary) > >> "CustomMetricSummary(2024-03-21 15:52:13.392178+00:00 - 2024-03-22 15:52:13.392168+00:00: > {'id': '65fd9b1c0c1a840bc6751ce0', 'name': 'Test METRIC', 'value': 215.0, 'sample_count': 13, > 'baseline_value': 12.0, 'percent_change': 24.02})" > ``` #### get_values_over_batch(batch_ids=None, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None) Retrieve values of a single custom metric over batches. - Parameters: - Returns: custom_metric_over_batch – The queried custom metric values over batch information. - Return type: CustomMetricValuesOverBatch > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > # all batch metrics all model specific > values_over_batch = custom_metric.get_values_over_batch(model_package_id='6421df32525c58cc6f991f25') > > values_over_batch.bucket_values > >>> {'6572db2c9f9d4ad3b9de33d0': 35.0, '6572db2c9f9d4ad3b9de44e1': 105.0} > > values_over_batch.bucket_sample_sizes > >>> {'6572db2c9f9d4ad3b9de33d0': 6, '6572db2c9f9d4ad3b9de44e1': 8} > > values_over_batch.get_buckets_as_dataframe() > >>> batch_id batch_name value sample_size > >>> 0 6572db2c9f9d4ad3b9de33d0 Batch 1 - 03/26/2024 13:04:46 35.0 6 > >>> 1 6572db2c9f9d4ad3b9de44e1 Batch 2 - 03/26/2024 13:06:04 105.0 8 > ``` #### get_batch_summary(batch_ids=None, model_package_id=None, model_id=None, segment_attribute=None, segment_value=None) Retrieve the summary of a custom metric over a batch. - Parameters: - Returns: custom_metric_summary – The batch summary of the custom metric. - Return type: CustomMetricBatchSummary > [!NOTE] Examples > ``` > from datarobot.models.deployment import CustomMetric > > custom_metric = CustomMetric.get( > deployment_id="5c939e08962d741e34f609f0", > custom_metric_id="65f17bdcd2d66683cdfc1113" > ) > # all batch metrics all model specific > batch_summary = custom_metric.get_batch_summary(model_package_id='6421df32525c58cc6f991f25') > > print(batch_summary) > >> CustomMetricBatchSummary({'id': '6605396413434b3a7b74342c', 'name': 'batch metric', 'value': 41.25, > 'sample_count': 28, 'baseline_value': 123.0, 'percent_change': -66.46}) > ``` ### class datarobot.models.deployment.custom_metrics.CustomMetricValuesOverTime Custom metric over time information. Added in version v3.4. - Variables: #### classmethod get(deployment_id, custom_metric_id, start, end, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None, bucket_size='P7D') Retrieve values of a single custom metric over a time period. - Parameters: - Returns: custom_metric_over_time – The queried custom metric values over time information. - Return type: CustomMetricValuesOverTime #### property bucket_values : Dict[datetime, int] The metric value for all time buckets, keyed by start time of the bucket. - Returns: bucket_values - Return type: Dict #### property bucket_sample_sizes : Dict[datetime, int] The sample size for all time buckets, keyed by start time of the bucket. - Returns: bucket_sample_sizes - Return type: Dict #### get_buckets_as_dataframe() Retrieves all custom metrics buckets in a pandas DataFrame. - Returns: buckets - Return type: pd.DataFrame ### class datarobot.models.deployment.custom_metrics.CustomMetricSummary The summary of a custom metric. Added in version v3.4. - Variables: #### classmethod get(deployment_id, custom_metric_id, start, end, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None) Retrieve the summary of a custom metric over a time period. - Parameters: - Returns: custom_metric_summary – The summary of the custom metric. - Return type: CustomMetricSummary ### class datarobot.models.deployment.custom_metrics.CustomMetricValuesOverBatch Custom metric over batch information. Added in version v3.4. - Variables: #### classmethod get(deployment_id, custom_metric_id, batch_ids=None, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None) Retrieve values of a single custom metric over batches. - Parameters: - Returns: custom_metric_over_batch – The queried custom metric values over batch information. - Return type: CustomMetricValuesOverBatch #### property bucket_values : Dict[str, int] The metric value for all batch buckets, keyed by batch ID - Returns: bucket_values - Return type: Dict #### property bucket_sample_sizes : Dict[str, int] The sample size for all batch buckets, keyed by batch ID. - Returns: bucket_sample_sizes - Return type: Dict #### get_buckets_as_dataframe() Retrieves all custom metrics buckets in a pandas DataFrame. - Returns: buckets - Return type: pd.DataFrame ### class datarobot.models.deployment.custom_metrics.CustomMetricBatchSummary The batch summary of a custom metric. Added in version v3.4. - Variables: metric ( Dict ) – The summary of the batch custom metric. #### classmethod get(deployment_id, custom_metric_id, batch_ids=None, model_id=None, model_package_id=None, segment_attribute=None, segment_value=None) Retrieve the summary of a custom metric over a batch. - Parameters: - Returns: custom_metric_summary – The batch summary of the custom metric. - Return type: CustomMetricBatchSummary ### class datarobot.models.deployment.custom_metrics.HostedCustomMetricTemplate Template for hosted custom metric. #### classmethod list(search=None, order_by=None, metric_type=None, offset=None, limit=None) List all hosted custom metric templates. - Parameters: - Returns: templates - Return type: List[HostedCustomMetricTemplate] #### classmethod get(template_id) Get a hosted custom metric template by ID. - Parameters: template_id ( str ) – ID of the template. - Returns: template - Return type: HostedCustomMetricTemplate ### class datarobot.models.deployment.custom_metrics.HostedCustomMetric Hosted custom metric. #### classmethod list(job_id, skip=None, limit=None) List all hosted custom metrics for a job. - Parameters: job_id ( str ) – ID of the job. - Returns: metrics - Return type: List[HostedCustomMetric] #### classmethod create_from_template(template_id, deployment_id, job_name, custom_metric_name, job_description=None, custom_metric_description=None, sidecar_deployment_id=None, baseline_value=None, timestamp=None, value=None, sample_count=None, batch=None, schedule=None, parameter_overrides=None) Create a hosted custom metric from a template. A shortcut for 2 calls: Job.from_custom_metric_template(template_id) HostedCustomMetrics.create_from_custom_job() - Parameters: - Returns: metric - Return type: HostedCustomMetric #### classmethod create_from_custom_job(custom_job_id, deployment_id, name, description=None, baseline_value=None, timestamp=None, value=None, sample_count=None, batch=None, schedule=None, parameter_overrides=None, geospatial_segment_attribute=None) Create a hosted custom metric from existing custom job. - Parameters: - Returns: metric - Return type: HostedCustomMetric #### update(name=None, description=None, units=None, directionality=None, aggregation_type=None, baseline_value=None, timestamp=None, value=None, sample_count=None, batch=None, schedule=None, parameter_overrides=None) Update the hosted custom metric. - Parameters: - Returns: metric - Return type: HostedCustomMetric #### delete() Delete the hosted custom metric. - Return type: None ### class datarobot.models.deployment.custom_metrics.DeploymentDetails Information about a hosted custom metric deployment. ### class datarobot.models.deployment.custom_metrics.MetricBaselineValue The baseline values for a custom metric. ### class datarobot.models.deployment.custom_metrics.SampleCountField A weight column used with columnar datasets if pre-aggregated metric values are provided. ### class datarobot.models.deployment.custom_metrics.ValueField A custom metric value source for when reading values from a columnar dataset like a file. ### class datarobot.models.deployment.custom_metrics.MetricTimestampSpoofing Custom metric timestamp spoofing. Occurs when reading values from a file, like a dataset. By default, replicates pd.to_datetime formatting behavior. ### class datarobot.models.deployment.custom_metrics.BatchField A custom metric batch ID source for when reading values from a columnar dataset like a file. ### class datarobot.models.deployment.custom_metrics.HostedCustomMetricBlueprint Hosted custom metric blueprints provide an option to share custom metric settings between multiple custom metrics sharing the same custom jobs. When a custom job of a hosted custom metric type is connected to the deployment, all the custom metric parameters from the blueprint are automatically copied. #### classmethod get(custom_job_id) Get a hosted custom metric blueprint. - Parameters: custom_job_id ( str ) – ID of the custom job. - Returns: blueprint - Return type: HostedCustomMetricBlueprint #### classmethod create(custom_job_id, directionality, units, type, time_step, is_model_specific, is_geospatial=None) Create a hosted custom metric blueprint. - Parameters: - Returns: blueprint - Return type: HostedCustomMetricBlueprint #### update(directionality=None, units=None, type=None, time_step=None, is_model_specific=None, is_geospatial=None) Update a hosted custom metric blueprint. - Parameters: - Returns: updated_blueprint - Return type: HostedCustomMetricBlueprint ### class datarobot.models.deployment.custom_metrics.CustomMetricValuesOverSpace Custom metric values over space. Added in version v3.7. - Variables: #### classmethod get(deployment_id, custom_metric_id, start=None, end=None, model_package_id=None, model_id=None) Retrieve custom metric values over space. - Parameters: - Returns: values_over_space – Custom metric values over geospatial hexagons. - Return type: CustomMetricValuesOverSpace