MLOps > Deployment > Prepare custom models for deployment > Custom model assembly > Define custom model runtime parameters

Define custom model runtime parameters¶

Add runtime parameters to a custom model through the model metadata, making your custom model code easier to reuse. To define runtime parameters, you can add the following runtimeParameterDefinitions in model-metadata.yaml:

Key	Description
`fieldName`	Define the name of the runtime parameter.
`type`	Define the data type the runtime parameter contains: `string`, `boolean`, `numeric` `credential`, and `deployment`.
`defaultValue`	(Optional) Set the default string value for the runtime parameter (the credential type doesn't support default values).
`minValue`	(Optional) For `numeric` runtime parameters, set the minimum numeric value allowed in the runtime parameter.
`maxValue`	(Optional) For `numeric` runtime parameters, set the maximum numeric value allowed in the runtime parameter.
`credentialType`	(Optional) For `credential` runtime parameters, set the type of credentials the parameter should contain.
`allowEmpty`	(Optional) Set the empty field policy for the runtime parameter: `True`: (Default) Allows an empty runtime parameter. `False`: Enforces providing a value for the runtime parameter before deployment.
`description`	(Optional) A description of the purpose or contents of the runtime parameter.

Default values

If you define a runtime parameter without specifying a defaultValue, the default value is None.

DataRobot reserved runtime parameters¶

The following runtime parameters are reserved by DataRobot for custom model configuration:

Runtime parameter	Type	Description
`CUSTOM_MODEL_WORKERS`	numeric	Allows each replica to handle a set number of concurrent processes. This option is intended for process safe custom models, primarily in generative AI use cases. To determine the appropriate number of concurrent processes to allow per replica, monitor the number of requests and the median response time for the custom model. The median response time for the custom model should be close to the median response time from LLM. If the response time of the custom model exceeds the LLM's response time, stop increasing the number of concurrent processes and instead increase the number of replicas. Default value: `1` Max value: `40`

Custom model process safety

When enabling and configuring CUSTOM_MODEL_WORKERS, ensure that your model is process safe. This configuration option is only intended for process safe custom models, it is not intended for general use with custom models to make them more resource efficient. Only process safe custom models with I/O-bound tasks (like proxy models) benefit from utilizing CPU resources this way.

Define custom model metadata¶

Before you define runtimeParameterDefinitions in model-metadata.yaml, define the custom model metadata required for the target type:

Binary classificationRegressionText generationAnomaly detectionUnstructuredMulticlass

Example: model-metadata.yaml for a binary model


name: binary-example
targetType: binary
type: inference
inferenceModel:
  targetName: target
  positiveClassLabel: "1"
  negativeClassLabel: "0"

Example: model-metadata.yaml for a regression model


name: regression-example
targetType: regression
type: inference

Example: model-metadata.yaml for a text generation model


name: textgeneration-example
targetType: textgeneration
type: inference

Example: model-metadata.yaml for an anomaly detection model


name: anomaly-example
targetType: anomaly
type: inference

Example: model-metadata.yaml for an unstructured model


name: unstructured-example
targetType: unstructured
type: inference

Example: model-metadata.yaml for a multiclass model


name: multiclass-example
targetType: multiclass
type: inference

Then, below the model information, you can provide the runtimeParameterDefinitions:

Example: runtimeParameterDefinitions in model-metadata.yaml


name: runtime-parameter-example
targetType: regression
type: inference

runtimeParameterDefinitions:
- fieldName: my_first_runtime_parameter
  type: string
  description: My first runtime parameter.

- fieldName: runtime_parameter_with_default_value
  type: string
  defaultValue: Default
  description: A string-type runtime parameter with a default value.

- fieldName: runtime_parameter_boolean
  type: boolean
  defaultValue: true
  description: A boolean-type runtime parameter with a default value of true.

- fieldName: runtime_parameter_numeric
  type: numeric
  defaultValue: 0
  minValue: -100
  maxValue: 100
  description: A boolean-type runtime parameter with a default value of 0, a minimum value of -100, and a maximum value of 100.

- fieldName: runtime_parameter_for_credentials
  type: credential
  credentialType: basic
  allowEmpty: false
  description: A runtime parameter containing a dictionary of credentials; credentials must be provided before registering the custom model.

- fieldName: runtime_parameter_for_connected_deployment
  type: deployment
  description: A runtime parameter defined to accept the deployment ID of another deployment to connect to the deployed custom model.

Connected deployments

When you define a deployment runtime parameter, you can provide a deployment ID through the defaultValue or through the the UI. This deployment ID can be used for various purposes, such as connected deployments.

Provide credentials through runtime parameters¶

The credential runtime parameter type supports any credentialType value available in the DataRobot REST API. The credential information included depends on the credentialType, as shown in the examples below:

Credential types

For more information on the supported credential types, see the API reference documentation for credentials.

Credential Type	Example
`basic`	basic: credentialType: basic description: string name: string password: string user: string
`azure`	azure: credentialType: azure description: string name: string azureConnectionString: string
`gcp`	gcp: credentialType: gcp description: string name: string gcpKey: string
`s3`	s3: credentialType: s3 description: string name: string awsAccessKeyId: string awsSecretAccessKey: string awsSessionToken: string
`api_token`	api_token: credentialType: api_token apiToken: string name: string

Provide override values during local development¶

For local development with DRUM, you can specify a .yaml file containing the values of the runtime parameters. The values defined here override the defaultValue set in model-metadata.yaml:

Example: .runtime-parameters.yaml


my_first_runtime_parameter: Hello, world.
runtime_parameter_with_default_value: Override the default value.
runtime_parameter_for_credentials:
  credentialType: basic
  name: credentials
  password: password1
  user: user1

When using DRUM, the new --runtime-params-file option specifies the file containing the runtime parameter values:

Example: --runtime-params-file


drum score --runtime-params-file .runtime-parameters.yaml --code-dir model_templates/python3_sklearn --target-type regression --input tests/testdata/juniors_3_year_stats_regression.csv

Import and use runtime parameters in custom code¶

To import and access runtime parameters, you can import the RuntimeParameters module in your code in custom.py:

Example: custom.py


from datarobot_drum import RuntimeParameters


def mask(value, visible=3):
    return value[:visible] + ("*" * len(value[visible:]))


def transform(data, model):
    print("Loading the following Runtime Parameters:")
    parameter1 = RuntimeParameters.get("my_first_runtime_parameter")
    parameter2 = RuntimeParameters.get("runtime_parameter_with_default_value")
    print(f"\tParameter 1: {parameter1}")
    print(f"\tParameter 2: {parameter2}")

    credentials = RuntimeParameters.get("runtime_parameter_for_credentials")
    if credentials is not None:
        credential_type = credentials.pop("credentialType")
        print(
            f"\tCredentials (type={credential_type}): "
            + str({k: mask(v) for k, v in credentials.items()})
        )
    else:
        print("No credential data set")
    return data

Define custom model runtime parameters¶

DataRobot reserved runtime parameters¶

Define custom model metadata¶

Provide credentials through runtime parameters¶

Provide override values during local development¶

Import and use runtime parameters in custom code¶

Was this page helpful?

Great! Let us know what you found helpful.

What can we do to improve the content?