Skip to content

Click in-app to access the full platform documentation for your version of DataRobot.

Installation and configuration

When the agent is running, it looks for buffered messages in the configured directory, or a message queuing system and forwards them. In order to forward buffered messages from the MLOps library to DataRobot MLOps, install and configure the MLOps agent as indicated below.

Installing the MLOps agent

Unpacking the MLOps .tar file

> tar -xvf datarobot_mlops_package-*.tar.gz

Updating the configuration file

> cd datarobot_mlops_package-*;
> <your-favorite-editor> ./conf/mlops.agent.conf.yaml

Using the MLOps agent

Running the MLOps agent

Start the agent using the config file:

> cd datarobot_mlops_package-*;
> ./bin/start-agent.sh

Alternatively, start the agent using environment variables:

> export AGENT_CONFIG_YAML=<path/to/conf/mlops.agent.conf.yaml> ; \
         export AGENT_LOG_PROPERTIES=<path/to/conf/mlops.log4j2.properties>; \
         export AGENT_JVM_OPT=-Xmx4G \
         export AGENT_JAR_PATH=<path/to/bin/mlops-agent-ver.jar> \
         sh -x /bin/start-agent.sh

 # AGENT_CONFIG_YAML      environmment variable to override the default path to mlops.agent.conf.yaml
 # AGENT_LOG_PROPERTIES   environmment variable to override the default path to mlops.log4j2.properties
 # AGENT_JVM_OPT          environmment variable to override the default JVM option `-Xmx4G`
 # AGENT_JAR_PATH         environmment variable to override the default JAR file path to agent-<ver>.jar

Checking the agent status

> ./bin/status-agent.sh

Retrieving the agent's status with real-time resource usage

> ./bin/status-agent.sh --verbose

Shutting down the agent

> ./bin/stop-agent.sh

Configuring the MLOps agent

To edit the agent configuration file, conf\mlops.agent.conf.yaml, you must update the values for mlopsUrl and apiToken. By default, the agent will use the file system channel. If you use the file system channel, make sure you create the spooler directory (by default this is /tmp/ta). If you want to use a different channel, update the agent configuration file following the comments in the file.

# URL to the DataRobot MLOps service
mlopsUrl: "https://<DATAROBOT_HOST>"

# DataRobot API token
apiToken: "FILL_IN_HERE"

# Execute the agent once, then exit
runOnce: false

# When dryrun mode is true, do not report the metrics to MLOps service
dryRun: false

# When verifySSL is true, SSL certification validation will be performed when
# connecting to MLOps DataRobot. When verifySSL is false, these checks are skipped.
# Note: It is highly recommended to keep this config variable as true.
verifySSL: true

# Path to the agent's log file
logPath: "./logs/mlops.agent.log"

# Path to write agent stats
statsPath: "/tmp/tracking-agent-stats.json"

# Number of times the agent will retry sending a request to the MLOps service on failure.
httpRetry: 1

# Http client timeout in milliseconds (30sec timeout)
httpTimeout: 30000

# Comment out and configure the lines below for the spooler type(s) you are using.
# Note: the spooler configuration must match that used by the MLOps library.
# Note: Spoolers must be set up before using them.
#       - For the filesystem spooler, create the directory that will be used.
#       - For the SQS spooler, create the queue.
#       - For the PubSub spooler, create the project and topic.
channelConfigs:
  - type: "FS_SPOOL"
    details: {name: "filesystem", directory: "/tmp/ta"}
#  - type: "SQS_SPOOL"
#    details: {name: "sqs", queueUrl: "your SQS queue URL", queueName: "<your AWS SQS queue name>"}
#  - type: "RABBITMQ_SPOOL"
#    details: {name: "rabbit", queueName: <your rabbitmq queue name>, queueUrl: "amqp://<ip address>"}
#  - type: "PUBSUB_SPOOL"
#    details: {name: "pubsub", projectId: <your project ID>, topicName: <your topic name>}

# The number of threads that the agent will launch to process data records.
agentThreadPoolSize: 4

# The maximum number of records each thread will process per fetchNewDataFreq interval.
agentMaxRecordsTask: 100

# Maximum number of records to aggregate before sending to MMM
agentMaxAggregatedRecords: 500

# A timeout for pending records before aggregating and submitting
agentPendingRecordsTimeoutMs: 2500

Updated September 2, 2021
Back to top