Configuring AWS MQ RabbitMQ as an External Service in DataRobot¶

This guide will walk you through the steps to configure AWS MQ RabbitMQ as an external service in the DataRobot application. By following these steps, you will be able to integrate RabbitMQ with DataRobot for efficient data processing and model deployment.

Prerequisites¶

Before you begin, ensure you have the following:

• An AWS account with permissions to create and manage AWS MQ instances.
• A DataRobot account with access to the application.

1: Create an AWS MQ RabbitMQ custom configuration¶

• Config name: Choose a unique name for your Config.
• Choose RabbitMQ as the broker engine.
• Engine version: Select 3.11.2.
• Review your settings and click Create.
• Navigate to the newly create configuration
• set consumer_timeout to 86400000 (which is equivalent to 24 hours).

2: Create an AWS MQ RabbitMQ Instance¶

1. Log in to the AWS Management Console.
2. Navigate to the Amazon MQ service.
3. Click on Create a broker.
4. Choose RabbitMQ as the broker engine.
5. Choose the deployment type
6. SINGLE_INSTANCE: For a simple setup with a single broker.
7. CLUSTER_MULTI_AZ: For a highly available setup across multiple Availability Zones.
8. Configure the broker settings:
9. Broker name: Choose a unique name for your broker.
10. Set up the username and password for RabbitMQ.
11. Engine version: Select 3.11.2.
12. Instance type: Select mq.m5.large.
13. Broker configuration use Select an existing configuration and chose the configuration done on Step 1
14. Configure the Access and security settings:
15. Configure the VPC and subnet settings as needed.
16. Review your settings and click Create broker.

3. Configure DataRobot to Use AWS MQ¶

When DataRobot is configured to utilize an external service, additional YAML override values must be provided.

global:
  rabbitmq:
    internal: false
    hostname: "YOUR_AWS_MQ_INSTANCE_DNS"
    port: 5671
    http_port: 15671
    amqp_protocol: amqps
    http_protocol: https
    auth:
      username: YOUR_AWS_MQ_INSTANCE_USERNAME
      password: YOUR_AWS_MQ_INSTANCE_PASSWORD

Troubleshooting¶

If you encounter issues while connecting DataRobot to RabbitMQ, consider the following:

• Verify that the RabbitMQ broker is running and accessible.
• Check the security group settings in AWS to ensure that the necessary ports are open.
• Ensure that the username and password are correct.
• Review the DataRobot logs for any error messages.

Conclusion¶

You have successfully configured AWS MQ RabbitMQ as an external service in DataRobot. This integration allows you to leverage RabbitMQ's messaging capabilities for enhanced data processing and model deployment. For further assistance, refer to the official documentation for AWS MQ and DataRobot.