# CI/CD setup for application templates
URL: https://docs.datarobot.com/en/docs/wb-apps/app-templates/pulumi-tasks/cicd-tutorial.html

> Set up continuous integration and continuous delivery for application templates using GitLab and GitHub.

# CI/CD setup for application templates

Manually managing custom applications via scripts, ad-hoc commands, or the UI, can quickly become a tedious and error-prone task. Each deployment often requires careful coordination, repeated steps, and constant double-checking to avoid mistakes. Over time, this manual process can increase the risk of human error, inconsistent environments, and accidental downtime.

The goal of [application templates](https://docs.datarobot.com/en/docs/wb-apps/app-templates/index.html) is to provide blueprints for building your own applications. These blueprints offer a "batteries included" starting point from which you can customize and extend the workflow. Getting started with application templates is easy when working with a [codespace](https://docs.datarobot.com/en/docs/workbench/wb-notebook/codespaces/index.html#codespaces). As you progress through the experimentation phase, you're going to want to build production-grade DevOps around your modifications.

This tutorial outlines how to set up Continuous Integration and Continuous Delivery (CI/CD) for two source control platforms, [GitLab](https://gitlab.com) with [GitLab CI/CD pipelines](https://docs.gitlab.com/ci/pipelines/) and [GitHub](https://github.com) with [GitHub actions](https://github.com/features/actions).

This tutorial aims to accomplish the following:

1. Enable merge request or pull request checks for testing, static code analysis, and linting.
2. Enable "Review applications", which deploy a full stack from the branch under a dedicated name for developers to validate and demonstrate proposed changes.
3. Enable continuous delivery upon merging to a shared application deployment that stays continuously updated.
4. Demonstrate how you can use cloud-enabled Pulumi state management for shared and secure Pulumi stack tracking, no matter what environment you have.
5. Show how to integrate CI/CD platform secrets and a variable storage system with application templates to securely manage credentials.

This tutorial breaks down how to tackle all of these goals for each respective platform along the way using forks of the [Talk to My Data Agent](https://docs.datarobot.com/en/docs/workbench/wb-apps/app-templates/at-talk-agent.html) application template as the basis.

## Configure Gitlab merge request tests and linters

> [!NOTE] Code examples
> All the code and live examples of merge requests for Gitlab are available [here](https://gitlab.com/datarobot-oss/demo-data-agent).

To make testing CI the same as local development, use [Task](https://taskfile.dev) to simplify the process. Task is a compatible and lightweight tool for running tasks with a simple definition. The Talk to My Data Agent has both Python and TypeScript components for its React frontend, and Task can help simplify setting up environments and running everything in the application template from one space.

This tutorial provides the [root Taskfile](https://gitlab.com/datarobot-oss/demo-data-agent/-/blob/main/Taskfile.yaml) and the [React/Typescript Taskfile](https://gitlab.com/datarobot-oss/demo-data-agent/-/blob/main/frontend_react/react_src/Taskfile.yaml). They include each other, and make it easier to reference in [.gitlab-ci.yml](https://gitlab.com/datarobot-oss/demo-data-agent/-/blob/main/.gitlab-ci.yml).

The code snippet below from the root `Taskfile.yml` outlines fast and cheap per-merge request testing and linting.

```
version: '3'
dotenv:
  - .env
includes:
  react:
    taskfile: ./frontend_react/react_src/Taskfile.yaml
    dir: ./frontend_react/react_src/
tasks:
...
  python-lint:
    desc: 🧹 Lint Python code
    cmds:
        - ruff format .
        - ruff check . --fix
        - mypy --pretty .
  python-lint-check:
    desc: 🧹 Lint Python code with fixes
    cmds:
        - ruff format --check .
        - ruff check .
        - mypy --pretty .
  lint:
    deps:
      - react:lint
      - python-lint
    desc: 🧹 Lint all code
  lint-check:
    deps:
      - react:lint-check
      - python-lint-check
    desc: 🧹 Lint all code
```

The above snippet includes the React/Typescript Taskfile to bring in its tasks and define the Python tasks for linting. Now you can use those tasks from the Taskfile to define what you need to run linters in GitLab, as shown in `.gitlab-ci.yml` below.

```
image: cimg/python:3.11-node

before_script:
  - pip install go-task-bin
  - task install
  - source .venv/bin/activate

lint:
  stage: check
  script:
    - task lint-check
  only:
    - merge_requests
```

The above snippet pre-installs Task, installs the app template's dependencies using `before_script`, and configures linting. You can then follow the same pattern for testing.

```
test:
  stage: check
  script:
    - task test
  only:
    - merge_requests
```

The above snippet uses the same `check` stage for both testing and linting so that they run in parallel to speed up the merge request checks and deliver faster feedback.

## Review deployments

After configuring testing for `merge_requests`, you can add the support for "Review apps." Use this section to add a "manual" step that is shown in the PR:

This task/pipeline lets developers spin up the entire Talk to My Data Agent stack with a click of the button to share with colleagues, or validate themselves.

The following code examples stem from `.gitlab-ci.yml`. You need to define the variables needed from the [.env-template](https://gitlab.com/datarobot-oss/demo-data-agent/-/blob/main/.env.template?ref_type=heads).

```
variables:
  DATAROBOT_ENDPOINT: https://app.datarobot.com/api/v2
  FRONTEND_TYPE: react
  # The following variables are set on the GitLab CI/CD settings page:
  # DATAROBOT_API_TOKEN: "$DATAROBOT_API_TOKEN"
  # PULUMI_CONFIG_PASSPHRASE: "$PULUMI_CONFIG_PASSPHRASE"
  # OPENAI_API_VERSION: "$OPENAI_API_VERSION"
  # OPENAI_API_BASE: "$OPENAI_API_BASE"
  # OPENAI_API_DEPLOYMENT_ID: "$OPENAI_API_DEPLOYMENT_ID"
  # OPENAI_API_KEY: "$OPENAI_API_KEY"
  # # Used for the Pulumi DIY backend bucket: https://www.pulumi.com/docs/iac/concepts/state-and-backends/#azure-blob-storage
  # AZURE_STORAGE_ACCOUNT: "$AZURE_STORAGE_ACCOUNT"
  # AZURE_STORAGE_KEY: "$AZURE_STORAGE_KEY"
  # GITLAB_API_TOKEN: "$GITLAB_API_TOKEN"
```

For GitLab, define these variables in the project's `ci_cd variables` settings:

You will re-use these variables for both CI and CD, and they will cover the required information for Pulumi and the application (e.g., the LLM keys or any other data connection information required).

For `DATAROBOT_API_TOKEN`, use an API key from a dedicated service account rather than a personal user login. A service account is a DataRobot user created specifically for automation—ask your DataRobot administrator to provision one (for example, `ci-bot@your-org.com`), then generate an API key from the DataRobot UI via the Developer Tools → API Keys & Tools under that account (see [API keys and tools](https://docs.datarobot.com/en/docs/platform/acct-settings/api-key-mgmt.html) for more details).

Assign the account the [Apps Admin](https://docs.datarobot.com/en/docs/reference/misc-ref/rbac-ref.html#apps-admin) role, which grants the permissions needed to create and update custom applications, deployments, and related resources. This ensures pipelines keep working when team members change, and makes it straightforward to audit or revoke CI/CD access independently of individual users.

With the variables configured, move on to the manual stage:

```
review_app:
  stage: review
  script:
  # Installs a "MR review app" for the branch being merged into main
    - curl -fsSL https://get.pulumi.com | sh
    - export PATH="~/.pulumi/bin:$PATH"
    - pulumi login --cloud-url "azblob://dr-ai-apps-pulumi"
    - pulumi stack select --create gitlab-mr-$CI_MERGE_REQUEST_IID
    - pulumi up --yes --stack gitlab-mr-$CI_MERGE_REQUEST_IID
    - echo "Deploying review app for branch gitlab-mr-$CI_MERGE_REQUEST_IID"
    - STACK_OUTPUT="<br><br>$(pulumi stack output --shell)"
    - STACK_OUTPUT="${STACK_OUTPUT//$'\n'/<br>}"
    - |
      curl --header "PRIVATE-TOKEN: $GITLAB_API_TOKEN" \
         --data "body=Review Deployment: $STACK_OUTPUT" \
         "$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes"
  only:
    - merge_requests
  when: manual
```

In this snippet, make a `when: manual` pipeline stage so that it is optional to save resources. You should only do this when a developer feels it's necessary to save on resources. The snippet installs Pulumi, creates a stack associated with the merge request number to make it unique, stands it up, and comments on the request with a link to review the application. See [MR #5](https://gitlab.com/datarobot-oss/demo-data-agent/-/merge_requests/5) for an example of running this live.

As the developer updates the merge request, rerunning this stage updates the application template relying on the idempotent nature of the Pulumi IaC and the DataRobot Declarative API.

To tear the stack down, there is a manual job executable both in the merge request and after the fact for clean up. The snippet below uses the merge request number as a manual variable and then tears it down via `pulumi destroy` and `pulumi stack rm` to keep the centralized Pulumi backend clean afterwards with minimal resource usage.

```
destroy_review_app:
  stage: cleanup
  script:
    - curl -fsSL https://get.pulumi.com | sh
    - export PATH="~/.pulumi/bin:$PATH"
    - pulumi login --cloud-url "azblob://dr-ai-apps-pulumi"
    - pulumi destroy --yes --stack gitlab-mr-$CI_MERGE_REQUEST_IID
    - pulumi stack rm gitlab-mr-$CI_MERGE_REQUEST_IID --yes
    - echo "Destroyed review app stack for MR gitlab-mr-$CI_MERGE_REQUEST_IID"
  only:
    - merge_requests
    - main
  when: manual
  needs:
    - job: review_app
      optional: true
```

## Continuous delivery

The configuration for continuous delivery looks similar to the review apps setup, except it occurs on merge and it has no destroy mechanism. It stays persistent as all of the changes get merged.

Review the relevant pipeline yaml below.

```
deploy_ci:
  stage: deploy
  script:
    - curl -fsSL https://get.pulumi.com | sh
    - export PATH="~/.pulumi/bin:$PATH"
    - pulumi login --cloud-url "azblob://dr-ai-apps-pulumi"
    - pulumi stack select ci
    - pulumi up --yes --stack ci
    - echo "Deployed CI stack"
  only:
    - main
  when: on_success
```

The pipeline is more straightforward than the review app because it has a fixed stack name, and the example uses Azure Blob storage to store the stack to stay persistent. You can also review a [sample execution](https://gitlab.com/datarobot-oss/demo-data-agent/-/jobs/10022973079).

## GitHub actions

GitHub actions is quite similar to GitLab pipelines. You can use this example to achieve the same results using a different Pulumi backend, and the GitHub actions CI/CD configurations.

```
name: Pulumi Deployment
on:
  pull_request:
    types: [opened, synchronize, reopened]
env:
  PULUMI_STACK_NAME: github-pr-${{ github.event.repository.name }}-${{ github.event.number }}
jobs:
  update:
    name: pulumi-update-stack
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Decrypt Secrets
        run: gpg --quiet --batch --yes --decrypt --passphrase="$LARGE_SECRET_PASSPHRASE" --output .env .env.gpg
        env:
          # https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions
          LARGE_SECRET_PASSPHRASE: ${{  secrets.LARGE_SECRET_PASSPHRASE }}
      - uses: actions/setup-python@v5
        with:
          python-version: 3.12
      - name: Install Pulumi
        run: |
          curl -fsSL https://get.pulumi.com | sh
          echo "$HOME/.pulumi/bin" >> $GITHUB_PATH
      - name: Setup Project Dependencies
        run: |
          command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
          uv venv .venv
          source .venv/bin/activate
          uv pip install -r requirements.txt
      - name: Plan Pulumi Update
        id: plan_pulumi_update
        run: |
          source .venv/bin/activate
          export $(grep -v '^#' .env | xargs)
          pulumi stack select --create $PULUMI_STACK_NAME
          pulumi up --yes
          # Store JSON output once and parse it for all values
          PULUMI_OUTPUT=$(pulumi stack output --json)
          APPLICATION_URL=$(echo "$PULUMI_OUTPUT" | jq -r 'to_entries[] | select(.key | startswith("Data Analyst Application")) | .value')
          DEPLOYMENT_URL=$(echo "$PULUMI_OUTPUT" | jq -r 'to_entries[] | select(.key | startswith("Generative Analyst Deployment")) | .value')
          APP_ID=$(echo "$PULUMI_OUTPUT" | jq -r '.DATAROBOT_APPLICATION_ID // empty')
          LLM_ID=$(echo "$PULUMI_OUTPUT" | jq -r '.LLM_DEPLOYMENT_ID // empty')
          echo "application_url=${APPLICATION_URL}" >> $GITHUB_OUTPUT
          echo "deployment_url=${DEPLOYMENT_URL}" >> $GITHUB_OUTPUT
          echo "app_id=${APP_ID}" >> $GITHUB_OUTPUT
          echo "llm_id=${LLM_ID}" >> $GITHUB_OUTPUT
        env:
          PULUMI_ACCESS_TOKEN: ${{  secrets.PULUMI_ACCESS_TOKEN }}
      - name: Comment PR with App URL
        uses: peter-evans/create-or-update-comment@v4
        with:
          token: ${{  secrets.GITHUB_TOKEN }}
          issue-number: ${{  github.event.number }}
          body: |
            # 🚀 Your application is ready!
            ## Application Info
            - **Application URL:** [${{  steps.plan_pulumi_update.outputs.application_url }}](${{  steps.plan_pulumi_update.outputs.application_url }})
            - **Application ID:** `${{  steps.plan_pulumi_update.outputs.app_id }}`
            ## LLM Deployment
            - **Deployment URL:** [${{  steps.plan_pulumi_update.outputs.deployment_url }}](${{  steps.plan_pulumi_update.outputs.deployment_url }})
            - **Deployment ID:** `${{  steps.plan_pulumi_update.outputs.llm_id }}`
            ### Pulumi Stack
            - **Stack Name:** `${{  env.PULUMI_STACK_NAME }}`
```

You now have `pulumi up` running on every pull request and the Pulumi stack is backed up to the cloud, allowing it to be traceable between commits. Once Pulumi completes the changes, you are presented with information about the app. Review this [GitHub pull request](https://github.com/datarobot-forks/demo-talk-to-my-data-agent/pull/1) to see the workflows in action.

There are a few caveats that require additional configuration, outlined in the following sections.

## Pulumi

The easiest and most straightforward way of setting up Pulumi locally is to install it and log in to the [Pulumi cloud backend](https://www.pulumi.com/docs/iac/concepts/state-and-backends/#pulumi-cloud-backend). This will make sure your local Pulumi stack is in sync with a cloud backup and let you access the stack from other edge nodes (e.g., GitHub actions). Just add a [PULUMI_ACCESS_TOKEN](https://www.pulumi.com/docs/pulumi-cloud/access-management/access-tokens/) secret to your [GitHub repository secrets](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions). When `PULUMI_ACCESS_TOKEN` is present as an environment variable, Pulumi authenticates with Pulumi Cloud automatically, so no interactive `pulumi login` step is needed in CI.

If that is a security concern, or you have other reasons not to use the default Pulumi Cloud approach (e.g., due to network access from your environment, cost, or corporate policy), you can rely on [DIY Pulumi backends](https://www.pulumi.com/docs/iac/concepts/state-and-backends/#using-a-diy-backend). The [Gitlab tutorial](https://docs.datarobot.com/en/docs/wb-apps/app-templates/pulumi-tasks/cicd-tutorial.html#review-deployments) uses the Azure DIY backend (line 11 in the code example for this section), for example, which could be used in GitHub with Azure or AWS backends.

The GitHub CI/CD focuses on a recommended Pulumi-based method of making infrastructure changes, but you can follow the GitLab DIY method using Azure as an example if you want to use S3 or another Pulumi state provider.

## GitHub actions secrets

When it comes to managing sensitive data like API keys, database credentials, or cloud provider tokens, GitHub actions offer a built-in way to handle secrets. The most straightforward approach is to store each secret individually in GitHub and inject them into your workflows as environment variables. This method can become messy and error-prone as the number of secrets grows. It's easy to misconfigure variables, leak values into logs, or lose track of what is actually in use.

A more robust alternative, recommended by GitHub itself, is to bundle your secrets into a file (like  .env), encrypt it with [GPG](https://www.gnupg.org/), and store only the encrypted version in your repository. This way, you only need a single decryption key in your GitHub Secrets, and the rest of your secrets stay tightly managed and versioned—without scattering them across your workflow files. It’s a clean, auditable approach that reduces risk and simplifies management at scale.

Whichever approach you choose, ensure that `DATAROBOT_API_TOKEN` is sourced from a dedicated service account rather than a personal user login. If the engineer who originally set up the pipeline leaves the team, a personal token would need to be rotated and every downstream secret updated. A service account token can be revoked and replaced independently. Ask your DataRobot administrator to provision a dedicated automation user, assign it the [Apps Admin](https://docs.datarobot.com/en/docs/reference/misc-ref/rbac-ref.html#apps-admin) role, and generate an API key from Developer Tools → API Key under that account.

The steps below outline how to make that work.

1. Rungpg --symmetric --cipher-algo AES256 .env. You will be prompted to enter a passphrase. Remember the passphrase, because you'll need to create a new secret on GitHub that uses the passphrase as the value.
2. Create a new secret in the GitHub repo that contains the passphrase. For example, create a new secret with the nameLARGE_SECRET_PASSPHRASEand set the value of the secret to the passphrase you used in the previous step.
3. Copy your encrypted file to a path in your repository and commit it. In this example, the encrypted file is .env.gpg.

```
git add .env.gpg
git commit -m "Add new GPG encrypted secret file"
git push
```

## Destroy resources with GitHub actions

After successfully creating a Use Case with all underlying resources, including the actual application, this sections outlines how to delete it. Use GitHub actions to create a workflow with a manual trigger.

Unlike the previous `pulumi up` example that used GPG-encrypted secrets, this manual workflow uses GitHub's environment variables directly. The workflow requires a stack name as input. It uses the `PULUMI_ACCESS_TOKEN` from GitHub secrets to authenticate with Pulumi Cloud.

```
name: Pulumi Stack Destroy
on:
  workflow_dispatch:
    inputs:
      stack_name:
        description: 'Stack name to destroy (e.g. github-pr-foobar-42)'
        required: true
        type: string

env:
  PULUMI_STACK_NAME: github-pr-${{  github.event.repository.name }}-${{  github.event.number }}
  PULUMI_ACCESS_TOKEN: ${{  secrets.PULUMI_ACCESS_TOKEN }}

jobs:
  destroy:
    name: pulumi-destroy-stack
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: 3.12
      - name: Install Pulumi
        run: |
          curl -fsSL https://get.pulumi.com | sh
          echo "$HOME/.pulumi/bin" >> $GITHUB_PATH
      - name: Setup Project Dependencies
        run: |
          command -v uv >/dev/null 2>&1 || curl -LsSf https://astral.sh/uv/install.sh | sh
          uv venv .venv
          source .venv/bin/activate
          uv pip install -r requirements.txt
      - name: Destroy Pulumi Stack
        run: |
          source .venv/bin/activate
          pulumi stack select ${{  github.event.inputs.stack_name }}
          pulumi destroy --yes
          pulumi stack rm --yes ${{  github.event.inputs.stack_name }}
```

Triggering the flow with the stack name you received on your PR will be sufficient to safely destroy all resources tied to the Use Case.

## Pulumi state management

It's recommended to use a centralized Pulumi state backend. One of the primary reasons is that it lets you use Pulumi across machines, across CI/CD platforms, and both in and out of codespaces without losing track of the resources you've provisioned.

With a shared storage Pulumi backend, you can spin up a new codespace in the DataRobot application with your code to create a new experiment or update an existing stack. To do so, it is as simple as cloning the code you want, running `pulumi login <backend>` and then using the snippet below:

```
 pulumi stack ls -a
 ```
 will give you something like
 ```
 NAME                                 LAST UPDATE   RESOURCE COUNT
organization/agent_sre/sredev1       2 months ago  0
organization/talk-to-my-docs/x-dev0  2 days ago    12
dev0                           2 weeks ago   13
ci    6 days ago   13
```

```
pulumi stack select ci
```

With the examples here, you can manage or update your review apps using this approach.

## Conclusion

Bringing DevOps best practices to AI applications is essential for unlocking the full potential of AI for your developers and organization. Integrating modern CI/CD pipelines, review-app flows, and infrastructure as code with Pulumi transforms how your team builds, tests, and delivers AI-powered solutions.

With DataRobot at the core, these DevOps patterns ensure that every change to your AI app is automatically validated, securely deployed, and easily reviewed in a real environment, long before it reaches production. Review apps make collaboration seamless and risk-free, while cloud-backed Pulumi state and secure secrets management keep your infrastructure safe and manageable at scale.

Adopting these approaches means your AI applications can evolve rapidly and reliably, with every team member empowered to contribute and innovate. As you iterate and adapt these patterns, you’ll find that production-grade DevOps for AI is not only achievable, but also a catalyst for delivering real value to your users—securely, confidently, and at the speed of modern business.

---
