Agent Assist skill¶

The Agent Assist skill (datarobot-agent-assist) packages the same agent design, coding, and deployment workflows as the dr assist terminal assistant for use inside third-party coding agents. Instead of running Agent Assist in a dedicated terminal session, you install the skill into your preferred agent to drive the workflow through natural language in that environment.

The skill can be found in the DataRobot Agentic Skills repository alongside other DataRobot skills. It guides you through designing an agent, scaffolding from the Agentic Starter application template, implementing tools and code, optionally testing the implementation with Adversarial Swarm Evaluation, and deploying to DataRobot.

Install the skill¶

Install all DataRobot skills (including Agent Assist and the required datarobot-setup skill) with the universal skills installer:

npx ai-agent-skills install datarobot-oss/datarobot-agent-skills

By default, the installer copies skills to all supported coding agents on your machine. To target a specific agent, add the --agent flag:

npx ai-agent-skills install datarobot-oss/datarobot-agent-skills --agent cursor
npx ai-agent-skills install datarobot-oss/datarobot-agent-skills --agent claude

You can also install from agent-specific marketplaces or extension catalogs:

For agent-specific installation details and the full list of supported agents, see DataRobot agentic skills.

Verify installation¶

After installing, ask your coding agent What DataRobot skills do I have available?.

You should see datarobot-setup and datarobot-agent-assist listed. The skill source and helper scripts are in skills/datarobot-agent-assist/ in the repository.

Configure your environment¶

Before designing or coding an agent, run the datarobot-setup skill once per workspace. It checks Python and other dependencies, configures your DataRobot API token, and prepares a project directory.

There are two ways to trigger setup:

• Type: Run datarobot-setup
• Or use the slash command: /datarobot-setup

Accept the prompts to complete environment configuration. If setup runs again in a later session, allow it to ensure the prerequisites are current.

Use Agent Assist in your coding agent¶

Start the Agent Assist workflow with either:

• Run datarobot-agent-assist
• /datarobot-agent-assist

The skill presents the same three options as dr assist:

1. Design an AI agent: Clarify requirements and produce agent_spec.md.
2. Code an AI agent: Scaffold from the agent template and implement the spec.
3. Deploy an AI agent: Deploy the implemented agent to DataRobot.

Typical workflow¶

The following sequence takes you from idea to deployed agent in one session:

1. Design: Describe what you want to build in plain language. Agent Assist asks clarifying questions and writes an agent_spec.md blueprint before any code is written.
2. Rehearse (optional): Run a simulation to chat with your agent concept as an end user. When satisfied, type Done to move to coding. This design rehearsal is separate from Adversarial Swarm Evaluation, which runs after the agent is implemented.

3. Test locally: Say Let's code it. The skill handles dependencies, tools, and project structure. Watch for Lint clean and Tests passed in the output. Run the agent locally with:

   dr run dev
   

   Open the localhost URL shown in the terminal to verify the agent is running.

4. Test (optional): After coding, choose the option to test your agent. Adversarial Swarm Evaluation runs attack, behavior, and persistence tracks against the implementation, proposes patches for breaches, and writes eval_report.md. See Adversarial Swarm Evaluation.

5. Deploy: Enter Deploy my agent to publish it to DataRobot. Deployment typically takes 10 to 30 minutes; keep the session open while it runs. Ask What's the URL for my deployed agent? to retrieve the live URL.

For field definitions and examples of agent_spec.md, see Agent specification reference. For environment variables and configuration used during coding and deployment, see Environment and commands reference.

Adversarial Swarm Evaluation¶

After your agent is coded, Agent Assist can automatically test the implementation before you deploy. The coding agent offers it as a next step after implementation; use this section when you want more detail on what it tests and how the run works.

The swarm runs three tracks against your implemented agent:

When a scenario breaches, the skill proposes a minimal fix (usually a system-prompt addition, sometimes a code guard), applies approved changes, and retests. A typical full swarm run takes up to 5 minutes before any fix rounds.

Prerequisites¶

You need both of the following in the project:

• An agent_spec.md with a system_prompt.
• Implementation code (for example agent.py, myagent.py, tools.py, or app.py).

If either is missing, finish the Design and Code steps first, then return to battle-testing.

Run a swarm evaluation¶

1. Build an agent with Agent Assist in your coding tool (for example DataRobot OpenCode, Claude Code, or Cursor).
2. After the coding step, choose the option to battle-test your agent, or ask in natural language (for example, "Battle-test my agent" or "Run adversarial swarm evaluation").
3. Answer the configuration prompts. The skill asks for a user persona, optional grounding context (sample queries or real requests), how many fixing rounds to allow (default: 3), evaluation mode (standard pass/fail or scored by severity), and which model to use. If the spec includes read-only tools, you can optionally let the swarm perform actual calls for those tools instead of simulating their returns.
4. Review the generated scenarios by track. You can add or remove scenarios, ask the skill to explain any of them, then confirm to start the run.
5. Watch the swarm narrate pass, breach, and error outcomes. For each breach, approve or reject the proposed patch; approved prompt changes are applied to both agent_spec.md and the matching system prompt in the implementation so they stay in sync.
6. When convergence finishes, review eval_report.md.

The report includes pass/fail outcomes, unresolved or exhausted scenarios, readiness to deploy, and a Changes Applied list of patches made during the run. Intermediate swarm artifacts are written under .datarobot/swarm/ and removed when the report step completes. Keep eval_report.md (and evaluation_criteria.md if you want a record of the scenarios) for review before deployment.

After the report¶

From the post-run menu you can:

• Review eval_report.md for outcomes and unresolved scenarios.
• Re-run the simulation after further changes.
• Test locally with the project's usual local-run command.
• Deploy the hardened agent to DataRobot.

If any scenario is marked exhausted (could not be resolved within the fixing-round limit), address those cases before deploying. The report calls them out explicitly.

Gap analysis¶

Adversarial Swarm Evaluation hardens an agent you built with Agent Assist. Gap analysis serves a different purpose: assessing whether an agent repository is production-ready, regardless of whether the agent was built with DataRobot.

The gap analysis skill (datarobot-gap-analysis) scores any agent repository, whether supplied as a GitHub URL or a local path, against an enterprise-readiness framework. Unlike Agent Assist, it needs nothing but the repository. There is no requirement for an agent_spec.md or a prior design session, allowing it to work on agents built entirely outside DataRobot.

The skill is installed with the same command as Agent Assist. See Install the skill.

The skill scores seven risk pillars across four evaluation layers. Findings are grouped in the report by pillar: Security (SEC), Identity and access (IDN), AI and LLM governance (AIG), Operations and observability (OPS), Reliability (REL), IT conformance (ITA), and Regulatory and policy (POL).

Prerequisites¶

The skill checks for the following prerequisites and reports any that are missing:

• Git, Python 3.11 or later, and uv.
• The DataRobot CLI, authenticated with dr auth login. The code reasoning and regulatory layers run through dr opencode; the skill skips those layers if the CLI isn't available.
• (Optional) trivy, gitleaks, and hadolint. The skill uses whichever scanners are installed, and the report lists any checks it skipped.

Run a gap analysis¶

To run the analysis:

1. Start the skill with Run datarobot-gap-analysis, the /datarobot-gap-analysis slash command, or a natural language request such as "Check whether this agent is production-ready."

2. Provide the repository as a GitHub URL or a local path. For a private repository, set GITHUB_TOKEN in your environment. Optionally, you can provide:

   • A branch, tag, or commit to scan. Without one, the skill scans the repository's default branch.
   • A path to an organization policy file. Without a policy file, the skill uses its built-in default policy.

3. Wait for the scan to finish. The skill reports the results of each layer as it completes. On a large repository, a broad scan at the highest reasoning effort can take several minutes. To shorten the run, ask the skill to lower the reasoning effort, or to run without LLM checks.

4. Review the report. The skill prints a link to an HTML report and summarizes the finding count, the severity breakdown, the recommended approach, and the highest-severity findings with file and line evidence.

The run writes gap-report.md, an HTML report alongside it, and gap-findings.json.

Run without LLM checks¶

For a quicker pass, ask the skill for a run without LLM checks, the layers that use dr opencode. The skill then works as follows:

• The deterministic scanning and policy conformance layers run as usual.
• The code reasoning layer is skipped.
• The regulatory layer retrieves your policy but doesn't evaluate the code against it. It lists each requirement as required but not assessed.

This mode still needs DataRobot credentials so that the regulatory layer can retrieve your policy. The deterministic scanning and policy conformance layers run without them.

Remediation approach¶

The report recommends one of three approaches, based on what it found:

For each finding, the report gives you one of two ways to fix it:

• Deterministic fixes come with a fix command you can copy and run.
• Other findings come with a self-contained prompt that includes the citation, evidence, and fix guidance. Paste it into your coding agent.

Regulatory findings¶

Regulatory findings come from your organization's own DataRobot risk management policy, not a fixed checklist. Each finding names the DataRobot platform feature that meets the requirement, such as deployment monitoring, GenAI guards, RBAC, or Model Registry documentation, and explains how to enable it.

Because these findings are usually satisfied by adopting a platform feature rather than by changing code, their remediation often points toward deploying the agent through DataRobot. This is a pre-deployment check. It doesn't inspect deployed entities. Instead, it assesses whether the repository would satisfy your policy once deployed.

If the skill can't reach DataRobot risk management, either because the feature isn't enabled for your organization or because no matching policy exists, the regulatory layer reports no findings. The report explains why.

Re-platform the agent¶

When structural gaps dominate, the skill recommends re-platforming: rebuilding the agent on the DataRobot application framework, carrying your existing business logic across. A scaffolded replacement gets logging, tracing, monitoring, guards, and RBAC by construction, which is how a repository with many unsatisfied regulatory findings becomes compliant.

What re-platforming involves depends on how the repository is built:

• The repository already uses DataRobot application template components. Keep the application. The usual gap is that the agent or LLM calls don't go through a DataRobot deployment. To fix this, add one. The skill does not regenerate the repository in this case.
• The repository doesn't use application template components. The skill hands off to Agent Assist. Agent Assist pulls the agent's business logic into an agent_spec.md file for you to review. After you approve the spec, it builds a replacement application from that spec.

The skill continues using the agent framework already in the repository unless you ask to change it.

Agent Assist skill vs. dr assist¶

Both paths use the same agent_spec.md format, agent template, and deployment model. Choose the implementation that fits your workflow. Adversarial Swarm Evaluation is available through the Agent Assist skill experience in supported coding agents.