LLM gateway permissions and evaluation¶
This page describes how DataRobot evaluates whether a user can access an LLM, given the policies configured in LLM gateway management.
LLM rules take precedence over publisher rules, which take precedence over provider rules. Evaluation uses the most specific level that has any rule. If that level is empty, access is determined by the parent. If every level is empty, access is denied.
Within the level that applies, any matching Deny overrides any matching Allow. A more specific Allow cannot override a Deny at the same level.
Terms¶
The following describes the terms used to describe LLM gateway management.
Everything configured on a single resource (one LLM, one publisher, or one provider). A permission can contain rules for many subjects at once—the organization, several groups, and several individual users.
A subject and state pair inside a permission. For each resource and subject combination, the state is Allow, Deny, or unset (the permission does not mention that subject). For example, one permission on azure/gpt-4o might contain:
organization:acme allow
group:engineering deny
group:contractors deny
A rule applied to group:engineering matches every user in that group; a rule applied to organization:acme matches every user in the organization.
An LLM, publisher, or provider. Each LLM belongs to one publisher. Each publisher belongs to one provider.
provider
└── publisher
└── llm
azure
└── azure__openai
└── azure/gpt-4o
└── azure/gpt-4o-mini
Who a rule applies to: organization, group, or user. A user belongs to one organization but can be a member of any number of groups. A request for that user matches their user ID, every group they belong to, and their organization.
How access is evaluated¶
When a user requests an LLM, DataRobot evaluates access in two steps.
-
Select the level. Check LLM > publisher > provider. Use the first level that has any rule. That level alone determines access. Rules from other levels are not combined.
条件 Level that applies The LLM has any rule. LLM The LLM has no rule, and the publisher has any rule. パブリッシャー The LLM and publisher have no rules. プロバイダー Every level is empty. User is denied -
Evaluate matching rules. On the selected level, apply every rule whose subject matches the user (
effective = allow_set − deny_set).条件 出力 Any matching Deny. Denied No matching Allow. Denied At least one matching Allow and no matching Deny. Allowed
If access is denied, the user receives 404 - Not found with the message Model not found in catalog.
Deleting a policy in LLM gateway management clears every rule on that resource. The permission becomes empty, evaluation moves to the parent, and the resource inherits the parent policy. If you remove the last rule on a publisher, the publisher badge becomes Inherits.
A new organization with no policies has no accessible LLMs. Allow the organization at the provider level for each provider you want to open.
Evaluation examples¶
The following examples use organization Acme, groups Engineering and Executives, and users Alice, Bob, and Carol.
An LLM-level rule replaces parent rules
The following policy is in effect:
provider azure organization:acme allow
publisher azure__openai user:alice allow
llm azure/gpt-4o group:engineering deny
Alice is a member of Engineering in the Acme organization.
The LLM has a rule, so DataRobot evaluates that level first. Alice matches the Engineering Deny, so access to gpt-4o is denied. Other models from publisher level are allowed to Alice.
Deny overrides Allow at the same level
The following policy is in effect on one resource:
user:bob allow
group:eng-leads allow
group:engineering deny
group:contractors allow
organization:acme allow
Bob is a member of Eng-leads, Engineering, in the Acme organization.
| ルール | Matches Bob | 設定 |
|---|---|---|
| user:bob | はい | 許可する |
| group:eng-leads | はい | 許可する |
| group:engineering | はい | 拒否する |
| group:contractors | いいえ | — |
| organization:acme | はい | 許可する |
Allow and Deny both match, so access is denied. The user-level Allow does not override the group-level Deny.
Organization Deny cannot be overridden by a group Allow
The following policy is in effect:
provider azure organization:acme deny
provider azure group:engineering allow
Bob is a member of Engineering and organization Acme. He matches the organization Deny, so access is denied. The group Allow does not grant access.
To grant access to some groups only, leave the organization rule unset and add group Allow rules. See Common policies.
Organization Allow with a group Deny
The following policy is in effect:
provider azure organization:acme allow
provider azure group:engineering deny
| ユーザー | Member of Engineering | 出力 |
|---|---|---|
| Alice | いいえ | Allowed |
| Bob | はい | Denied |
The organization Allow matches every user in Acme. The group Deny matches only members of Engineering.
An LLM Deny for one group can deny every user on that LLM
The following policy is in effect:
provider azure organization:acme allow
llm azure/gpt-4o group:engineering deny
DataRobot evaluates the LLM first because azure/gpt-4o has a rule. That permission contains only the Engineering Deny.
| ユーザー | グループ | Access to GPT-4o |
|---|---|---|
| Alice | Executives | Denied (no matching Allow at the LLM) |
| Bob | Engineering | Denied (matches Deny) |
| Carol | なし | Denied (no matching Allow at the LLM) |
The provider organization Allow is not used when requesting access to GPT-4o. To deny Engineering and keep access for other users, add an LLM-level organization Allow:
provider azure organization:acme allow
llm azure/gpt-4o organization:acme allow
llm azure/gpt-4o group:engineering deny
| ユーザー | Allow at LLM | Deny at LLM | 出力 |
|---|---|---|---|
| Alice | organization:acme | — | Allowed |
| Bob | organization:acme | group:engineering | Denied |
| Carol | organization:acme | — | Allowed |
Sibling LLMs are unaffected. azure/gpt-4o-mini has no LLM-level rule, so evaluation uses the provider organization Allow.
A rule on an LLM affects only that LLM. A rule on a publisher affects LLMs under that publisher that have no LLM-level rule. A rule on a provider affects LLMs under that provider that have no publisher- or LLM-level rule.
Common policies¶
The following table lists policy patterns for common access goals. Apply the rules at the provider, publisher, or LLM level that should determine access.
| Goal | ポリシー |
|---|---|
| Allow one group only | group:engineering allow (no organization rule). Members of Engineering are allowed; everyone else is denied because no Allow matches. |
| Allow everyone except one group | organization:acme allow and group:engineering deny. The organization Allow matches every user; the group Deny matches members of Engineering. |
| Restrict one LLM and keep the rest of the provider open | Provider: organization:acme allow. LLM: organization:acme allow and group:engineering deny. The LLM-level organization Allow is required. See An LLM Deny for one group can deny every user on that LLM. |
| Disable one LLM for everyone | LLM: organization:acme deny. Sibling LLMs are unaffected. |
| Remove a policy | Delete the policy in LLM gateway management. The resource inherits from its parent. |
The following configurations do not produce the intended result:
| 設定 | 結果 |
|---|---|
| Organization Deny and group Allow | Every user is denied (Deny overrides Allow at the same level). |
| Organization Allow, group Deny, and user Allow | The user is denied (Deny overrides Allow at the same level). |
| LLM Deny for one group, expecting other users to keep access from the provider | Every user is denied on that LLM (no Allow at the level that applies). |
Badges¶
Each row in LLM gateway management displays a badge that summarizes the policy on that resource. The badge describes the policy shape; it does not by itself determine a user's outcome.
A Custom publisher permission may contain the same rules as its provider. As soon as the publisher has any rule, evaluation stops at the publisher for LLMs under it.