Add files from remote repos to custom models¶

If you add a model to the Custom Model Workshop, you can add files to that model from a wide range of repositories, including Bitbucket, GitHub, GitHub Enterprise, S3, GitLab, and GitLab Enterprise. After adding a repository to DataRobot, you can pull files from the repository and include them in the custom model.

Add a remote repository¶

The following steps show how to add a remote repository so that you can pull files into a custom model:

After you select the type of repository to register, follow the relevant process from the list below:

• Bitbucket Server
• GitHub
• GitHub Enterprise
• S3
• GitLab
• GitLab Enterprise

Bitbucket Server repository¶

To register a Bitbucket Server repository, in the Add Bitbucket Server repository modal, configure the required fields:

After you configure the required fields, click Test to verify connection to the repository. Once you verify the connection, click Add repository.

GitHub repository¶

To add a GitHub repository, in the Add GitHub repository modal, the steps for connecting to the repository depend on the connection method.

After you configure the required fields, click Test to verify connection to the repository. Once you verify the connection, click Add repository.

GitHub organization repository access¶

If you belong to a GitHub organization, you can request access to an organization's repository for use with DataRobot. A request for access notifies the GitHub admin, who then who approves or denies your access request.

GitHub Enterprise repository¶

To register a GitHub Enterprise repository, in the Add GitHub Enterprise repository modal, configure the required fields:

After you configure the required fields, click Test to verify connection to the repository. Once you verify the connection, click Add repository.

Git Large File Storage¶

Git Large File Storage (LFS) is supported by default for GitHub integrations. Reference the Git documentation to learn more. Git LFS support for GitHub always requires having the GitHub application installed on the target repository, even if it's a public repository. Any non-authorized requests to the LFS API will fail with an HTTP 403.

S3 repository¶

To register an S3 repository, in the Add S3 repository modal, configure the required fields.

After you configure the required fields, click Test to verify connection to the repository. Once you verify the connection, click Add repository.

AWS S3 access configuration¶

DataRobot requires the AWS S3 ListBucket and GetObject permissions in order to ingest data. These permissions should be applied as an additional AWS IAM Policy for the AWS user or role the cluster uses for access. For example, to allow ingestion of data from a private bucket named examplebucket, apply the following policy:

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Effect": "Allow",
          "Action": ["s3:ListBucket"],
          "Resource": ["arn:aws:s3:::examplebucket"]
        },
        {
          "Effect": "Allow",
          "Action": ["s3:GetObject"],
          "Resource": ["arn:aws:s3:::examplebucket/*"]
        }
      ]
    }

GitLab Cloud repository¶

To add a GitLab repository, in the Add GitLab repository modal, the steps for connecting to the repository depend on the connection method.

After you configure the required fields, click Test to verify connection to the repository. Once you verify the connection, click Add repository.

GitLab Enterprise repository¶

To register a GitLab Enterprise repository, in the Add GitLab Enterprise repository modal, configure the required fields:

After you configure the required fields, click Test to verify connection to the repository. Once you verify the connection, click Add repository.

Create a personal access token for GitLab Enterprise¶

To create a personal access token:

1. Navigate to GitLab.

2. Enter a name for the new token, set the mandatory scopes (read_api and read_repository), and click Create personal access token.

   

   The newly generated token appears at the top of the page.

   

3. Enter the new token into the Personal access token field in the Add GitLab Enterprise repository window.

Pull files from the repository¶

After you add a repository to DataRobot, you can pull files from the repository and include them in the custom model.

To pull files from a repository:

1. In the top navigation bar, click Model Registry.

2. Click Custom Model Workshop, click the Models tab, and select a model from the list.

3. Under Assemble Model, click Select remote repository.

   

   Select a base environment

   If the Model group box is empty, select a Base Environment for the model.

4. In the Select a remote repository dialog box, select a repository in the list and click Select content.

   

5. In the Pull from GitHub repository dialog box, select the checkbox for any files or folders you want to pull into the custom model.

   You can also click Select all to select every file in the repository, or, after selecting one or more files, click Deselect all to clear your selections.

   Repository type

   This step uses GitHub as an example; however, the process is the same for each repository type.

   

   Tip

   You can see how many files you have selected at the bottom of the dialog box (e.g., + 4 files will be added).

6. Once you select the files you want to pull into the custom model, click Pull.

   The added files appear under the Model header as part of the custom model.