Covalent¶

DataRobot offers an open-source distributed computing platform, Covalent, a code-first solution that simplifies building and scaling complex AI and high-performance computing applications. You can define your compute needs (CPUs, GPUs, storage, deployment, etc.) directly within Python code and Covalent handles the rest, without dealing with the complexities of server management and cloud configurations. Covalent accelerates agentic AI application development with advanced compute orchestration and optimization.

As a DataRobot user, you can access the Covalent SDK in a Python environment (whether that be in a DataRobot notebook or your own development environment) and use your DataRobot API key to leverage all Covalent features, including fine-tuning and model serving. The Covalent SDK enables compute-intensive workloads, such as model training and testing, to run as server-managed workflows. The workload is broken down into tasks that are arranged in a workflow. The tasks and the workflow are Python functions decorated with Covalent’s electron and lattice interfaces, respectively.

All documentation for Covalent can be accessed through the Covalent documentation site.

Covalent interfaces¶

The tasks and the workflow that Covalent builds are Python functions decorated with Covalent’s electron and lattice interfaces, respectively.

Electron¶

The simplest unit of computational work in Covalent is a task, called an electron, created in the Covalent API by using the @covalent.electron decorator on a function.

In discussing object-oriented code, it’s important to distinguish between classes and objects. Here are notational conventions used in this documentation:

Electron (capital “E”)

The Covalent API class representing a computational task that can be run by a Covalent executor.

electron (lower-case “e”)

An object that is an instantiation of the Electron class.

@covalent.electron

The decorator used to:

(1) Turn a function into an electron. (2) Wrap a function in an electron. (3) Instantiate an instance of Electron containing the decorated function.

All three descriptions are equivalent.

The @covalent.electron decorator makes the function runnable in a Covalent executor. It does not change the function in any other way.

The function decorated with @covalent.electron can be any Python function; however, it should be thought of, and operate as, a single task. Best practice is to write an electron with a single, well-defined purpose; for example, performing a single transformation of some input or writing or reading a record to a file or database.

Here is a simple electron that adds two numbers:

import covalent as ct

@ct.electron
def add(x, y):
    return x + y

An electron is a building block, from which you compose a lattice.

Lattice¶

A runnable workflow in Covalent is called a lattice, created with the @covalent.lattice decorator. Similar to electrons, here are the notational conventions:

Lattice (capital “L”)

The Covalent API class representing a workflow that can be run by a Covalent dispatcher.

lattice (lower-case “l”)

An object that is an instantiation of the Lattice class.

@covalent.lattice

The decorator used to create a lattice by wrapping a function in the Lattice class. (The three synonymous descriptions given for electron hold here as well.)

The function decorated with @covalent.lattice must contain one or more electrons. The lattice is a workflow, a sequence of operations on one or more datasets instantiated in Python code.

For Covalent to work properly, the lattice must operate on data only by calling electrons. By “work properly,” we mean “dispatch all tasks to executors.” The flexibility and power of Covalent comes from the ability to assign and reassign tasks (electrons) to executors, which has two main advantages, hardware independence and parallelization.

Hardware independence¶

The task’s code is decoupled from the details of the hardware it is run on.

Parallelization¶

Independent tasks can be run in parallel on the same or different backends. Here, independent means that for any two tasks, their inputs are unaffected by each others’ execution outcomes (that is, their outputs or side effects). The Covalent dispatcher can run independent electrons in parallel. For example, in the workflow structure shown below, electron 2 and electron 3 are executed in parallel.

Example usage¶

Review the code snippet below that creates tasks out of functions and dispatches the resulting workflow.

import covalent as ct
import covalent_cloud as cc

import sklearn
import sklearn.svm
import yfinance as yf
from sklearn.model_selection import train_test_split


cc.create_env(
    name="sklearn",
    pip=["numpy==2.2.4", "pytz==2025.2", "scikit-learn==1.6.1", "yfinance==0.2.55"],
    wait=True,
)

cpu_ex = cc.CloudExecutor(
    env="sklearn",
    num_cpus=2,
    memory="8GB",
    time_limit="2 hours"
)


# One-task workflow when stacking decorators:
@ct.lattice(executor=cpu_ex)
@ct.electron
def fit_svr_model_and_evaluate(ticker, n_chunks, C=1):

    ticker_data = yf.download(ticker, start='2022-01-01', end='2023-01-01')
    data = ticker_data.Close.to_numpy()
    X = [data[i:i+n_chunks].squeeze() for i in range(len(data) - n_chunks)]
    y = data[n_chunks:].squeeze()

    # Split data into train and test sets
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.2, shuffle=False)

    # Fit SVR model
    model = sklearn.svm.SVR(C=C).fit(X_train, y_train)

    # Predict and calculate MSE
    predictions = model.predict(X_test)
    mse = sklearn.metrics.root_mean_squared_error(y_test, predictions)

    return model, mse


# Run the one-task workflow
runid = cc.dispatch(fit_svr_model_and_evaluate)('AAPL', n_chunks=6, C=10)
model, mse = cc.get_result(runid, wait=True).result.load()

print(model, mse)