Covalent¶

DataRobot offers an open-source distributed computing platform, Covalent, a code-first solution that simplifies building and scaling complex AI and high-performance computing applications. ユーザーは、コンピューティングのニーズ（CPU、GPU、ストレージ、デプロイなど）をPythonコード内で直接定義できます。残りの処理はCovalentが行うため、複雑なサーバー管理やクラウド設定に煩わされることはありません。 Covalentは、高度なコンピューティングオーケストレーションと最適化により、エージェント型AIアプリケーションの開発を加速します。

DataRobotのユーザーであれば、Python環境（DataRobotのノートブックまたは独自の開発環境）でCovalent SDKにアクセスし、DataRobot APIキーを使用して、ファインチューニングやモデルの提供など、Covalentのすべての機能を利用できます。 Covalent SDKを使用すると、モデルのトレーニングやテストといった計算負荷の高いワークロードを、サーバーで管理されるワークフローとして実行できます。 ワークロードは、ワークフローに配置されたタスクに分割されます。 タスクとワークフローは、それぞれCovalentの電子インターフェイスと格子インターフェイスで装飾されたPython関数です。

All documentation for Covalent can be accessed through the [Covalent documentation site]{0}.

Covalent interfaces¶

The tasks and the workflow that Covalent builds are Python functions decorated with Covalent’s [electron]{0} and [lattice]{1} interfaces, respectively.

Electron¶

The simplest unit of computational work in Covalent is a task, called an electron, created in the Covalent API by using the @covalent.electron decorator on a function.

In discussing object-oriented code, it’s important to distinguish between classes and objects. Here are notational conventions used in this documentation:

Electron (capital “E”)

The [Covalent API class]{0} representing a computational task that can be run by a Covalent executor.

electron (lower-case “e”)

An object that is an instantiation of the Electron class.

@covalent.electron

The decorator used to:

(1) Turn a function into an electron. (2) Wrap a function in an electron. (3) Instantiate an instance of Electron containing the decorated function.

All three descriptions are equivalent.

The @covalent.electron decorator makes the function runnable in a Covalent executor. It does not change the function in any other way.

The function decorated with @covalent.electron can be any Python function; however, it should be thought of, and operate as, a single task. Best practice is to write an electron with a single, well-defined purpose; for example, performing a single transformation of some input or writing or reading a record to a file or database.

Here is a simple electron that adds two numbers:

import covalent as ct

@ct.electron
def add(x, y):
    return x + y 

An electron is a building block, from which you compose a [lattice]{0}.

Lattice¶

A runnable workflow in Covalent is called a lattice, created with the @covalent.lattice decorator. Similar to electrons, here are the notational conventions:

Lattice (capital “L”)

The [Covalent API class]{0} representing a workflow that can be run by a Covalent dispatcher.

lattice (lower-case “l”)

An object that is an instantiation of the Lattice class.

@covalent.lattice

The decorator used to create a lattice by wrapping a function in the Lattice class. (The three synonymous descriptions given for electron hold here as well.)

The function decorated with @covalent.lattice must contain one or more electrons. The lattice is a workflow, a sequence of operations on one or more datasets instantiated in Python code.

For Covalent to work properly, the lattice must operate on data only by calling electrons. By “work properly,” we mean “dispatch all tasks to executors.” The flexibility and power of Covalent comes from the ability to assign and reassign tasks (electrons) to executors, which has two main advantages, hardware independence and parallelization.

Hardware independence¶

The task’s code is decoupled from the details of the hardware it is run on.

Parallelization¶

Independent tasks can be run in parallel on the same or different backends. Here, independent means that for any two tasks, their inputs are unaffected by each others’ execution outcomes (that is, their outputs or side effects). The Covalent dispatcher can run independent electrons in parallel. For example, in the workflow structure shown below, electron 2 and electron 3 are executed in parallel.

使用例¶

Review the code snippet below that creates tasks out of functions and dispatches the resulting workflow.

import covalent as ct
import covalent_cloud as cc

import sklearn
import sklearn.svm
import yfinance as yf
from sklearn.model_selection import train_test_split


cc.create_env(
    name="sklearn",
    pip=["numpy==2.2.4", "pytz==2025.2", "scikit-learn==1.6.1", "yfinance==0.2.55"],
    wait=True,
)

cpu_ex = cc.CloudExecutor(
    env="sklearn",
    num_cpus=2,
    memory="8GB",
    time_limit="2 hours"
)


# One-task workflow when stacking decorators:
@ct.lattice(executor=cpu_ex)
@ct.electron
def fit_svr_model_and_evaluate(ticker, n_chunks, C=1):

    ticker_data = yf.download(ticker, start='2022-01-01', end='2023-01-01')
    data = ticker_data.Close.to_numpy()
    X = [data[i:i+n_chunks].squeeze() for i in range(len(data) - n_chunks)]
    y = data[n_chunks:].squeeze()

    # Split data into train and test sets
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.2, shuffle=False)

    # Fit SVR model
    model = sklearn.svm.SVR(C=C).fit(X_train, y_train)

    # Predict and calculate MSE
    predictions = model.predict(X_test)
    mse = sklearn.metrics.root_mean_squared_error(y_test, predictions)

    return model, mse


# Run the one-task workflow
runid = cc.dispatch(fit_svr_model_and_evaluate)('AAPL', n_chunks=6, C=10)
model, mse = cc.get_result(runid, wait=True).result.load()

print(model, mse)