{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ガバナンスされたカスタムLLMのデプロイ\n", "\n", "このノートブックをダウンロード\n", "\n", "This notebook outlines how to create a deployment endpoint that interfaces with a large language model (LLM). Doing so allows you to engage with it via a DataRobot API key. In exchange for creating a deployment around the LLM, you get all of the benefits of DataRobot MLOps governance capabilities, such as text drift monitoring, request history, and usage statistics. \n", "\n", "続行する前に、このノートブックでタスクを実行するために[必要なアセットをダウンロード](https://datarobot-doc-assets.s3.us-east-1.amazonaws.com/bolt_on_governance.zip)します。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 環境の初期化\n", "\n", "環境を起動し、DataRobotライブラリをインポートして、LLMの資格情報が機能していることを確認します。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "import datarobot as dr\n", "from openai import AzureOpenAI\n", "\n", "os.environ['PULUMI_CONFIG_PASSPHRASE'] = 'default'\n", "\n", "assert 'DATAROBOT_API_TOKEN' in os.environ, 'Please set the DATAROBOT_API_TOKEN environment variable'\n", "assert 'DATAROBOT_ENDPOINT' in os.environ, 'Please set the DATAROBOT_ENDPOINT environment variable'\n", "\n", "assert 'OPENAI_API_BASE' in os.environ, 'Please set the OPENAI_API_BASE environment variable'\n", "assert 'OPENAI_API_KEY' in os.environ, 'Please set the OPENAI_API_KEY environment variable'\n", "assert 'OPENAI_API_VERSION' in os.environ, 'Please set the OPENAI_API_VERSION environment variable'\n", "\n", "\n", "dr_client = dr.Client()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def test_azure_openai_credentials():\n", " \"\"\"Test the provided OpenAI credentials.\"\"\"\n", " model_name = os.getenv(\"OPENAI_API_DEPLOYMENT_ID\")\n", " try:\n", " client = AzureOpenAI(\n", " api_key=os.getenv(\"OPENAI_API_KEY\"),\n", " azure_endpoint=os.getenv(\"OPENAI_API_BASE\"),\n", " api_version=os.getenv(\"OPENAI_API_VERSION\"),\n", " )\n", " client.chat.completions.create(\n", " messages=[{\"role\": \"user\", \"content\": \"hello\"}],\n", " model=model_name, # type: ignore[arg-type]\n", " )\n", " except Exception as e:\n", " raise ValueError(\n", " f\"Unable to run a successful test completion against model '{model_name}' \"\n", " \"with provided Azure OpenAI credentials. Please validate your credentials.\"\n", " ) from e\n", "\n", "\n", "test_azure_openai_credentials()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## プロジェクトの設定\n", "\n", "Pulumiスタックを作成または破棄する関数を設定します。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from pulumi import automation as auto\n", "\n", "\n", "def stack_up(project_name: str, stack_name: str, program: callable) -> auto.Stack:\n", " # create (or select if one already exists) a stack that uses our inline program\n", " stack = auto.create_or_select_stack(\n", " stack_name=stack_name, project_name=project_name, program=program\n", " )\n", "\n", " stack.refresh(on_output=print)\n", "\n", " stack.up(on_output=print)\n", " return stack\n", "\n", "\n", "def destroy_project(stack: auto.Stack):\n", " \"\"\"Destroy pulumi project\"\"\"\n", " stack_name = stack.name\n", " stack.destroy(on_output=print)\n", "\n", " stack.workspace.remove_stack(stack_name)\n", " print(f\"stack {stack_name} in project removed\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 宣言型LLMデプロイの作成\n", "\n", "ガバナンスLLMモデルのデプロイは複雑ではありませんが、以下の3つの理由から、標準的な分類モデルのカスタムモデルデプロイを作成するよりも手間がかかります。\n", "\n", "1. LLMエンドポイントとその他のメタデータを指定して、デプロイに関する*ランタイムパラメーター*を設定する必要があります。\n", "2. APIキーのような非表示にすべきモデルメタデータには、*資格情報*を設定して適用する必要があります。作成した非表示の資格情報は、ランタイムパラメーターの1つになります。\n", "3. デプロイを通じてAPI呼び出しを送信するのに適した、サーバーレス予測環境と呼ばれる特別な環境を使用する必要があります。この環境を、モデル専用に1つ設定する必要があります。\n", "\n", "資格情報とランタイムパラメーターを設定したら、ソースコードをDataRobotにアップロードし、モデルを登録して、デプロイを初期化します。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\n", "import pulumi_datarobot as datarobot\n", "import pulumi\n", "\n", "\n", "def setup_runtime_parameters(\n", " credential: datarobot.ApiTokenCredential,\n", ") -> list[datarobot.CustomModelRuntimeParameterValueArgs]:\n", " \"\"\"Setup runtime parameters for bolt on goverance deployment.\n", "\n", " Each runtime parameter is a tuple trio with the key, type, and value.\n", "\n", " Args:\n", " credential (datarobot.ApiTokenCredential):\n", " The DataRobot credential representing the LLM api token\n", " \"\"\"\n", " return [\n", " datarobot.CustomModelRuntimeParameterValueArgs(\n", " key=key,\n", " type=type_,\n", " value=value, # type: ignore[arg-type]\n", " )\n", " for key, type_, value in [\n", " (\"OPENAI_API_KEY\", \"credential\", credential.id),\n", " (\"OPENAI_API_BASE\", \"string\", os.getenv(\"OPENAI_API_BASE\")),\n", " (\"OPENAI_API_VERSION\", \"string\", os.getenv(\"OPENAI_API_VERSION\")),\n", " (\n", " \"OPENAI_API_DEPLOYMENT_ID\",\n", " \"string\",\n", " os.getenv(\"OPENAI_API_DEPLOYMENT_ID\"),\n", " ),\n", " ]\n", " ]\n", "\n", "\n", "def make_bolt_on_governance_deployment():\n", " \"\"\"\n", " Deploy a trained model onto DataRobot's prediction environment.\n", "\n", " Upload source code to create a custom model version.\n", " Then create a registered model and deploy it to a prediction environment.\n", " \"\"\"\n", "\n", " # ID for Python 3.11 Moderations Environment\n", " python_environment_id = \"65f9b27eab986d30d4c64268\"\n", "\n", " custom_model_name = \"App Template Minis - OpenAI LLM\"\n", " registered_model_name = \"App Template Minis - OpenAI Registered Model\"\n", " deployment_name = \"App Template Minis - Bolt on Goverance Deployment\"\n", "\n", " prediction_environment = datarobot.PredictionEnvironment(\n", " resource_name=\"App Template Minis - Serverless Environment\",\n", " platform=dr.enums.PredictionEnvironmentPlatform.DATAROBOT_SERVERLESS,\n", " )\n", "\n", " llm_credential = datarobot.ApiTokenCredential(\n", " resource_name=\"App Template Minis - OpenAI LLM Credentials\",\n", " api_token=os.getenv(\"OPENAI_API_KEY\"),\n", " )\n", "\n", " runtime_parameters = setup_runtime_parameters(llm_credential)\n", "\n", " deployment_files = [\n", " (\"./model_package/requirements.txt\", \"requirements.txt\"),\n", " (\"./model_package/custom.py\", \"custom.py\"),\n", " (\"./model_package/model-metadata.yaml\", \"model-metadata.yaml\"),\n", " ]\n", "\n", " custom_model = datarobot.CustomModel(\n", " resource_name=custom_model_name,\n", " runtime_parameter_values=runtime_parameters,\n", " files=deployment_files,\n", " base_environment_id=python_environment_id,\n", " target_type=dr.enums.TARGET_TYPE.TEXT_GENERATION,\n", " target_name=\"content\",\n", " language=\"python\",\n", " replicas=2,\n", " )\n", "\n", " registered_model = datarobot.RegisteredModel(\n", " resource_name=registered_model_name,\n", " custom_model_version_id=custom_model.version_id,\n", " )\n", "\n", " deployment = datarobot.Deployment(\n", " resource_name=deployment_name,\n", " label=deployment_name,\n", " registered_model_version_id=registered_model.version_id,\n", " prediction_environment_id=prediction_environment.id,\n", " )\n", "\n", " pulumi.export(\"serverless_environment_id\", prediction_environment.id)\n", " pulumi.export(\"custom_model_id\", custom_model.id)\n", " pulumi.export(\"registered_model_id\", registered_model.id)\n", " pulumi.export(\"deployment_id\", deployment.id)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## まとめる\n", "\n", "では、スタックを実行します。そうすることで、`model_package`ディレクトリにあるファイルが取り出されて、DataRobotにカスタムモデルとして配置され、そのモデルの登録と結果のデプロイが行われます。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "project_name = \"AppTemplateMinis-BoltOnGovernance\"\n", "stack_name = \"MarshallsExtraSpecialLargeLanguageModel\"\n", "\n", "stack = stack_up(project_name, stack_name, program=make_bolt_on_governance_deployment)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 出力の操作\n", "\n", "ガバナンスのデプロイができたので、OpenAI SDKを使用して直接操作できます。唯一の違いは、LLMの資格情報の代わりにDataRobot APIキーを渡すことです。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from pprint import pprint\n", "from openai import OpenAI\n", "\n", "deployment_id = stack.outputs().get(\"deployment_id\").value\n", "deployment_chat_base_url = dr_client.endpoint + f\"/deployments/{deployment_id}/\"\n", "client = OpenAI(api_key=dr_client.token, base_url=deployment_chat_base_url)\n", "\n", "messages = [\n", " {\"role\": \"user\", \"content\": \"Why are ducks called ducks?\"},\n", "]\n", "response = client.chat.completions.create(messages=messages, model=\"gpt-4o\")\n", "\n", "pprint(response.choices[0].message.content)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 作業内容のクリア\n", "\n", "デプロイを維持する必要がない場合もあります。このセルを使用してスタックをシャットダウンし、DataRobotで作成されたアセットを削除します。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "destroy_project(stack)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 添付資料\n", "\n", "### スコアリングコードはどのように機能するか？\n", "\n", "以下のコードは、DataRobotがカスタムモデルとの対話方法を認識できるようにアップロードする内容を示しています。最小限の変換でカスタム推論モデルをデプロイするため、モデルと対話するためのフックは2つしか定義されていませんが、[その他のフックも](https://docs.datarobot.com/ja/docs/api/code-first-tools/drum/custom-model-components.html)追加できます。この例のモデルは標準的なScikit-learn二値分類器であるため、フックを定義しなくても、DataRobotはモデルとの対話方法を判断できます。しかし、ほとんどのモデルアーティファクトには何らかのカスタムスコアリングロジックが必要であるため、この例には`custom.py`ファイルも含まれています。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import Code\n", "\n", "Code(filename=\"./model_package/custom.py\", language=\"python\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 4 }