{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Pulumiでカスタムモデルをデプロイする\n", "\n", "このノートブックをダウンロード\n", "\n", "このノートブックでは、Pulumiを使用してScikit-learn分類器をデプロイする方法の概要を説明します。続行する前に、このノートブックでタスクを実行するために[必要なアセットをダウンロード](https://datarobot-doc-assets.s3.us-east-1.amazonaws.com/deploy_custom_model.zip)します。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 環境の初期化\n", "\n", "環境を起動して、DataRobotライブラリをインポートします。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "import datarobot as dr\n", "\n", "os.environ['PULUMI_CONFIG_PASSPHRASE'] = 'default'\n", "\n", "assert 'DATAROBOT_API_TOKEN' in os.environ, 'Please set the DATAROBOT_API_TOKEN environment variable'\n", "assert 'DATAROBOT_ENDPOINT' in os.environ, 'Please set the DATAROBOT_ENDPOINT environment variable'\n", "\n", "dr.Client()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## プロジェクトの設定\n", "\n", "Pulumiスタックを作成または破棄する関数を設定します。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from pulumi import automation as auto\n", "\n", "\n", "def stack_up(project_name: str, stack_name: str, program: callable) -> auto.Stack:\n", " # create (or select if one already exists) a stack that uses our inline program\n", " stack = auto.create_or_select_stack(\n", " stack_name=stack_name, project_name=project_name, program=program\n", " )\n", "\n", " stack.refresh(on_output=print)\n", "\n", " stack.up(on_output=print)\n", " return stack\n", "\n", "\n", "def destroy_project(stack: auto.Stack):\n", " \"\"\"Destroy pulumi project\"\"\"\n", " stack_name = stack.name\n", " stack.destroy(on_output=print)\n", "\n", " stack.workspace.remove_stack(stack_name)\n", " print(f\"stack {stack_name} in project removed\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 宣言型カスタムモデルデプロイの作成\n", "\n", "このセルを使用して、カスタムモデルのデプロイを作成します。ソースコードをDataRobotに追加して、モデルを登録し、デプロイを初期化します。以下の`make_custom_deployment`関数は、これを宣言的に行う方法を示しています。すべてのアプリケーションテンプレートで、この関数のバリエーションを確認できます。\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import pulumi_datarobot as datarobot\n", "import pulumi\n", "\n", "\n", "def make_custom_inference_deployment():\n", " \"\"\"\n", " Deploy a trained model onto DataRobot's prediction environment.\n", "\n", " Upload source code to create a custom model version.\n", " Then create a registered model and deploy it to a prediction environment.\n", " \"\"\"\n", "\n", " # ID for Python 3.9 Scikit learn drop in environment\n", " base_environment_id = \"5e8c889607389fe0f466c72d\"\n", "\n", " # ID for the default prediction server\n", " default_prediction_server_id = \"5dd7fa2274a35f003102f60d\"\n", "\n", " custom_model_name = \"App Template Minis - Readmitted Custom Model\"\n", " registered_model_name = \"App Template Minis - Readmitted Registered Model\"\n", " deployment_name = \"App Template Minis - Readmitted Deployed Model\"\n", "\n", " deployment_files = [\n", " (\"./model_package/requirements.txt\", \"requirements.txt\"),\n", " (\"./model_package/custom.py\", \"custom.py\"),\n", " (\"./model_package/model.pkl\", \"model.pkl\"),\n", " ]\n", "\n", " custom_model = datarobot.CustomModel(\n", " resource_name=custom_model_name,\n", " files=deployment_files,\n", " base_environment_id=base_environment_id,\n", " language=\"python\",\n", " target_type=\"Binary\",\n", " target_name=\"readmitted\",\n", " )\n", "\n", " registered_model = datarobot.RegisteredModel(\n", " resource_name=registered_model_name,\n", " custom_model_version_id=custom_model.version_id,\n", " )\n", "\n", " deployment = datarobot.Deployment(\n", " resource_name=deployment_name,\n", " label=deployment_name,\n", " registered_model_version_id=registered_model.version_id,\n", " prediction_environment_id=default_prediction_server_id,\n", " )\n", "\n", " pulumi.export(\"custom_model_id\", custom_model.id)\n", " pulumi.export(\"registered_model_id\", registered_model.id)\n", " pulumi.export(\"deployment_id\", deployment.id)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Pulumiスタックの実行\n", "\n", "これで、Pulumiスタックを実行できます。そうすることで、ダウンロードしたアセットから`model_package`ディレクトリ内のファイルが取り出されて、DataRobotにカスタムモデルとして配置され、そのモデルの登録と結果のデプロイが行われます。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "project_name = \"AppTemplateMinis-CustomInferenceModels\"\n", "stack_name = \"MarshallsCustomReadmissionsPredictor\"\n", "\n", "stack = stack_up(project_name, stack_name, program=make_custom_inference_deployment)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 出力の操作" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from datarobot_predict.deployment import predict\n", "import pandas as pd\n", "\n", "\n", "df = pd.read_csv(\n", " \"https://s3.amazonaws.com/datarobot_public_datasets/10k_diabetes.csv\"\n", ").tail(100)\n", "\n", "\n", "deployment_id = stack.outputs().get(\"deployment_id\").value\n", "deployment = dr.Deployment.get(deployment_id)\n", "\n", "predict(deployment, data_frame=df).dataframe.head(10).iloc[:, :2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 作業内容のクリア\n", "\n", "カスタムモデルを維持する必要がない場合もあります。このセルを使用してスタックをシャットダウンし、DataRobotで作成されたアセットを削除します。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "destroy_project(stack)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 添付資料\n", "\n", "### スコアリングコードはどのように機能するか？\n", "\n", "以下のコードは、DataRobotがカスタムモデルとの対話方法を認識できるようにアップロードする内容を示しています。最小限の変換でカスタム推論モデルをデプロイするため、モデルと対話するためのフックは2つしか定義されていませんが、[その他のフックも](https://docs.datarobot.com/ja/docs/api/code-first-tools/drum/custom-model-components.html)追加できます。この例のモデルは標準的なScikit-learn二値分類器であるため、フックを定義しなくても、DataRobotはモデルとの対話方法を判断できます。しかし、ほとんどのモデルアーティファクトには何らかのカスタムスコアリングロジックが必要であるため、この例には`custom.py`ファイルも含まれています。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import Code\n", "\n", "Code(filename=\"./model_package/custom.py\", language=\"python\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 何をデプロイしたか？\n", "\n", "そもそもどうやって適合モデルを取得したのか気になる場合は、`fit_custom_model.py`でデータセットとモデル適合のコードを確認できます。この例では、10K diabetesデータセットを使ってRandom Forest二値分類器をトレーニングします。以下のコードは、モデルのトレーニングとpickleに使用されます。テンプレートの実行には重要ではありません。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from IPython.display import Code\n", "\n", "Code(filename=\"./fit_custom_model.py\", language=\"python\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 4 }