Skip to content
バージョン11.0を実行しているセルフマネージドAIプラットフォームのユーザーは、オンプレミスプラットフォームのドキュメントを参照してください
NextGen UIに関するドキュメント > AIエクスペリメント > モデルインサイトでの評価 > ハイパーパラメーターのチューニング

ハイパーパラメーターのチューニング

タブ 説明
詳細 Allows manually setting of model hyperparameters, overriding the DataRobot selections for a model. In some cases, by experimenting with hyperparameter settings, you can improve model performance.

Hyperparameter Tuning allows you to manually set model hyperparameters, overriding the DataRobot selections for a model, and create a new Leaderboard model using the new settings. In some cases, experimenting with hyperparameter settings can improve model performance. When you create models via this insight, the newly created Leaderboard models can later be blended together or further tuned.

When you provide new exploratory values, save, and build using new hyperparameter values, DataRobot creates a new child model using the best of each parameter value and adds it to the Leaderboard.

You can further tune that model and create a child of the child. In other words, you do not iterate and continue to tune a single child model, instead you modify the child and create another new model, providing a lineage of changes. You can also create a new child from the original parent.

Notes on hyperparameter tuning usage
  • チューニングされたモデル用にスコアを計算するには、トレーニングデータセット内の内部「グリッド検索」パーティションを使用します。 一般的なパーティションは80/20のトレーニング/検定分布ですが、DataRobotは5分間の交差検定を適用します。
  • Grid search is only available preprocessing parameters; tuning is available for all parameters.
  • You cannot tune blended models.
  • See also architecture, augmentation, and tuning options specific to Visual Artificial Intelligence (AI) image augmentation projects.

要素 説明
1 検索 Filters the display to include only those hyperparameters matching the search strings. If a task has no matching hyperparameters, it does not display.
2 タスク名 Provides the pre- or post-processing steps (tasks) that make up the model, with a link to the DataRobot model documentation, which describes the task in detail. Under each task are listed:
  • Parameter: The individual hyperparameters, which are tunable, that make up that task.
  • Current value: The single best value of all searched values for preprocessing or final prediction model hyperparameters. This is the value that was used to build the model. On the editing page, this value is displayed in the Parent column, because tuning results in a new model.
  • Searched values: All values DataRobot searched before selecting the best value.
3 パラメーターのチューニング Opens the editing window, allowing you to modify hyperparameter values.

Create a tuned model

Customize hyperparameter settings, testing individual values or ranges, to create new Leaderboard models. You can set new values for preprocessing tasks, with availability dependent on the model and project type. To create a new model—a version (child) of the parent but with hyperparameter modifications—do the following:

  1. Click Tune parameters to open the hyperparameter tuning editor.

  2. Select the hyperparameter to tune, either by scrolling though the list or filtering by search string. Click on the name.

  3. Edit the value; save changes for each modification.

  4. Select a search method

  5. Click Train new model. Once clicked, DataRobot begins training one or more new models. Expand the right panel to follow progress.

  6. On the Leaderboard, find the new child by searching for the Tuned badge and comparing the model IDs. The model card also includes a summary of changes.

  7. Open the new model, return to Details > Hyperparameter tuning and evaluate the new child compared model.

Edit hyperparameter values

To edit a hyperparameter value, first click Tune parameters and then click a selection to open. You can modify up to 12 hyperparameters per model.

Avoid memory errors

次の状況ではメモリーエラーが発生することがあります。

  • Setting any hyperparameter that accepts multiple values such that it results in a grid search that exceeds 25 grid points. Be aware that the search is multiplicative, not additive (for example, max_depth=1,2,3,4,5 with learning_rate=.1,.2,.3,.4,.5 results in 25 grid points, not 10).

  • 検索対象のハイパーパラメーターの範囲を拡大して、モデルサイズが大きくなる場合（XGBoostモデルの推定器の数またはツリー深度を増やした場合など）。

There are a variety of accepted value types and ranges; each hyperparameter includes instruction and reports the parent value.

Many values provide the option to select User-defined. When selected, the entry expands to include a field for entering the desired value. Validation checks the entry, indicating the issue and turning from red to white when the entry passes.

Click Save changes after each modification; hyperparameters are marked with an Edited badge and the entry is added to the edited parameters list in the right panel. Before training a new tuned model, you must modify at least one parameter.

Select a search method

Set the search type to configure the level of search detail, which in turn affects resource usage.

検索タイプ 説明
ベイズ探索 Provides a balanced search by using Bayesian optimization to intelligently balance exploration with time spent tuning. That is, it uses previous results to direct next search actions. When selecting Bayesian, also set the max iterations, max time (in hours), and random seed.
スマート検索 Performs a sophisticated pattern search (optimization) that emphasizes areas where the model is likely to do well and, for faster results, skips hyperparameter points that are less relevant to the model. DataRobot explores parameter values by starting with initial grid points and then iteratively finding the best-performing points. Results are returned when all promising regions are explored or maximum iterations are reached.
ブルートフォース Evaluates each data point and all possible parameter value combinations for maximized accuracy, likely increasing time and resource use.
Deep dive: Smart search heuristics

以下は、DataRobotがスマート検索を実行する方法について説明します。

  1. 指定した場合、開始値を使用します。 それ以外の場合は、最初のグリッドパスが、各パラメーターのグリッドポイントとスコアのうち25番目と75番目のパーセンタイルの外積であるように初期化します。
  2. 検索されたグリッドポイントの中から、パフォーマンスが最高のグリッドポイントを見つけます。
  3. 検索されていないグリッドポイントから、最もパフォーマンスの高い検索グリッドポイントの近傍のグリッドポイントを見つけ、それらのポイントをスコアリングします。
  4. これらの値を「検索済み」リストに追加し、手順2から繰り返します。
  5. 近傍が見つからない場合は、隣接するすべての近傍（edges）を検索します。 近傍が検出された場合は、手順2から繰り返します。
  6. それでも近傍が見つからない場合は、近傍の検索半径を小さくします。 近傍が検出された場合は、手順2から繰り返します。
  7. 検索する新しい近傍が存在しないか、最大反復数（max_iter）に達するまで、手順2～5を繰り返します。

Once a method is selected, optionally enter a tune description. The panel also provides a summary of all hyperparameters modified, reporting an error if too many have been tuned.

Evaluate child models

Once a child model is trained, it is listed on the Leaderboard with the Tuned badge and a summary of modifications. Open the model and navigate to the same insight—Details > Hyperparameter Tuning—to see changes. From there, you can:

Understanding iterations
モデルタイプ 説明 Generation
Parent The original model built in the experiment. First
Child A model in which hyperparemters of the parent were modified. Second
Child of the child A model in which hyperparemters of a child were modified. Third and later
要素 説明
1 Parameters table/Grid search Compare and evaluate changes—use Parameters table for list view or Grid search for a graphical format.
2 パラメーターを検索 When in table view, use Show only tuned parameters to see only those you have changed. If you choose to re-tune the model, all parameters are displayed.
3 パラメーターのチューニング Re-tune the model based on the settings of this child model.
3 親モデルを開く Click to change the view to the parent model. If you are making modifications to a child model, the listed parent model is the original parent, not the child that the "child of the child" was trained from.

パラメーターテーブル

The table view lists all tunable hyperparameters of the model, in the same breakdown as the when tuning was available for the parent model. In this view, however, in addition to the current and searched values, DataRobot displays the value used by the original parent.

From here, you can create an iteration of the child by clicking Tune parameters or return to the parent (Open parent model) to create a "second generation" model

The grid search visualization, available on models for which DataRobot ran a grid search, plots parameters against score for child models. It visualizes patterns across numeric parameter-to-score relationships, helping to identify which parameter combinations lead to the best model performance. モデルタイプによりグラフの数値と詳細は異なります。 Not all model types or hyperparameters generate grid search graphs since not all use a grid search to find the best value.

DataRobotは、数値または小数値をとるこれらのパラメーターをグラフ化し、スコアに対して表示します。 In the example above, the top three graphs each plot one of the parameters used to build the current model. グラフにある点は、DataRobotがそのパラメーターに対して試行した値を示しています。 The starred dot is the value selected, and dots are represented in a worst-to-best color scheme.

The additional rows of graphs provide parameter-to-parameter information that illustrates an analysis of co-occurrence—plotting the parameter values against each other. The graphs can be helpful in experimenting with parameter selection because they provide a visual indicator of the values that DataRobot tried. たとえば、全く異なる試みとして、グラフの中の何もない領域を探し、その領域内のある部分に一致するパラメーター値を設定することができます。 Or, if you want to try tweaking something that you know did well, you can identify values in the region near to the star that represents the best value.