Generative Syftr search.¶

syftr is an optimizer that helps to find the best LLM blueprints for your data.¶

class datarobot.models.genai.syftr_search.SearchStudy¶

Metadata for a DataRobot syftr search study.

• Parameters:
  • search_space (Optional[Dict[str, Any]]) – Search space configuration used for the study.
  • use_case_id (str) – The ID of the use case the search study is linked to.
  • grounding_dataset_id (str) – The ID of the dataset used to build vector databases.
  • eval_dataset_id (str) – The ID of the evaluation dataset.
  • grounding_dataset_name (str) – The name of the grounding dataset.
  • eval_dataset_name (str) – The name of the evaluation dataset.
  • user_id (str) – The ID of the user.
  • user_name (str) – The name of the user who ran the study.
  • num_trials (int) – The number of search trials to sample.
  • num_concurrent_trials (int) – The number of simultaneously running trials.
  • optimization_objectives (List[Tuple[str, str]]) – Optimization objectives of the study, defined as (objective, direction) pairs.
  • playground_id (str) – The ID of the associated playground.
  • temp_playground_id (Optional[str]) – The ID of the temporary playground.
  • pareto_front (Optional[List[Dict[str, Any]]]) – Pareto frontier of the study.
  • datetime_start (str) – Study start time.
  • datetime_end (Optional[str]) – Study end time.
  • study_status (str) – Status of the study (e.g., RUNNING, COMPLETED, FAILED).
  • search_study_id (str) – The ID of the search study.
  • name (str) – Name of the search study.
  • job_id (Optional[str]) – The ID of the worker job (UUID4).
  • trials_running (Optional[int]) – Number of currently running trials.
  • trials_failed (Optional[int]) – Number of failed trials.
  • trials_success (Optional[int]) – Number of completed trials.
  • all_trials (Optional[List[Dict[str, Any]]]) – Trials history.
  • existing_blueprint_ids (Optional[List[str]]) – IDs of existing LLM blueprints for comparative evaluation.
  • eval_results (Optional[List[Any]]) – Results of the comparative evaluation.
  • error_message (Optional[str]) – Error message if the study fails.

classmethod create(use_case_id, playground_id, grounding_dataset_id, eval_dataset_id, num_trials, num_concurrent_trials, optimization_objectives, search_space, name)¶

Create a new search search study with the specified parameters.

• Parameters:
  • use_case_id (str) – The ID of the use case the search study is linked to.
  • playground_id (str) – The ID of the existing playground associated with the search.
  • grounding_dataset_id (str) – The ID of the dataset used to build vector databases.
  • eval_dataset_id (str) – The ID of the evaluation dataset.
  • num_trials (int) – The number of search trials to sample.
  • num_concurrent_trials (int) – The number of simultaneously running trials.
  • optimization_objectives (List[Tuple[ObjectiveType, DirectionType]]) – Optimization objectives of the study, defined as (objective, direction) pairs.
  • search_space (SearchSpaceDict) – Search space configuration for the search.
• Returns: search study – The created search study.
• Return type: SearchStudy

classmethod get(search_study_id)¶

Read an existing search study.

• Parameters: search_study_id (str) – ID of the search study used for creation.
• Returns: search study – The created search study database.
• Return type: SearchStudy

classmethod list(use_case, playground=None, offset=0, limit=200, search=None, sort=None)¶

List all syftr search studies associated with a specific use case available to the user.

• Parameters:
  • use_case (UseCaseLike) – The returned search studies are filtered to those associated with a specific Use Case(s) if specified or can be inferred from the context. Accepts either the entity or the ID.
  • playground (Optional[Union[Playground, str]], optional) – The returned search studies are filtered to those associated with a specific playground if it is specified. Accepts either the entity or the ID.
  • search (Optional[str]) – String for filtering search studies. Search studies that contain the string in name will be returned. If not specified, all search studies will be returned.
  • sort (Optional[str]) – Property to sort search studies by. Prefix the attribute name with a dash to sort in descending order, e.g., sort=’-creationDate’. Currently supported options are “name”.
• Returns: search studies – A list of search studies available to the user.
• Return type: list[SearchStudy]

delete()¶

Delete the search study and all its related artifacts.

• Return type: None