Skip to content

On-premise users: click in-app to access the full platform documentation for your version of DataRobot.

Generative AI > Use generative AI > GenAI overview

GenAI overview

By leveraging DataRobot's AI platform for generative AI you can:

  • Safely extend LLMs with proprietary data.
  • Build and deploy generative AI solutions using your tool-of-choice.
  • Confidently manage and govern LLMs in production.
  • Unify generative and predictive AI workflows end-to-end.
  • Continuously improve GenAI applications with predictive modeling and user feedback.

DataRobot's generative AI offering builds off of DataRobot's predictive AI experience to provide confidence scores and enable you to bring your favorite libraries, choose your LLMs, and integrate third-party tools. You can embed or deploy AI wherever it will drive value for your business, and leverage built-in governance for each asset in the pipeline.

While there is a perception that "it's all about the model," in reality, the value depends more on the GenAI end-to-end strategy. Quality of the vector database (if used), prompting strategy, and monitoring, maintenance, and governance are all critical components of success.

With DataRobot GenAI capabilities, you can generate text content using a variety of pre-trained large language models (LLMs). Additionally, you can tailor the content to your data by building vector databases and leveraging them in the LLM blueprints.

For trial users

If you are a DataRobot trial user, see the FAQ for information on trial-specific capabilities. To start a DataRobot trial of predictive and generative AI, click Start a free trial at the top of this page.

See the GenAI walkthrough, which compares multiple retrieval-augmented generation (RAG) pipelines. When completed, you'll have multiple end-to-end pipelines with built-in evaluation, assessment, and logging, providing governance and guardrails.

Watch the full video here

See the list of considerations to keep in mind when working with DataRobot GenAI.

Best practices for prompt engineering

Prompt engineering refers to the process of carefully crafting the input prompts that you give to an LLM to maximize the usefulness of the output it generates. This can be a critical step in getting the most out of these models, as the way you phrase your prompt can significantly influence the response. The following are some best practices for prompt engineering:

Characteristic Explanation
Specificity Make prompts as specific as possible. Instead of asking “What’s the weather like?”, ask “What’s the current temperature in San Francisco, California?” The latter is more likely to yield the information you’re looking for.
Explicit instructions If you have a specific format or type of answer in mind, make that clear in your prompt. For example, if you want a list, ask for a list. If you want a yes or no answer, ask for that.
Contextual information If relevant, provide some context to guide the model. For instance, if you’re asking for advice on writing a scientific type of content, make sure to mention that in your prompt.
Use of examples When you want the model to generate in a particular style or format, giving an example can help guide the output. For instance, if you want a rhyming couplet, you could include an example of one in your prompt.
Prompt length While it can be useful to provide context, remember that longer prompts may lead the model to focus more on the later parts of the prompt and disregard earlier information. Be concise and to the point.
Bias and ethical considerations Be aware that the way you phrase your prompt can influence the output in terms of bias and harmful response. Ensure your prompts are as neutral and fair as possible, and be aware that the model can reflect biases present in its training data.
Temperature and Top P Settings In addition to the prompt itself, you can also adjust the ‘temperature’ and ‘Top P’ settings. Higher temperature values make output more random, while lower values make it more deterministic. Top P controls the diversity of the output by limiting the model to consider only a certain percentile of most likely next words.
Token limitations Be aware of the maximum token limitations of the model. For example, GPT-3.5 Turbo has a maximum token limit of 4096 tokens. If your prompt is too long, it could limit the length of the model’s response.

These are some of the key considerations in prompt engineering, but the exact approach depends on the specific use case, model, and kind of output you’re looking for. It’s often a process of trial and error and can require a good understanding of both your problem domain and the capabilities and limitations of the model.

Read more

Updated July 23, 2024