Monitor generative AI with custom metrics¶

Access this AI accelerator on GitHub

While it gets easier to build generative AI solutions with each passing day, it is becoming evident that it is critical to effectively monitor these solutions to measure alignment and ROI, and to provide guardrails. Monitoring generative AI solutions or LLMOps is a multi-faceted endeavor and requires more than simple out-of-the-box metrics. Each business is unique and the solutions they build require customized monitoring metrics.

This accelerator illustrates how businesses can use DataRobot to effectively and holistically monitor generative AI solutions, using metrics that segment into themes for covering the monitoring requirements of a majority of businesses.