Memory service¶

The memory service provides persistent memory management for AI agents. It stores and retrieves conversational context across sessions, manages memory lifecycle (TTL, event counts), and uses PostgreSQL with pgvector for vector-based semantic search over memories. The service includes a bundled embedding service that runs CPU-only ONNX inference for text embedding and reranking, with the model weights baked into the container image.

Enabling the service¶

To enable the Memory Service, edit the values.yaml file:

memoryservice:
  enabled: true 

Database configuration¶

The Memory Service uses PostgreSQL with the pgvector and pg_trgm extensions. It connects to the shared DataRobot PostgreSQL instance by default. If you use an external PostgreSQL instance, ensure the pgvector and pg_trgm extensions are available—see the external PostgreSQL guides for cloud-specific setup instructions.

LLM configuration¶

Memory extraction (the mem0-compatible API) uses an LLM that each memory space selects at creation time via the llmModelName field. By default, LLM requests are routed through the DataRobot LLM Gateway, so the Generative AI service must be enabled and configured with provider credentials. Model names are validated against the LLM Gateway catalog and a built-in allowlist; see LLM availability for the models offered per provider.

A memory space can instead point at a custom OpenAI-compatible endpoint via the llmBaseUrl field. Custom endpoints are restricted to a single allowed host, configured with the allowedLlmUrlHost value (exposed to the service as the ALLOWED_LLM_URL_HOST environment variable):

memoryservice:
  allowedLlmUrlHost: "llm.example.com" 

When allowedLlmUrlHost is unset, the allowed host falls back to global.domain. If neither is set, custom llmBaseUrl values are rejected and memory spaces can only use the LLM Gateway.

Model provisioning¶

The models the memory service uses ship inside its container image, so pods start without downloading anything—no init container runs and no CDN access is required.

The image includes the following models:

Install in offline clusters¶

No additional steps are required. The models arrive inside the container image through your registry, so the memory service requires neither CDN egress nor pre-provisioned model weights in blob storage, and it no longer depends on the Generative AI service static files migration. To run different models, configure custom embedding models, which also work in offline clusters because the models come from your own blob storage.

Custom embedding models¶

If your installation must run an embedder (and/or reranker) not included in the DataRobot defaults, configure memoryservice.embeddingService.customModels. The shipped container image remains the only image—custom models are delivered from your blob storage at pod startup by a download-models init container, which shadows the baked-in models.

When customModels is set, the init container downloads STORAGE_PREFIX/MODEL_NAME/* from blob storage using the platform storage configuration (the datarobot-modeling-envvars ConfigMap or global.envFromSecret). Ambient cloud identity—IRSA, instance profiles, or workload identity—works without explicit credentials. Requests are always signed, so anonymous public buckets are not supported. Every downloaded file is verified against the SHA256 checksums you supply in the values; verification cannot be skipped. The chart automatically propagates the embedder name to the memory service API, which tags each memory with the model that embedded it—do not configure the model name anywhere else.

要件¶

• Models must be in ONNX format and loadable by the shipped ONNX runtime: the model directory needs tokenizer.json plus the ONNX file named by onnxFileName.
• The embedder must produce 768-dimensional vectors. The vector column in PostgreSQL is fixed at 768 dimensions, and the memory service refuses other dimensions at startup.
• Model names must be in ORG/MODEL form (for example, intfloat/multilingual-e5-base-fp16).
• Configure custom models during installation. The memory service filters vector search by the embedder that created each memory, so memories embedded by a different model become invisible after a switch—there is no re-embedding flow.

Configure custom models¶

To configure custom models:

1. Export the model to ONNX (e.g., with optimum-cli export onnx), producing tokenizer.json, the .onnx file, and any configuration files the model needs.

2. Generate SHA256 checksums for every file in the model directory—files not listed in the checksums, and listed files that are missing, both fail verification:

   find . -type f -exec sha256sum {} + 
   

3. Upload the model directory to blob storage under STORAGE_PREFIX/ORG/MODEL/.

4. Set the values and deploy. To only run a custom embedder, omit the reranker block (unless memoryservice.reranker.enabled is true, in which case it is required).

   Sample customModels configuration

   The following example uses the default models and their real checksums for illustration—replace the names, file lists, and hashes with those of your own models:

   memoryservice:
     embeddingService:
       customModels:
         storagePrefix: "custom-models"
         embedder:
           name: "intfloat/multilingual-e5-base-fp16"
           onnxFileName: "model_quantized.onnx"
           checksums:
             LICENSE: "002c2696d92b5c8cf956c11072baa58eaf9f6ade995c031ea635c6a1ee342ad1"
             config.json: "62f182f2784f80cb741882b08260485bfaf02ca9cee68f3d30aa2d986d17a03a"
             model_quantized.onnx: "fec06cf9ed6f901e89983d53817b04abf13cadb438b343ad07a53c398b4e2ae1"
             ort_config.json: "2433ab5020ed8bd959f2e56d96e4793b519dd8f27fe7db95dcf410fe1640e657"
             sentencepiece.bpe.model: "cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865"
             special_tokens_map.json: "8c785abebea9ae3257b61681b4e6fd8365ceafde980c21970d001e834cf10835"
             tokenizer.json: "f59925fcb90c92b894cb93e51bb9b4a6105c5c249fe54ce1c704420ac39b81af"
             tokenizer_config.json: "f90024142df07163e5e6c5b9a6ad7c8c68b22a9112af11e3db4559a9ff90f737"
         reranker:
           name: "cross-encoder/mmarco-mMiniLMv2-L12-H384-v1"
           onnxFileName: "model_quantized.onnx"
           checksums:
             LICENSE: "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4"
             config.json: "773bb79a85adaacc7c15b3fb1c5f78246a2167e81df4aa593933c8c8dc096005"
             model_quantized.onnx: "c98861b08cc3a99f1aaea95452d6671e89835fce70a42f5cca9c2cec8f6441cc"
             sentencepiece.bpe.model: "cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865"
             special_tokens_map.json: "8c785abebea9ae3257b61681b4e6fd8365ceafde980c21970d001e834cf10835"
             tokenizer.json: "3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20"
             tokenizer_config.json: "ccf223ba3d5b3cc7fa6c3bf451f3bb40557a5c92b0aa33f63d17802ff1a96fd9" 
   

Troubleshoot custom models¶

Download failures are reported in the download-models init container logs (kubectl logs POD_NAME -c download-models):

• Storage credential errors indicate missing or invalid blob storage configuration.
• No files found under 'PREFIX/NAME' indicates a wrong storagePrefix or an upload layout that doesn't match STORAGE_PREFIX/ORG/MODEL/.
• Checksum errors (a mismatch, an unexpected file, or a missing file) indicate a bad upload or a checksum map that doesn't match the uploaded files.

A model that downloads but fails to load, crashes the main container. An embedder load failure is fatal (CrashLoopBackOff), while a reranker load failure is non-fatal—reranking returns 503 but embeddings continue to work.

The downloaded model cache is ephemeral, so pods re-download the models on every start. Startup time scales with model size and blob storage throughput, and a blob storage outage prevents new embedding service pods from starting (running pods are unaffected).