Monitoring and observability¶
The Workload API provides built-in monitoring capabilities through OpenTelemetry (OTel) integration.
Available metrics¶
| Category | Metrics |
|---|---|
| Service health | Number of requests (succeeded / failed), latency, error rate, requests per minute. |
| Resource utilization | Number of replicas; CPU and memory consumption by container. |
| OTel metrics | OTel-compliant metrics emitted by your application. |
Accessing logs and traces¶
You can instrument your applications for OpenTelemetry-compliant tracing and leverage the open-source OpenTelemetry (OTel) libraries for seamless auto-instrumentation.
Tracer configuration example¶
from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
resource = Resource.create({"service.namespace": "my-service"})
def configure_tracer() -> TracerProvider:
trace_exporter = OTLPSpanExporter()
trace_provider = TracerProvider(resource=resource)
trace_provider.add_span_processor(BatchSpanProcessor(trace_exporter))
trace.set_tracer_provider(trace_provider)
return trace_provider
trace_provider = configure_tracer()
tracer = trace.get_tracer(__name__)
# Usage example
with tracer.start_as_current_span("Generate Text") as span:
span.set_attribute("foo", "bar")
span.add_event(name="ack", attributes={"john": "doe"})
Logger configuration example¶
import logging
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
from opentelemetry._logs import set_logger_provider
resource = Resource.create({"service.namespace": "my-service"})
def configure_logging() -> LoggerProvider:
log_exporter = OTLPLogExporter()
log_provider = LoggerProvider(resource=resource)
log_provider.add_log_record_processor(BatchLogRecordProcessor(log_exporter))
set_logger_provider(log_provider)
# Bridge Python logging to OTel so logger.info() / logger.warning() are exported via OTLP
handler = LoggingHandler(level=logging.NOTSET, logger_provider=log_provider)
logging.getLogger().setLevel(logging.NOTSET)
logging.getLogger().addHandler(handler)
return log_provider
log_provider = configure_logging()
logger = logging.getLogger(__name__)
# Usage
logger.info("Logging info.", extra={"extra": "INFO details"})
logger.warning("Logging warning.", extra={"extra": "WARNING details"})
API access to telemetry¶
You can retrieve specific traces, logs, and metrics directly via the API:
| Endpoint | Description |
|---|---|
GET /otel/workload/{workloadId}/traces |
Get traces for a workload. |
GET /otel/workload/{workloadId}/traces/{traceId} |
Get a specific trace. |
GET /otel/workload/{workloadId}/logs |
Get logs. |
GET /otel/workload/{workloadId}/metrics/summary |
Get metrics summary. |