LLM Observability Evals & Gateways brand directory

Braintrust is an end-to-end AI observability and evaluation platform that connects production trace logging with structured evaluation workflows in a single developer-centric product. It captures every LLM call, tool invocation, and agent reasoning step as hierarchical spans; scores outputs using LLM-as-a-judge, heuristic, or human annotation; manages versioned prompts; and enables teams to build regression datasets directly from production failures. Its Loop AI agent automates prompt optimization and dataset generation based on trace data, while Brainstore—a purpose-built database for AI logs—powers high-speed full-text search and querying across millions of traces. Braintrust is framework-agnostic, supports 13+ native integrations, and offers enterprise security including SOC 2 Type II, HIPAA compliance, and hybrid deployment.

Confident AI is the commercial cloud platform built atop DeepEval — the open-source LLM evaluation framework — providing an integrated workspace for LLM evaluation, observability, dataset management, prompt versioning, and AI red teaming. It enables engineering, QA, and product teams to benchmark, safeguard, and continuously improve LLM applications from prototyping through production.

LangChain offers an integrated agent engineering stack: LangSmith (commercial SaaS) for observability, evaluation, deployment, and no-code Fleet agents; LangChain (open source) for rapid LLM application development with 100+ provider integrations; LangGraph (open source) for graph-based, stateful multi-agent orchestration; and Deep Agents for long-horizon autonomous task execution. LangSmith is framework-agnostic and supports any LLM stack via Python, TypeScript, Go, and Java SDKs plus OpenTelemetry, targeting the full agent development lifecycle from prototype to production.

Langfuse is an open-source, MIT-licensed LLM engineering platform that provides end-to-end tooling for the full AI application development lifecycle: hierarchical trace-based observability, versioned prompt management with one-click deploys, multi-method evaluation (LLM-as-a-judge, human annotation, user feedback, custom pipelines), structured experiment comparison, and cost/latency/quality analytics dashboards. It is OpenTelemetry-native, integrates with 80+ frameworks and model providers, and can be deployed on Langfuse Cloud or self-hosted on Docker, Kubernetes, AWS, GCP, or Azure. Since its January 2026 acquisition by ClickHouse, Langfuse runs on a ClickHouse OLAP backend enabling millisecond-latency queries over billions of monthly observations.

Arize AX is an enterprise AI and agent engineering platform providing end-to-end LLM tracing, online/offline evaluation, prompt management, and production monitoring — complemented by Arize Phoenix, an open-source and self-hostable observability and evaluation toolkit built on OpenTelemetry/OpenInference standards.

Galileo is an AI observability and eval engineering platform that transforms offline evaluations into production guardrails for GenAI applications and multi-step AI agents. Built around its proprietary Luna-2 small language models, the platform delivers 20+ research-backed evaluation metrics at low latency and cost, an autotune system that calibrates metrics from live feedback, a real-time Protect layer that blocks policy violations before they reach users, and an Insights Engine that automatically surfaces agent failure modes and prescribes fixes. It supports the full eval engineering lifecycle—from experiment management and CI/CD integration to production monitoring and runtime protection—across SaaS, VPC, and on-premises deployments.

LiteLLM (by BerriAI) is an open-source AI Gateway and Python SDK that standardizes access to 100+ LLM providers under a single OpenAI-format API. It can be used as an embedded Python library or deployed as a standalone FastAPI proxy server with virtual keys, spend tracking, guardrails, load balancing, observability integrations, and an admin dashboard — enabling platform teams to give developers governed LLM access at scale.

Helicone is an open-source LLM observability platform and AI gateway that lets developers instrument their LLM applications with a single line of code. It captures all request and response data, provides dashboards for cost, latency, and quality metrics, and acts as a multi-provider gateway supporting 100+ models with caching, fallbacks, and rate limiting. The platform is self-hostable under the Apache 2.0 license and was used by over 16,000 organizations before being acquired by Mintlify in March 2026.

Traceloop is an LLM reliability and observability platform that turns LLM logs, traces, and evaluations into a continuous feedback loop for production AI applications. Its core is OpenLLMetry, an open-source OpenTelemetry extension that instruments LLM calls, vector DB queries, and agent actions in Python, TypeScript, Go, and Ruby. On top of this telemetry layer, the Traceloop platform provides built-in quality evaluators (faithfulness, relevance, safety, PII/toxicity detection), trainable custom evaluators, real-time drift monitoring, automated CI/CD quality gates, prompt management, and an experiment framework for model and prompt comparisons—all deployable in cloud, on-prem, or air-gapped environments.

Portkey is a full-stack LLMOps platform serving as a unified control plane for production AI. It offers an AI Gateway (routing, fallbacks, load balancing, and semantic caching across 1,600+ LLMs), real-time observability (logs, traces, cost tracking, and 40+ metrics), guardrails (PII redaction, content filtering, and prompt injection prevention), a Prompt Engineering Studio (versioning, deployment, and playground), an MCP Gateway, and enterprise governance (RBAC, SSO, audit logs, and budget controls). Deployable as managed SaaS, hybrid, or fully self-hosted with a 3-line code integration claim and 0.999% uptime SLA.

Patronus AI provides an automated LLM evaluation, monitoring, and AI agent optimization platform for enterprise engineering teams, anchored by proprietary research-backed evaluators (Lynx hallucination detector, GLIDER judge) and Percival, an intelligent agent debugger. The platform covers the full AI deployment lifecycle: adversarial test generation and benchmarking pre-deployment, continuous production logging and failure monitoring post-deployment, and agentic trace analysis for multi-step AI workflows. In 2025 the company extended its scope to simulation infrastructure, introducing RL Environments and Generative Simulators that enable AI agents to learn and improve through dynamic, feedback-driven digital practice environments—positioning Patronus as both an enterprise evaluation tool and an emerging AGI simulation research lab.