Braintrust

Braintrust is the AI observability platform for tracing production, running evals, and catching regressions before they reach users.

Early

Public description

Company overview

Braintrust is the AI observability platform for tracing production, running evals, and catching regressions before they reach users. Take the eval maturity assessment Agent observability and evals for the whole team. From engineering to product, in one platform. Observability See what actually happened in production. Inspect every agent trace and tool call, search across millions of logs, and track latency, cost, and quality in real time. Evals Define what good looks like before you ship. Run experiments against real datasets, compare prompts and models side-by-side, and score outputs with LLMs, code, or humans. Discovery Understand agent behavior and improve quality at scale. Apply intelligence to your traces, identify important behavior, verify the evidence, test an improvement, and measure the result in one system. Cold cache Warm cache 0.00× Faster retrieval of latest matches Competitor A 0 MS Competitor B 0 MS Brainstore 0 MS 0.00× Faster phrase counting Competitor A 0 MS Competitor B Incorrect results Brainstore 0 MS 0.00× Faster single-match retrieval Competitor A 0 MS Competitor B 0 MS Brainstore 0 MS Secure by default. Compliant from day one. SOC 2 Type II certified. GDPR and HIPAA compliant. SSO and RBAC out of the box. Visit trust center Built for teams running agents in production. From first ship to enterprise scale. Meet all the teams Kylie Czajkowski , Agent Experience Manager “ When it comes to resolving issues for our agents , Pa Website: https://www.braintrust.dev/ Source: Company site

Named financing

Ownership and financing

Stage: Early

Product

Product and technology

Trace every step of every LLM call: the prompt, the response, tool calls, retrieved context, and the final output. Score that traffic against quality criteria automatically, and get alerted when something drifts. Go from zero observability to full tracing with minimal effort. Use the gateway as your default capture layer, and the SDK for deeper context on your agent’s tool calls, planning steps, or loops. OpenAI Anthropic Gemini cURL Set the base URL to https://gateway.braintrust.dev . TypeScript import { OpenAI } from "openai"; const client = new OpenAI({ baseURL: "https://gateway.braintrust.dev", apiKey: process.env.BRAINTRUST_API_KEY, }); async function main() { const response = await client.responses.create({ model: "gpt-5-mini", input: [{ role: "user", content: "Say hello!" }], }); console.log(response.output_text); } main(); Gateway Change your base URL to the gateway and pass your Braintrust API key. No other code changes needed. You get a unified API across OpenAI, Anthropic, Google, AWS, and other providers, with automatic caching and observability on every request.

Product

Product and technology

Three ways to score. Use them together. Combine deterministic checks, LLM judges, and human review in one platform. The same scorers run offline in experiments and online in production. 95% Accuracy 62% Helpfulness Instruction following Automated scorers Deterministic checks for format, structure, accuracy. Name Helpfulness Model GPT 5 Mini Score 62% Rationale The response directly answers the order status question with clear tracking details and a friendly follow-up offer. Add to dataset Find a dataset Production failures User reported issues High latency traces Golden responses Create new dataset Datasets Versioned test cases. Build from production logs or manual curation. % Accuracy GPT 5.2 87.3% AVG 1 1 Claude 4.5 Opus 92.1% +4.8% 2 Gemini 3 Pro 85.0% -2.3% 1 Mistral Large 78.2% -9.1% 2 Experiments Compare prompts, models, strategies. Side-by-side diffs. Created Input Factuality Helpfulness Sentiment 1 Jan 25 2:34 PM Hi! What's the status of order #12345? 94% 91% 82% 2 Jan 25 2:19 PM My package arrived damaged 92% 84% 45% 3 Jan 25 2:08 PM Where is my refund? It's been 2 weeks! 78% 55% 32% 4 Jan 25 2:31 PM Can I return my order from last week? 91% 88% 75% 5 Jan 25 2:22 PM Do you

Product

Product and technology

New Analyze trace 714c44e3 1h Find runs where the agent 3h Please add the following classification 1d What are the common failure modes 2d Fresh Tool Grounding passes weak fallback answers 4d Ask questions about logs, @ for context, / for commands GPT-5.6 Sol Medium Understand what happened and what to improve When your agents handle thousands of requests each day, you can’t read every trace. Braintrust continuously applies intelligence to production data to surface important behavior, investigate the evidence, and help you decide what to improve. I'll use Braintrust's pattern-analysis workflow to identify high-confidence examples, then the evaluator workflow to build and validate a regression scorer, and finally the automations workflow to configure ongoing monitoring. I'll use the current project already selected in Loop. I'm starting with read-only SQL triage and raw trace inspection. I'll look for the specific signature “work completed in tools/state, but the root response is empty or never communicates the result,” and I'll include contrasting successful handoffs so the scorer does not merely flag every tool-heavy run. Turn ID: e900c7e4-dc4a-5f2f-99d1-dd3cf14a362c The existing

  • Braintrust

    Company · Subject