LangWatch is a AI model training quality platform for AI Agent and LLM applications, covering agent simulations, offline/real-time evaluation, OpenTelemetry, native observable Prompt management, DSPy optimization, user feedback, and self-hosted/hybrid deployment.

LangWatch tool text

Core parameters and statistics

LangWatch's product page positions it as an AI agent testing, LLM evaluation and LLM observability platform. Its core mission is to allow the team to see behavioral evidence of Agent, Prompt, model and tool calls during the development and production stages. It is not a basic model supplier, nor is it an ordinary chatbot, but puts testing, tracking, evaluation, feedback and optimization in the same quality concept.

Projects	Public Information
Product Name	LangWatch
Official entrance	langwatch.ai
Document entrance	docs.langwatch.ai
Application entrance	app.langwatch.ai
GitHub repository	langwatch/langwatch
Official Positioning	AI Agent Test LLM Evaluation LLM Observable Platform
Open Source License	Apache-2.0
Main language	TypeScript; maintains both Python SDK and TypeScript SDK
Community size	GitHub about 3.3k stars, about 323 forks, the warehouse was created on 2023-09-09
Deployment form	Cloud, self-hosted, hybrid deployment, enterprise on-prem
Core capabilities	Agent simulations, offline/real-time evaluations, observability, prompt management, analytics, human feedback, DSPy optimization
Data & Compliance	Cloud data region focused on EU; pricing page lists GDPR, ISO27001 Reports, enterprise security materials, and compliance review support

Positioning Boundary: The value of LangWatch lies in "quality and operational evidence", not in replacing the business application itself. It is suitable for connecting existing LLM application RAG, customer service agent, tool call chain and prompt word version, turning "whether the change is good" from a subjective feeling into a traceable experiment, evaluation and production monitoring result.

User and market recognition

Open source community signal: The official GitHub repository is described as "The platform for LLM evaluations and AI agent testing", the license is Apache-2.0, and the topic tags cover evaluation, observability, llm-ops, prompt-engineering, DSPy, datasets and other directions. About 3.3k stars and the continued release of SDK/Skills version notes, LangWatch already has a visible developer adoption base, rather than just staying on a closed-source demo site.

Enterprise and team signals: The official website and pricing page clearly sell the product to "teams that are releasing complex AI Agents", and display quotes from customers such as Adesso on the page, emphasizing safe, traceable, and optimizeable GenAI delivery. The pricing page also provides Enterprise/Regulated tiers, including custom SSO/RBAC, audit logs, SLA, ISO27001 reports, DPA, cloud market billing and other capabilities, indicating that its business focus is not limited to trial use by individual developers.

Ecological position: The difference between LangWatch and traditional APM is that it is oriented to LLM and Agent semantics: not only looking at whether the request is successful, but also looking at answer quality, hallucinations, user feedback, scenario simulation Prompt changes and model behavior. Compared with lightweight tools that only do log viewing, it emphasizes the continuous quality engineering of "simulation before going online + real-time evaluation after going online".

Cost advantage

LangWatch's pricing structure revolves around seats and events. The event here refers to the event ingested during the Cloud billing cycle. The pricing page explains that the billable event can be a span in the trace or a scenario run used to test the AI ​​Agent. This caliber is suitable for integrating Agent debugging, production tracking and test running into the same usage model.

Cost Hierarchy	Current Exposed Structure	Cost Implications
C-side/individual	Developer plan is free, no credit card required; includes 50,000 events/month, 14 days of data access to 2 users, 3 scenarios, 3 simulations and 3 custom evaluations	Individual developers can verify tracing, evaluation and simulation first without entering the enterprise procurement process first
Developer/API	Growth plan starts from €59/month, billed on a core-seat/month basis; includes 200,000 events, excess €0,00005/event; includes 30 days of data retention, additional retention of €3/GB	Suitable for small teams to bring testing and production monitoring to the same platform, the cost increases with seats and events
Enterprise / Privatization	Enterprise / Regulated is a customized price, supporting hybrid, self-hosted, on-prem, custom retention SSO/RBAC, audit log SLA and security materials	Explicit costs require business confirmation, and implicit costs mainly come from deployment, compliance review, upgrades and internal platform operation and maintenance

Cost structure judgment: For early-stage teams, the Developer free tier can cover the smallest pilot; for teams that already have production traffic, the real cost depends on the trace granularity scenario run frequency, retention period, and the number of core collaboration seats. For data-sensitive industries, self-hosting or hybrid deployment can reduce data outflow concerns, but will shift part of the cost to infrastructure and operation and maintenance processes.

Main functions

• AI Agent Simulations: Use simulated users and scenarios to test complex Agent behaviors, helping the team discover multiple rounds of dialogue, tool calls, target offsets, and boundary condition issues before going online. The mechanism is to extend a single Prompt test into a complete scenario run; the effect is to reduce missed detections caused by manual point testing alone.
• Offline and real-time evaluations: Prompts, models and tool strategies can be compared during the development phase, and real input and output can be continuously evaluated during the production phase. Suitable for quality indicators such as illusion, relevance, safety, task success rate, user satisfaction, etc.
• OpenTelemetry native observable: The product page emphasizes OpenTelemetry native integration, and the documentation page also uses tracing as the main line of integration. Mechanically, the team can use a more standard trace/span data structure to access the LLM call chain; the effect is to reduce the risk of lock-in in subsequent migrations and multi-system observations.
• Prompt Management and Experiment Tracking: The official website emphasizes that prompt, model or agent changes can be tracked and clear audit trails can be retained. Suitable for multi-person collaboration to control prompt version, grayscale release and regression risk.
• Human-in-the-loop and user feedback: The official product visual displays the Inbox, Annotations, Notes, Alignment and Labels fields, indicating that LangWatch supports the inclusion of human annotations, expert review and quality labels in the evaluation process.
• DSPy Optimization: Both the pricing page and the official website regard DSPy optimization as an important capability. It is suitable for teams that want to systematically optimize prompts, models and pipelines, rather than just manually changing prompt words.

Key points for acceptance: When evaluating LangWatch, you should not just look at whether the dashboard is beautiful. You should also verify whether the trace is complete, whether the evaluation indicators fit the business goals, whether the scenario run covers the critical failure path, and whether business experts can participate in feedback and review.

Model and version evolution

LangWatch is a continuously iterative cloud and open source platform. There is no single "product major version number" as the only anchor for all capabilities. The public version context is more suitable to be understood by warehouse SDK and platform milestones.

Open source warehouse and platform mainline

• 2023-09-09 / Open source warehouse creation: GitHub API shows that the langwatch/langwatch warehouse was created on 2023-09-09, marking the beginning of the main line of the LangWatch open source platform.
• 2026-06 / Current positioning of the documentation platform: The official document title describes LangWatch as a complete LLMOps platform, covering observability, evaluations and agent simulations, and emphasizes the open source platform and 3k+ GitHub stars.
• 2026-06-19 / High-frequency releases of SDK and Skills: GitHub Releases shows that TypeScript SDK v0.33.2, Skills v0.6.1 and other versions were released on 2026-06-19, indicating that the project is still in an active maintenance rhythm.

SDK and workflow version

Version Node	Release Date	Description
TypeScript SDK v0.33.2	2026-06-19	The latest public TypeScript SDK release, serving Node/TypeScript project access
LangWatch Skills v0.6.1	2026-06-19	Skills workflow for AI coding assistant, using natural language to quickly create evals, scenario tests and tracing
Python SDK v0.26.0	2026-06-12	Access version for Python LLM/Agent applications
TypeScript SDK v0.33.0	2026-06-11	TypeScript SDK 0.33 mainline node

Version meaning: This evolution line shows that LangWatch’s focus is not on a single major version release, but on continuous expansion around the access layer, scenario testing and evaluation workflow. For team implementation, SDK update frequency and document coverage can better reflect the maintenance status than traditional version numbers.

Technical advantages

OpenTelemetry standard route: If LLM observables only use the vendor's private event format, short-term access will be fast, but long-term migration and data connection costs will increase. LangWatch emphasizes OpenTelemetry native, and mechanically maps Agent and LLM calls to trace/span structures; the effect is that it is easier to coexist with existing back-end observations, logs, alarms, and data pipelines; the applicable scenario is teams with existing microservices or platform engineering foundations.

The same set of quality controls before and after going online: Traditional testing often stops at offline sample sets, and production monitoring only looks at error rates and delays. LangWatch puts offline evaluation, real-time monitoring, scenario simulation and human feedback on the same platform; in terms of mechanism, evaluation standards can be reused between development and production; the effect is to reduce the gap between "tests are passed but real users fail".

Agent-oriented scenario simulation: Failures of complex agents often come from multiple rounds of context, tool order, user intention drift, and boundary conditions. LangWatch's Agent Simulations let teams test "complete behaviors" rather than individual outputs; applicable scenarios include customer service agents, sales/operations agents, RAG assistants, and multi-tool automated processes.

Developers collaborate with domain experts: Annotation, labels, alignment and notes in the product interface explain that LangWatch not only serves engineering logs, but also structures the review results of business experts. Mechanically, expert feedback can be converted into evaluation data and quality labels; the effect is that "whether the answer is acceptable" is no longer just judged by engineers' feelings.

How to use

The minimum path to LangWatch usage can start with a Cloud or on-premises open source deployment. The official website pricing page states that the Developer plan can be started for free, and the documentation banner also indicates that evals, scenario tests and tracing can be quickly created through LangWatch Skills.

1. Create a workspace in app.langwatch.ai, or deploy it locally/in your own environment according to self-hosting documentation.
2. Create an API Key for the project and select Python SDK, TypeScript SDK or OpenTelemetry to access according to the application stack.
3. Record traces, spans, inputs, outputs, tool calls, and key metadata in the LLM/Agent call chain.
4. Establish evaluations for core tasks, such as hallucination, relevance, format compliance, task success rate, brand tone, or human review tags.
5. Use scenario simulations to cover key business paths, and then feed real traces from production into data sets and regression tests.
6. Combine Prompt Management and DSPy Optimization to compare different prompts, models and parameter settings to gradually form a reusable release threshold.

Access Boundary: If the team doesn't already have clear business success metrics, LangWatch can only record more data and cannot automatically define "what is a good answer". Before going online, the task objectives, failure types, manual review standards and data retention policies should be sorted out clearly.

Product Pricing

The LangWatch pricing page provides two paths: Cloud and Self-Managed. Cloud is suitable for quick pilots and managed usage, Self-Managed/Enterprise is suitable for data governance, compliance and high-volume scenarios.

Plan	Public price/caliber	Main inclusions
Developer	Free	50,000 events/month 14 days data access 2 users, 3 scenarios, 3 simulations, 3 custom evaluations, GitHub/Discord Community Support
Growth	From €59/month on a core-seat/month basis; with 200,000 events, overage €0,00005/event	Developer 30 days data retention for all capabilities, additional retention €3/GB, unlimited lite-users, unlimited eval scores/simulations/prompts, private Slack/Teams support
Enterprise / Regulated	Custom, business confirmation required	Hybrid/self-hosted/on-prem, custom retained SSO/RBAC, audit logs, SLA, ISO27001 reports, DPA, Forward Deployed Engineer, cloud market bill

The pricing page also lists the Cloud data region as EU, and Enterprise can cover regional needs such as EU/US/CA/APAC; payment methods extend from credit card to invoice and AWS, Google, and Azure Marketplace. The final price, region, contract terms and data retention should still be subject to the official real-time page and business contract.

Application scenarios

• Customer Service and Support Agent Quality Control: Combining real customer service conversations, simulated users, and manual annotations to continuously evaluate answer accuracy, brand tone, upgrade to artificial conditions, and hallucination risk. The focus of acceptance is bad case recall, manual review consistency and regression test coverage.
• RAG and Knowledge Assistant Monitoring: records search context, answers, user feedback and review scores to detect missing searches, citation errors and answers to irrelevant questions. The acceptance focus is on retrieval relevance, answer traceability, and low-scoring sample reflow speed.
• Prompt and model change release: Run offline evaluations and scenario simulations before switching prompts, models, or parameters, and then use production monitoring to observe the real impact. The focus of acceptance is version comparison, whether failed use cases are reproducible, and quality fluctuations after grayscale release.
• AI Audit for Regulated Enterprises: Satisfy internal reviews with self-hosted/hybrid deployments, audit logs RBAC, DPA, and security materials. Acceptance focuses on data boundaries, permissions models, log retention, and compliance team readability.
• AI product team feedback loop: Let product managers, domain experts and engineers collaborate in the same set of annotations, labels, and evaluations to turn user feedback into the next round of data sets and evaluation rules.

Applicable people

• AI Application Engineer: Need to track LLM call chain, debug Agent tool calls, compare Prompt/model changes, and integrate tests into CI/CD or release process.
• Product managers and domain experts: need to participate in reviewing model output, defining quality standards, marking bad examples and tracking production feedback, rather than relying solely on engineering logs.
• Platform Engineering/LLMOps Team: Need to provide unified tracing, evaluation, prompt management, permissions, data retention and deployment policies for multiple lines of business.
• Compliance & Security Team: Focus on data areas, self-hosted, hybrid deployment ISO27001 materials, audit log DPA and vendor review process.

The boundaries of incompatibility must also be clear: teams with only a few prototype calls, no stable user paths, no evaluation samples, and no artificial quality standards may not need the complete LangWatch platform in the short term; teams that only want basic token cost statistics may also use lighter gateways or logging tools.

Summary and Outlook

LangWatch's core competency lies in combining AI Agent testing, LLM evaluation, production observable Prompt management, user feedback and DSPy optimization into a quality engineering platform. It is especially suitable for teams that have moved from "being able to call models" to "stably delivering AI Agents": at this time the question is no longer whether a single answer looks good, but whether each change is verifiable, whether each failure can be reviewed, and whether each type of user feedback can be accumulated as the next round of testing assets.

There are currently three main categories of restrictions. First, some business terms still need to be subject to the official live page and contract, especially Enterprise's regional SLA, retention period and secure material access. Second, the effectiveness of LangWatch relies on the team's own definition of evaluation indicators and failure classification, and the platform cannot replace the judgment of business experts. Third, although self-hosting and hybrid deployments improve data control, they require additional costs for upgrades, backups, permissions, monitoring, and internal support.

Implementation recommendations focus on small-scale pilots: first select an Agent or RAG scenario with high frequency, low risk, and clearly defined failures, access tracing, establish 20-50 key scenarios, define 3-5 evaluation indicators, and use real user feedback to reflow the data set; when the low-scoring sample location time, regression discovery rate, and manual review efficiency can be quantified, then expand to more business lines and enterprise authority management.

Related tools: Hugging Face, replicate

Version Info

• TypeScript SDK v0.33.2 ：The latest public version of the TypeScript SDK in LangWatch's official GitHub Releases, for Node/TypeScript applications to access LangWatch tracing, evaluation and platform workflows. （2026-06-19）
• LangWatch Skills v0.6.1 ：Official GitHub Releases version of the Skills component for quickly building evals, scenario tests, and tracing workflows through coding assistants. （2026-06-19）
• Python SDK v0.26.0 ：The official Python SDK version released by GitHub Releases serves the Python LLM/Agent application access observation and evaluation process. （2026-06-12）
• TypeScript SDK v0.33.0 ：TypeScript SDK 0.33 mainline version node reflects LangWatch’s continuous maintenance of the JavaScript/TypeScript Agent engineering stack. （2026-06-11）