ArchestackGitHub ↗ (opens in a new tab)

Open-source alternatives to LangSmith

The highest-scoring open-source projects in Observability. They are picks, not exact replacements, so read the weaknesses before you switch.

Open-source alternatives to LangSmith
RankProjectScore
1LangfuseTracing, prompt management and evals for LLM apps on ClickHouse Live demo ↗ (opens in a new tab)Observability, 35.6k stars, custom license74 out of 100
2PhoenixLLM tracing, evals, datasets and prompt playground built on OpenTelemetryObservability, 11.8k stars, custom license74 out of 100
3promptfooCLI for evaluating and red-teaming prompts, agents and RAGObservability, 25.9k stars, MIT71 out of 100
#4MLflowTracing, evals, prompt registry and AI gateway plus classic ML tracking Live demo ↗ (opens in a new tab)Observability, 28.3k stars, Apache-2.070 out of 100
#5OpikTrace, evaluate and monitor LLM apps and agents, Apache-2.0 end to endObservability, 22.5k stars, Apache-2.064 out of 100
#6LatitudeAgent observability that groups failures and dispatches coding agents to fix themObservability, 4.7k stars, MIT62 out of 100
#7Future AGITracing, evals, simulation, guardrails and an LLM gateway for AI agents Live demo ↗ (opens in a new tab)Observability, 2.1k stars, Apache-2.061 out of 100
#8LangWatchAgent observability, simulation testing, AI gateway and governance in oneObservability, 4.9k stars, Apache-2.060 out of 100
#9LaminarRust-based agent tracing with SQL queries, signals and evalsObservability, 3.4k stars, Apache-2.057 out of 100
#10OpenLITOpenTelemetry-based tracing, evals and guardrails for LLM apps and coding agentsObservability, 2.8k stars, Apache-2.057 out of 100

Reviews

174 out of 100

Langfuse

Tracing, prompt management and evals for LLM apps on ClickHouse

Live demo ↗ (opens in a new tab)35.6k stars, custom license, last commit Oct 2026

Ingests traces of LLM calls, retrieval and agent steps via Python and JS/TS SDKs or drop-in OpenAI, LangChain, LlamaIndex, LiteLLM and Vercel AI SDK integrations, then adds prompt versioning with caching, LLM-as-a-judge and code evaluators, datasets and a playground. Stores data in ClickHouse; deploys with docker compose, Helm on Kubernetes, or Terraform for AWS, Azure and GCP. For teams debugging and evaluating LLM apps.

Strengths

  • Public OpenAPI spec, Postman collection and typed Python and JS/TS SDKs
  • Prompt management with server and client caching adds no request latency
  • Deployment paths from docker compose to Helm and Terraform templates
  • Integrations with Dify, Flowise, Langflow, OpenWebUI, LobeChat, CrewAI, smolagents

Weaknesses

  • MIT except the ee folders; enterprise features need a commercial license
  • Runs on ClickHouse plus other services; heavier than single-binary tools
  • Default compose inherits Docker json-file logging with no rotation; disk can fill
  • No Dockerfile at the repo root; images come from Docker Hub
  • no GPU
  • Compose
  • Needs ClickHouse
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
274 out of 100

Phoenix

LLM tracing, evals, datasets and prompt playground built on OpenTelemetry

11.8k stars, custom license, last commit Oct 2026

Phoenix collects traces from LLM applications through OpenTelemetry/OpenInference instrumentation and shows them in a web UI. It also covers LLM-based evals, versioned datasets, experiments, prompt management and a prompt playground that can replay traced calls. It runs via pip, uvx, Docker or a Helm chart, and exposes a remote MCP endpoint at /mcp for coding agents.

Strengths

  • Install with pip or uvx and run `phoenix serve`; no separate setup shown
  • Auto-instrumentation for LangGraph, LlamaIndex, CrewAI, DSPy, Vercel AI SDK and more
  • Built-in MCP server at /mcp lets Claude Code and Cursor query traces
  • Python and TypeScript packages for OTel, client and evals

Weaknesses

  • License reported as NOASSERTION; terms need checking before commercial use
  • Managed production workflows are pushed to the paid Arize AX product
  • Azure template serves plain HTTP and needs a TLS proxy in front
  • RAM, storage backend and default port not stated in the README excerpt
  • no GPU
  • Docker + Compose
  • Compose runs PostgreSQL
  • Models: OpenAI, Anthropic, Google GenAI, AWS Bedrock, OpenRouter
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
371 out of 100

promptfoo

CLI for evaluating and red-teaming prompts, agents and RAG

25.9k stars, MIT, last commit Oct 2026

Runs prompt and model evaluations from a YAML config via promptfoo eval, compares providers side by side, and generates red-team vulnerability reports; promptfoo view opens a local web viewer. Installs with npm, Homebrew or pip, runs in CI/CD, and can scan pull requests for LLM security issues, for developers testing prompts and agents before release.

Strengths

  • Evals run locally; prompts stay on your machine
  • Red-team scans produce vulnerability reports alongside quality evals
  • Live reload and caching for fast iteration; npx usage needs no install
  • MIT licensed and still open source after joining OpenAI

Weaknesses

  • Primarily a CLI; the web viewer is a local results UI, not a multi-user server
  • Most providers require an API key; local use needs Ollama or similar
  • README is short; config syntax, assertions and providers are only in the docs
  • Dockerfile exists at the root but the README gives no Docker instructions
  • no GPU
  • Docker
  • Needs Node.js (npm) or Python (pip), LLM provider API key or Ollama
  • Models: OpenAI, Anthropic, Azure, Bedrock, Ollama
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#470 out of 100

MLflow

Tracing, evals, prompt registry and AI gateway plus classic ML tracking

Live demo ↗ (opens in a new tab)28.3k stars, Apache-2.0, last commit Oct 2026

Single mlflow server (port 5000) that records OpenTelemetry traces from 60+ frameworks via one-line autolog, runs evaluations with 50+ metrics and LLM judges, versions and optimizes prompts, and fronts providers through an OpenAI-compatible AI Gateway with rate limits, fallbacks and traffic splitting. Keeps the original experiment tracking, model registry and deployment tooling. For teams wanting one platform for GenAI and ML.

Strengths

  • One-line autolog for 60+ frameworks in Python, TypeScript and Java; MCP and OTel native
  • Starts with uvx mlflow server; no separate database needed to begin
  • AI Gateway adds credential management, guardrails and A/B traffic splitting
  • Setup wizard lets Claude Code, Codex or OpenCode add tracing to a project

Weaknesses

  • README covers the quickstart; production backend store and auth setup live in docs
  • Broad scope (ML tracking plus GenAI) means a large install and UI surface
  • No Dockerfile or compose file at the repo root
  • TypeScript and Java coverage is smaller than Python (5 TS and 2 Java frameworks listed)
  • no GPU
  • Docker + Compose
  • Models: any LLM provider via autolog or the AI Gateway
  • port 5000
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#564 out of 100

Opik

Trace, evaluate and monitor LLM apps and agents, Apache-2.0 end to end

22.5k stars, Apache-2.0, last commit Oct 2026

Logs trace trees for LLM calls, tool executions and agent steps via Python and TypeScript SDKs, OpenTelemetry or framework integrations, then runs datasets, experiments and LLM-as-a-judge metrics for hallucination, moderation and RAG quality, with online evaluation rules in production. Self-hosts with ./opik.sh (Docker Compose, UI on port 5173) or a Helm chart. For ML engineers moving agents to production.

Strengths

  • Full platform (backend, web app, evals, prompt management) under Apache-2.0
  • Designed for 40M+ traces per day; online evaluation rules on production traffic
  • PyTest integration gates LLM pipelines in CI
  • MCP server lets Claude Code, Cursor, Codex or opencode query traces and run evals

Weaknesses

  • No Dockerfile or compose file at the repo root; install goes through opik.sh
  • Multi-service stack (databases, caches, backend, frontend); not a single binary
  • Guardrails and the optimizer are separate profiles and SDKs to enable
  • README is heavy with Comet Cloud links and UTM tracking
  • no GPU
  • Compose
  • Models: any LLM via SDK, OpenTelemetry or framework integrations (Google ADK, AG2, Autogen, Flowise)
  • port 5173
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#662 out of 100

Latitude

Agent observability that groups failures and dispatches coding agents to fix them

4.7k stars, MIT, last commit Oct 2026

Captures traces, sessions and tool calls via a one-line SDK (TypeScript, Python) or OpenTelemetry, groups failing traces into tracked signals, then dispatches Claude Code or Cursor with those traces to open a fix PR and replays fixes against regression datasets. The UI is also reachable from an MCP server and CLI; self-hosts from Docker Hub images via Compose or Helm. For teams operating agents in production.

Strengths

  • Signals auto-group failing traces with status, size and trend
  • Agent Dispatch sends sample traces to Claude Code or Cursor via Linear or webhooks
  • Regression datasets replay fixes against the real failing traces
  • MIT license; Compose and Helm paths plus Railway one-click

Weaknesses

  • README quickstart targets the cloud; self-host steps are in external docs
  • Automatic fixing depends on third-party coding agents and their subscriptions
  • Claude Code session capture is a separate telemetry package
  • Storage and service requirements are not stated in the README
  • no GPU
  • Docker + Compose
  • Compose runs PostgreSQL, ClickHouse, Redis
  • Models: OpenAI, Anthropic, Bedrock, Vercel AI SDK and LangChain apps, any OpenTelemetry source
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#761 out of 100

Future AGI

Tracing, evals, simulation, guardrails and an LLM gateway for AI agents

Live demo ↗ (opens in a new tab)2.1k stars, Apache-2.0, last commit Oct 2026

Future AGI is a Django and Go platform that traces agents over OpenTelemetry, scores outputs with 50+ evaluators, simulates multi-turn conversations, and applies guardrail scanners. It also ships an OpenAI-compatible gateway with routing, caching and virtual keys, plus prompt-optimization algorithms. The default install runs one app container with Postgres and ClickHouse; a distributed Compose setup and a Helm chart cover larger deployments.

Strengths

  • OTel tracing with instrumentors for 50+ frameworks in Python, TypeScript, Java and C#
  • Gateway is OpenAI-compatible with 100+ providers, semantic caching and virtual keys
  • Standalone install is one command and needs 2 vCPUs and 4 GB for Docker
  • Apache-2.0 core; Compose, production overlay, Helm and air-gapped modes documented

Weaknesses

  • No supported path to move Standalone data to Distributed or Helm later
  • Distributed setup needs 4+ vCPUs and 12-16 GB, plus Kafka and PeerDB
  • Many components (Postgres, ClickHouse, Redis, Temporal) make it heavy to operate
  • Benchmark figures come from the README; independent verification is unknown
  • RAM ≥ 4 GB
  • no GPU
  • Docker + Compose
  • Needs PostgreSQL, ClickHouse, Redis, Temporal, Docker Compose v2.24+
  • Models: OpenAI-compatible providers (100+ via gateway)
  • port 3000
Live demo ↗ (opens in a new tab)Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#860 out of 100

LangWatch

Agent observability, simulation testing, AI gateway and governance in one

4.9k stars, Apache-2.0, last commit Oct 2026

Traces LLM and agent calls through OpenTelemetry and SDK integrations, runs simulation-based agent tests and evaluations, manages prompts, and adds an OpenAI- and Anthropic-compatible gateway with virtual keys and budgets. Also tracks coding-agent sessions (Claude Code, Codex, Copilot) with cost per pull request, and starts locally with npx @langwatch/server. For platform teams governing AI use across a company.

Strengths

  • npx @langwatch/server starts a local instance with only Node.js installed
  • Coding-agent tracking: sessions and cost per PR for Claude Code, Codex, Copilot
  • Gateway virtual keys with budgets for customers or employees
  • Governance ingests Copilot Studio, Claude and OpenAI compliance APIs, Workato, S3 audit feeds

Weaknesses

  • Open-core: modules under platform/app/ee need a commercial license in production
  • No Dockerfile or compose file at the repo root; production setup is in external docs
  • README is a feature index; architecture and storage needs are not described
  • Cloud signup is the first call to action; self-host gets one line
  • no GPU
  • Needs Node.js
  • Models: OpenAI, Anthropic, Azure OpenAI, Vertex AI, Bedrock
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#957 out of 100

Laminar

Rust-based agent tracing with SQL queries, signals and evals

3.4k stars, Apache-2.0, last commit Sep 2026

OpenTelemetry-native tracing for Vercel AI SDK, LangChain, OpenAI, Anthropic, Gemini and more with one line of SDK code, stored in ClickHouse and queried with SQL from the UI, MCP server or CLI. Signals watch every run for behaviors described in plain English and ping Slack; evals run from an SDK and CLI. docker compose up serves the UI on port 5667, for teams debugging browser and tool-using agents.

Strengths

  • Signals: describe a failure in plain English and get a Slack ping when it occurs
  • SQL over traces, spans, metrics and events, also from your coding agent via MCP
  • Rust backend with 20x trace compression and a realtime trace viewer
  • Custom Postgres schema support for shared database deployments

Weaknesses

  • Anonymous usage telemetry is on by default; LAMINAR_TELEMETRY_DISABLED=true opts out
  • Production is steered to the managed platform or the heavier docker-compose-full stack
  • AI features (chat-with-trace, SQL-with-AI) need a configured LLM provider
  • ClickHouse upgrades need manual container recreation and log-table truncation
  • no GPU
  • Compose
  • Needs ClickHouse, PostgreSQL, LLM provider (optional, for AI features)
  • Models: Gemini, OpenAI and OpenAI-compatible gateways (LiteLLM, OpenRouter, vLLM), AWS Bedrock, Azure AI Foundry
  • port 5667
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
#1057 out of 100

OpenLIT

OpenTelemetry-based tracing, evals and guardrails for LLM apps and coding agents

2.8k stars, Apache-2.0, last commit Oct 2026

OpenLIT collects OpenTelemetry traces and metrics from LLM apps and agents through Python and TypeScript SDKs, and stores them in ClickHouse behind a web dashboard on port 3000. It adds cost tracking, LLM-as-a-judge evals, SDK guardrails, a versioned Prompt Hub, a Vault for API keys, and GPU monitoring. A CLI installs tracing for Claude Code, Cursor and Codex sessions.

Strengths

  • Follows OpenTelemetry GenAI conventions; accepts OTLP on :4317 (gRPC) and :4318 (HTTP)
  • One-line auto-instrumentation via openlit.init(); README claims 70+ integrations
  • Covers cost tracking, evals, guardrails, prompt versioning and secrets in one tool
  • Docker Compose quickstart; Apache-2.0 and free to self-host

Weaknesses

  • Needs ClickHouse as the telemetry store; RAM and disk requirements are not stated
  • Broad scope (Vault, Rule Engine, OpenGround) means a larger surface than a tracing-only tool
  • Guardrails run in the SDK, so they only protect instrumented code
  • No Dockerfile detected by our tools; the README only shows Compose
  • GPU optional
  • Compose
  • Needs ClickHouse
  • Models: OpenAI, Anthropic, Ollama, vLLM, Amazon Bedrock
  • port 3000
Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)