ArchestackGitHub ↗ (opens in a new tab)

#1010th of 13 in Observability

OpenLIT

OpenTelemetry-based tracing, evals and guardrails for LLM apps and coding agents

Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
Stars
2.8k
License
Apache-2.0
Last commit
Oct 2026
Last release
Sep 2026
Language
TypeScript

Overview

OpenLIT collects OpenTelemetry traces and metrics from LLM apps and agents through Python and TypeScript SDKs, and stores them in ClickHouse behind a web dashboard on port 3000. It adds cost tracking, LLM-as-a-judge evals, SDK guardrails, a versioned Prompt Hub, a Vault for API keys, and GPU monitoring. A CLI installs tracing for Claude Code, Cursor and Codex sessions.

Who it is for: Teams self-hosting tracing, cost and eval tracking for LLM apps and agents

Strengths

  • Follows OpenTelemetry GenAI conventions; accepts OTLP on :4317 (gRPC) and :4318 (HTTP)
  • One-line auto-instrumentation via openlit.init(); README claims 70+ integrations
  • Covers cost tracking, evals, guardrails, prompt versioning and secrets in one tool
  • Docker Compose quickstart; Apache-2.0 and free to self-host

Weaknesses

  • Needs ClickHouse as the telemetry store; RAM and disk requirements are not stated
  • Broad scope (Vault, Rule Engine, OpenGround) means a larger surface than a tracing-only tool
  • Guardrails run in the SDK, so they only protect instrumented code
  • No Dockerfile detected by our tools; the README only shows Compose

What it needs

  • GPU optional
  • Compose
  • Needs ClickHouse
  • Models: OpenAI, Anthropic, Ollama, vLLM, Amazon Bedrock
  • port 3000

Also in Observability

See all 13
Also in Observability
RankProjectScore
1LangfuseTracing, prompt management and evals for LLM apps on ClickHouse Live demo ↗ (opens in a new tab)35.6k stars, custom license74 out of 100
2PhoenixLLM tracing, evals, datasets and prompt playground built on OpenTelemetry11.8k stars, custom license74 out of 100
3promptfooCLI for evaluating and red-teaming prompts, agents and RAG25.9k stars, MIT71 out of 100
#4MLflowTracing, evals, prompt registry and AI gateway plus classic ML tracking Live demo ↗ (opens in a new tab)28.3k stars, Apache-2.070 out of 100
#5OpikTrace, evaluate and monitor LLM apps and agents, Apache-2.0 end to end22.5k stars, Apache-2.064 out of 100