ArchestackGitHub ↗ (opens in a new tab)

22nd of 14 in Gateways

LiteLLM

OpenAI-format gateway and Python SDK for calling 100+ LLM providers

Documentation ↗ (opens in a new tab)Website ↗ (opens in a new tab)Repository on GitHub ↗ (opens in a new tab)
Stars
60.9k
License
custom license
Last commit
Oct 2026
Last release
Oct 2026
Language
Python

Overview

LiteLLM translates calls to 100+ providers (OpenAI, Anthropic, Gemini, Bedrock, Azure and others) into the OpenAI format, either as a Python SDK or as a proxy server. The proxy adds virtual keys, spend tracking, guardrails, load balancing and an admin dashboard, and it also exposes A2A agent and MCP server gateways. The README reports 8ms P95 latency at 1k RPS.

Who it is for: Teams routing many LLM providers through one self-hosted API

Strengths

  • One OpenAI-style API across 100+ providers and many endpoint types
  • Proxy includes virtual keys, spend tracking, load balancing and admin dashboard
  • Also gateways A2A agents and MCP servers
  • Use as a Python library or as a standalone proxy

Weaknesses

  • License reported as NOASSERTION; an enterprise tier exists, feature split unclear from README
  • Proxy listens on port 4000 and its database requirements are not stated in the README excerpt
  • Provider coverage varies by endpoint; many providers support only chat-style endpoints
  • Python-based, so latency figures depend on the benchmark setup

What it needs

  • no GPU
  • Docker + Compose
  • Compose runs PostgreSQL
  • Models: OpenAI, Anthropic, Gemini, AWS Bedrock, Azure
  • port 4000

Also in Gateways

See all 14
Also in Gateways
RankProjectScore
1OmniRouteOpenAI-compatible gateway that routes requests across hundreds of AI providers74.9k stars, MIT86 out of 100
3freellmapiOpenAI-compatible router that fails over across free LLM provider tiers32.9k stars, MIT76 out of 100
#4ContextForge MCP GatewayRegistry and proxy federating MCP, A2A, REST and gRPC behind one endpoint4.6k stars, Apache-2.068 out of 100
#5HigressEnvoy-based API gateway with LLM proxy plugins and MCP server hosting Live demo ↗ (opens in a new tab)9.5k stars, Apache-2.061 out of 100
#6BifrostGo AI gateway with web UI, fallbacks, budgets and semantic caching8.7k stars, Apache-2.059 out of 100