LiteLLM vs freellmapi
Two of the top gateways, side by side: score, setup, license, activity and what each review found.
LiteLLM
OpenAI-format gateway and Python SDK for calling 100+ LLM providers
freellmapi
OpenAI-compatible router that fails over across free LLM provider tiers
| What we compare | LiteLLM | freellmapi |
|---|---|---|
| Score parts, out of 100 | ||
| Adoption | 89, widely used | 76, popular |
| Freshness | 100, active | 100, active |
| Maintenance | 83, healthy | 98, healthy |
| Easy to run | 83, very easy | 67, easy |
| Agent-ready | 30, minimal | 0, none |
| Facts from GitHub and the README | ||
| Stars | 61k | 33.1k |
| License | custom license (read the license) | MIT (permissive) |
| Last commit | Oct 2026 | Oct 2026 |
| Last release | Oct 2026 | Oct 2026 |
| Language | Python | TypeScript |
| Docker | Yes | Yes |
| GPU | Not needed | Not needed |
| arm64 or Apple Silicon | Not stated | Mentioned |
LiteLLM
LiteLLM translates calls to 100+ providers (OpenAI, Anthropic, Gemini, Bedrock, Azure and others) into the OpenAI format, either as a Python SDK or as a proxy server. The proxy adds virtual keys, spend tracking, guardrails, load balancing and an admin dashboard, and it also exposes A2A agent and MCP server gateways. The README reports 8ms P95 latency at 1k RPS.
Who it is for: Teams routing many LLM providers through one self-hosted API
Strengths
- One OpenAI-style API across 100+ providers and many endpoint types
- Proxy includes virtual keys, spend tracking, load balancing and admin dashboard
- Also gateways A2A agents and MCP servers
- Use as a Python library or as a standalone proxy
Weaknesses
- License reported as NOASSERTION; an enterprise tier exists, feature split unclear from README
- Proxy listens on port 4000 and its database requirements are not stated in the README excerpt
- Provider coverage varies by endpoint; many providers support only chat-style endpoints
- Python-based, so latency figures depend on the benchmark setup
- no GPU
- Docker + Compose
- Compose runs PostgreSQL
- Models: OpenAI, Anthropic, Gemini, AWS Bedrock, Azure
- port 4000
freellmapi
FreeLLMAPI exposes one /v1 endpoint (chat, responses, completions, embeddings, images, video, audio) and routes requests across free tiers from 34 providers, plus custom OpenAI-compatible endpoints. Provider keys are AES-256-GCM encrypted in SQLite, per-key RPM/RPD/TPM/TPD counters keep requests under quotas, and a 429 or 5xx triggers fallover to the next model. It also serves Anthropic Messages, Gemini and opt-in Ollama surfaces, and ships a React dashboard and desktop apps.
Who it is for: Developers stacking free LLM tiers behind one endpoint for coding agents or apps
Strengths
- Also speaks Anthropic, Gemini and Ollama formats, so Claude Code and Codex CLI connect
- Per-key rate counters and automatic fallover on 429/5xx across providers
- Keys AES-256-GCM encrypted in SQLite; apps only see one unified token
- Runs on Node 20+ at about 40 MB idle RSS, or via Docker
Weaknesses
- Free installs get new models 30 days after premium; same-day catalog costs $19/yr
- Single-user by design; no multi-user setup described
- Depends on free tiers that providers can change or retire without notice
- Catalog sync pulls a signed feed from freellmapi.co
- no GPU
- Docker + Compose
- Needs SQLite, Node 20+, provider API keys
- Models: OpenAI-compatible API, Anthropic Messages API, Gemini API, Ollama API
- port 3001