Runs on what you have
Apps and services you run yourself, grouped by what their README says about GPUs and arm64. Ranked by score in each group.
No GPU needed
102 projects in Best of Open-Source AI. Top 10 by score.
| Rank | Project | Score |
|---|---|---|
| 1 | OmniRouteOpenAI-compatible gateway that routes requests across hundreds of AI providers | 86 out of 100 |
| 2 | LiteLLMOpenAI-format gateway and Python SDK for calling 100+ LLM providers | 84 out of 100 |
| 3 | nanobotSmall Python agent runtime with bundled WebUI, TUI and chat channels | 84 out of 100 |
| #4 | LobeHubAgent workspace with builder, groups, scheduling and 10,000+ MCP skills | 80 out of 100 |
| #5 | OpenClawPersonal assistant gateway that answers in Discord, Slack, WhatsApp and Telegram | 79 out of 100 |
| #6 | LightpandaHeadless browser written in Zig for AI agents and scraping | 78 out of 100 |
| #7 | AnythingLLMDocument chat and agent app with built-in RAG, MCP and multi-user support | 77 out of 100 |
| #8 | MemPalaceLocal verbatim memory for coding agents on ChromaDB with 45 MCP tools | 77 out of 100 |
| #9 | n8nVisual workflow automation with code steps, AI agent nodes and 1500+ integrations | 76 out of 100 |
| #10 | LightRAGGraph-plus-vector RAG server with web UI and Ollama-compatible API | 76 out of 100 |
A GPU helps, not required
33 projects in Best of Open-Source AI. Top 10 by score.
| Rank | Project | Score |
|---|---|---|
| 1 | LocalAIOne OpenAI-compatible server for text, speech, image and video models | 82 out of 100 |
| 2 | MilvusDistributed vector database with dense, sparse and hybrid search at scale | 81 out of 100 |
| 3 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 79 out of 100 |
| #4 | Open WebUISelf-hosted chat UI for Ollama and OpenAI-compatible APIs with RBAC and RAG | 76 out of 100 |
| #5 | ComfyUINode-graph engine for diffusion image, video, audio and 3D models | 75 out of 100 |
| #6 | vLLMHigh-throughput LLM serving engine with OpenAI and Anthropic APIs | 75 out of 100 |
| #7 | OllamaRuns open-weight models locally behind a CLI and REST API | 74 out of 100 |
| #8 | VoiceboxLocal voice studio for cloning, TTS, dictation and agent speech | 72 out of 100 |
| #9 | colibriC inference engine that runs huge MoE models by streaming experts from disk | 70 out of 100 |
| #10 | Speech-to-SpeechModular voice-agent pipeline exposed through the OpenAI Realtime API | 67 out of 100 |
Needs a GPU
8 projects in Best of Open-Source AI. Top 8 by score.
| Rank | Project | Score |
|---|---|---|
| 1 | KTransformersCPU-GPU hybrid inference and fine-tuning for very large MoE models | 57 out of 100 |
| 2 | GPUStackGPU cluster manager that deploys models on vLLM, SGLang and TensorRT-LLM | 56 out of 100 |
| 3 | AI ToolkitFine-tuning suite for image, video and audio diffusion models, with GUI and CLI | 54 out of 100 |
| #4 | LMDeployLLM and VLM serving toolkit with the TurboMind and PyTorch engines | 54 out of 100 |
| #5 | Kohya's GUIGradio GUI and CLI for Kohya diffusion training scripts | 52 out of 100 |
| #6 | TabbyAPIOpenAI-compatible API server for running ExLlamaV3 models | 42 out of 100 |
| #7 | FluxGymWeb UI for training FLUX LoRAs on 12 to 20 GB GPUs | 39 out of 100 |
| #8 | OpenLLMOne-command OpenAI-compatible endpoints for curated open LLMs | 38 out of 100 |
A Mac or another arm64 machine
57 projects in Best of Open-Source AI. Top 10 by score.
| Rank | Project | Score |
|---|---|---|
| 1 | OmniRouteOpenAI-compatible gateway that routes requests across hundreds of AI providers | 86 out of 100 |
| 2 | nanobotSmall Python agent runtime with bundled WebUI, TUI and chat channels | 84 out of 100 |
| 3 | LocalAIOne OpenAI-compatible server for text, speech, image and video models | 82 out of 100 |
| #4 | MilvusDistributed vector database with dense, sparse and hybrid search at scale | 81 out of 100 |
| #5 | llama.cppC/C++ inference engine serving GGUF models over an OpenAI-compatible API | 79 out of 100 |
| #6 | hindsightAgent memory server with retain, recall and reflect operations | 78 out of 100 |
| #7 | LightpandaHeadless browser written in Zig for AI agents and scraping | 78 out of 100 |
| #8 | MemPalaceLocal verbatim memory for coding agents on ChromaDB with 45 MCP tools | 77 out of 100 |
| #9 | LightRAGGraph-plus-vector RAG server with web UI and Ollama-compatible API | 76 out of 100 |
| #10 | freellmapiOpenAI-compatible router that fails over across free LLM provider tiers | 76 out of 100 |
From each project's README: whether it needs a GPU and whether it mentions Apple Silicon or arm64. A project whose README says nothing about GPUs is left out. RAM needs are on each project page when the README states them.