Local AI & LLMsEditorially verified
llama.cpp
ggml-org/llama.cpp
Run quantized LLMs and vision-language models across CPU and accelerated hardware.
A C/C++ inference engine focused on efficient local and server inference. Its README documents CPU/GPU hybrid use and backends for NVIDIA CUDA, AMD HIP, and other hardware. Hardware and model-size requirements vary; GPU acceleration is supported but not the sole execution mode.
GitHub stars
133,250
Forks
25,097
Open issues
2,581
Primary language
C++
Repository details
- License
- MIT LicenseMIT
- Default branch
- master
- Last commit
- Nov 3, 2026
- Repository updated
- Nov 3, 2026
- Last metadata check
- 2026-11-03
- Setup difficulty
- intermediate
- Self-hosting
- Self-hostable
- GPU
- Not documented
- API key
- Not required
- Paid service
- Not required for documented default
Categories & platforms
Local AI & LLMsSelf-hosted Apps
Supported platforms
CPU · NVIDIA CUDA · AMD HIP · macOS
Topics
LLM inferencequantizationC++CUDACPU
Editorial notes
Public, active and unarchived. GitHub reports MIT. README confirms local/server inference and multiple CPU/GPU backends. Model weights are separately licensed; review the chosen model license and hardware needs.
View license on GitHub