Artificial Intelligence
On-premise AI architectures, local LLMs, RAG, autonomous agents. Intelligence you cannot afford to see switched off must be owned.
Explore →
Admina Enterprise
Open Source AI governance: audit trail, PII redaction, bidirectional ALLOW/BLOCK/REDACT policies on any model, local or remote.
Explore Admina →
Linux Services & Systems
Design, deployment and management of on-premise Linux infrastructure: from servers to AI hardware, with data that stays yours.
Explore →
Research & Development
Applied research and prototyping, from AI to infrastructure, in collaboration with universities and research centres.
Explore →What colibri is
colibri is an inference engine written in C, released by JustVugg under the Apache 2.0 license, that does one precise thing: it runs GLM 5.2, a Mixture-of-Experts model with 744 billion parameters, on a consumer machine with about 25 GB of RAM. Not a shrunk or distilled model: the actual large model, the same one we wrote about in GLM 5.2 and AI sovereignty, brought within reach of a mid-range PC.
The project is deliberately lean: zero external dependencies, no BLAS, no Python runtime, with hand-written quantized kernels and optional GPU backends. By mid-July 2026 it has roughly 11,500 stars and 900 forks on GitHub: a community project that clearly hit a nerve.
The idea: experts as citizens of the disk
The trick is in how colibri treats memory. Instead of requiring the whole model to sit in RAM (or worse, in VRAM), it treats VRAM, RAM and disk as a single hand-managed hierarchy. The dense components of the model, about 17 billion parameters (around 9.9 GB in int4), stay resident in RAM. The 21,504 routed experts, which make up the bulk of the model (about 370 GB on disk in int4), are not all loaded: they are streamed from the NVMe disk on demand, with a per-layer LRU cache and the operating system’s page cache as a second tier.
It is the same shift in perspective we highlighted for antirez’s DwarfStar 4: there it was the KV cache that became a citizen of the disk, here it is the experts of the MoE. In both cases the amount of RAM stops being a wall (“will this model run, yes or no?”) and becomes a continuous spectrum of speed: with enough space on a fast SSD, a huge model becomes practical, not just theoretical.
How it manages it
Under the hood colibri combines several recent efficient-inference techniques:
- Compressed KV cache with MLA attention: 576 values per token instead of 32,768, an order of magnitude less memory for context.
- Native speculative decoding with multi-token prediction, community-measured at around 2.2-2.8 tokens per forward.
- Quantized int8/int4/int2 kernels (AVX2), with optional CUDA and Metal backends for those with a GPU or an Apple Silicon Mac.
- KV-cache persistence across restarts, grammar-forced structured output and DSA sparse attention.
- An OpenAI-compatible API server, so it plugs into existing tools without changes.
The stated minimum requirements are modest for what they run: about 25 GB of RAM, 12 or more cores and a large NVMe SSD (the int4 model takes about 370 GB). Performance, measured by the community and to be taken as indicative because it depends heavily on hardware, ranges from 0.3 to about 1.2 tokens per second. It is not fast, and it is worth saying so: this is not a model for a fluid interactive chat, but for batch work, agents that reason patiently, or simply for having a frontier model at home that would otherwise require an entirely different infrastructure.
From the DGX cluster to the box under the desk
And this is where colibri really counts. The classic path to run a 744-billion-parameter model is a cluster: multiple NVIDIA DGX systems linked in a SuperPOD with NVLink and InfiniBand, as we described in the NVIDIA DGX platform. That is data-center hardware, with the costs and dependencies it brings.
colibri takes the opposite direction: the same model on a single machine, with 25 GB of RAM and an NVMe disk. It sits alongside the same family of tools as DwarfStar 4 and “AI on the desk” hardware like the NVIDIA GB10 workstations: different pieces of the same movement, carrying the frontier from the cluster in the machine room to the box under the desk. The trade-off is clear, speed for accessibility, but for many use cases it is exactly the right trade.
What we think
colibri is the practical demonstration of something we have repeated for a while: AI is too critical to be only a provided service. A frontier open-weight model that runs on your own hardware, without depending on an API someone else can throttle or switch off, is no longer a theoretical exercise but a git clone and a model downloaded from Hugging Face.
We have told the broader picture piece by piece: a government ordering the shutdown of a closed model (the Fable 5 case), frontier access becoming filtered upstream (GPT-5.6 Sol), and on the other side the open-weight models (GLM 5.2) and the engines that make them runnable locally. It is the sense of Open Intelligence, Secure Governance: own the operational floor and govern it, including with Admina, which brings audit and policy to any model, local or remote.
The limits are worth not hiding: colibri is slow, needs 370 GB of fast disk space, is a young community project, and remains tied to the availability of the quantized GLM 5.2 checkpoint. But the trajectory is the right one, and it is the direction we work in: on-premise AI, governed and sovereign, on Linux infrastructure that stays yours. A 744-billion-parameter model under the desk seemed like science fiction a few months ago. Today it is a matter of RAM, a large SSD and a bit of patience.