Colibrì Runs a 1.5TB AI Model on 25GB of RAM: The Local AI Breakthrough That Changes Everything
An Italian engineer built Colibrì — a proof-of-concept that runs a 744-billion-parameter model (1.5TB) on a standard CPU with 25GB of RAM. At 0.05 tokens/second it's not real-time, but the technique could transform local AI accessibility.
An Italian engineer named Vincenzo — known online as JustVugg — released a proof-of-concept called Colibrì that does something that shouldn't be possible. It runs GLM-5.2, a 744-billion parameter Mixture-of-Experts model weighing 1.5 terabytes, on a standard consumer PC with just 25 GB of RAM and a regular CPU. No GPU. No cloud credits. No special hardware.
At 0.05 to 0.1 tokens per second, it's not usable for real-time conversation. A single sentence takes minutes. But the architecture is what matters — and it could change how we think about local AI.
How Colibrì Works
The key insight is simple. Most frontier AI models use a Mixture-of-Experts (MoE) architecture. In MoE, only a fraction of the model's parameters are "active" for any given input. The experts are scattered across the model, and a gating network decides which experts to activate for each token.
Colibrì exploits this property in an aggressive way. Instead of keeping all parameters in RAM — which would require 1.5 TB — it stores the model on an NVMe drive and streams only the active experts into memory as needed. The 25 GB of RAM holds the model's embedding layers, the gating network, and a cache of recently used experts. When a new token requires an expert that isn't cached, Colibrì reads it from the NVMe drive in real time.
The technique is called on-demand expert streaming. It's a variation of memory-mapped model execution, but optimized for MoE architectures in a way that no one has publicly demonstrated at this scale.
The entire engine is written in pure C with zero external dependencies. This matters for two reasons: it keeps overhead minimal, and it means Colibrì can run on basically any system with a modern CPU and an NVMe slot.
Why 0.05 Tokens Per Second Still Matters
Let's be honest about the limitations. At 0.05 tokens per second, generating a standard paragraph takes about 20 minutes. A 500-word report would take roughly 3 hours. Real-time conversation is out of the question.
But Colibrì isn't a product. It's a demonstration that the bottleneck isn't fundamental. The model runs. It produces coherent output. The architecture works.
The question is whether the latency can be improved. Colibrì is bottlenecked by NVMe read speed — roughly 1 GB/s on a typical consumer NVMe drive. Each expert takes about 200 milliseconds to load. With more aggressive caching, smarter prefetching, and faster storage (PCIe 5.0 NVMe drives can reach 7 GB/s), the real-world speed could improve by 5x to 10x without any architectural changes.
There's also the possibility of dedicated hardware. If someone built a chip specifically for streaming expert inference — with a fast NVMe controller, an optimized memory hierarchy, and a simple systolic array for the active expert computation — the architecture could potentially reach 10 to 50 tokens per second. Not fast enough for real-time conversation, but fast enough for batch processing, code generation, and document analysis.
The Local AI Implications
If Colibrì's approach can be scaled to usable speeds, it would have significant implications for how AI models are deployed.
<<<BOLD>>>Privacy.<<<BOLDEND>>> Sensitive data never leaves the device. Enterprises in healthcare, finance, defense, and law could run frontier-level models entirely on-premises. No API calls. No data leaving the building. No cloud provider accessing inference logs.
<<<BOLD>>>Cost.<<<BOLDEND>>> Running a 744B-parameter model on a $2,000 consumer PC is dramatically cheaper than API inference at scale. The per-token cost effectively becomes electricity. For companies doing heavy inference workloads, the economics could flip.
<<<BOLD>>>Offline capability.<<<BOLDEND>>> Colibrì doesn't require internet access. A laptop with a fast NVMe drive and 25 GB of RAM could run frontier-level AI on a plane, in a rural area, or in a facility with no cloud connectivity.
<<<BOLD>>>The democratization argument.<<<BOLDEND>>> Currently, access to frontier AI models is controlled by a handful of API providers — OpenAI, Anthropic, Google, Meta. A local inference engine that can run their models on consumer hardware breaks that dynamic. If the model weights are available and the hardware is affordable, anyone can run frontier AI.
The Catch
There are significant catches. First, Colibrì currently only supports GLM-5.2, which is a Chinese model developed by Zhipu AI. Running frontier Western models — GPT-5.6, Claude Sonnet 5, Gemini 3.5 — would require those models to be available in a format Colibrì can consume. Most frontier model companies don't distribute their weights for local use.
Second, even with optimizations, the architecture is inherently slower than GPU-based inference. GPUs are massively parallel. Colibrì runs on CPUs which are sequential by design. Closing the gap entirely would require specialized hardware.
Third, 0.05 tokens per second is a proof of concept, not a product. Getting from "it works" to "it's useful" requires an engineering effort that goes well beyond what one developer can achieve alone.
Why It Matters Anyway
Colibrì's real significance isn't its current speed. It's the demonstration that the memory wall — the idea that large models require proportionally large amounts of RAM — might not be a hard constraint.
If the MoE streaming approach works at scale, it opens a design space for model architecture that nobody has seriously explored. Why optimize for RAM efficiency when you can optimize for streaming efficiency? Why design models that fit in GPU memory when you can design models that load their experts efficiently from fast storage?
This is the kind of breakthrough that looks impractical today but foundational tomorrow. It's also a reminder that the most important AI advances don't always come from well-funded labs. They sometimes come from one developer in Italy with a computer and a good idea.