So a few weeks back I was scrolling Hacker News like I do every night before sleeping, and I saw a post with like 700+ upvotes about someone running a 744 billion parameter AI model on a regular laptop. No GPU. Not even a good one. I thought it was a joke at first, or maybe some clickbait thing where the "laptop" actually has a hidden server rack attached to it.
It's not a joke. The project is called Colibri, and it was built by an Italian developer who goes by JustVugg on GitHub, real name Vincenzo. And I've been reading about it for the past week, actually going through the code myself, and I think it's one of the more interesting things to come out of the local AI space in 2026.
Here's the thing though. This is not going to replace your ChatGPT subscription tomorrow. There's a big catch, and I'll get to it. But first let's talk about what it actually does, because the "how" here is honestly the fun part.
What Colibri Actually Is
Colibri (it means hummingbird in Italian, kind of a cute name for something this heavy) is an inference engine. That means it's the thing that takes a model's weights and actually runs them to generate text, similar to what llama.cpp or vLLM does. Except this one is built to run GLM-5.2, which is Zhipu AI's massive open weight model, on hardware that has no business running something that size.
We're talking about a model with around 744 billion parameters. Some sources round it to 753B based on the Hugging Face listing. Either way it's huge. In the original 16-bit format the weights alone take up close to 1.5 terabytes. Even quantized down to INT4 it's still hundreds of gigabytes. A normal gaming laptop, or honestly most workstations, cannot fit that in RAM. Forget about VRAM, no consumer GPU even comes close.
So how does Vincenzo get this running on a machine with 25GB of RAM and no dedicated graphics card at all?
The trick is something called Mixture-of-Experts, or MoE for short. GLM-5.2 has 256 experts per layer, but for any single token, only about 8 or 9 of them actually get used. That means only around 40 billion parameters are "active" at any moment, out of the full 744 billion. Most of the model is just sitting there, unused, for that particular token.
Colibri exploits this. It doesn't load the whole model into memory. It keeps the parts that are always needed (attention layers, shared experts, embeddings, roughly 17 billion parameters worth) resident in RAM, and streams the rest from your SSD only when that specific expert gets activated. It's basically treating your hard drive as an extension of RAM.
I keep thinking of it like watching a movie online. You don't download the whole two hour file before pressing play. You stream the part you're watching right now, and the rest loads as you go. Colibri does that, except instead of video frames it's streaming chunks of a neural network.
The Code Itself
This is the part that actually impressed me the most, more than the RAM trick honestly. The core engine, the file called glm.c, is a single C file. Depending on which version you look at it's somewhere between 1,300 and 2,400 lines. No BLAS library. No CUDA required at runtime. No Docker. No Python needed to actually run inference, though you'll need it for some setup steps.
I spent a good two hours going through the source myself trying to understand the flow, and not gonna lie, some of the memory-mapping logic took me a while to wrap my head around. It reads from standard input and writes to standard output, that's it. That's the whole interface. Which sounds almost too simple for something running a 750 billion parameter model, but that simplicity is kind of the point. A file this contained can actually be read start to finish by one person, which is rare for anything in the LLM inference space these days.
From what I understand, it's essentially a C rewrite of the modeling_glm_moe_dsa.py file that Zhipu published through Hugging Face Transformers. Vincenzo took that Python reference implementation and rebuilt the math in raw C, with backends for CUDA if you do have an Nvidia card, and Metal if you're on a Mac. But the whole point is you don't need either.
Setting It Up
If you want to try this yourself (and honestly, I'd only recommend it if you're curious about the engineering, not if you actually need fast answers), the repo is public on GitHub under JustVugg/colibri. You clone it, download the GLM-5.2 INT4 model package that's built for Colibri specifically, point the engine at the model files, and you can start a local chat session. It can also run as an OpenAI-compatible API endpoint, so if you already have an app that talks to OpenAI's API, you just swap the base URL and it more or less works.
Basic requirements are around 25GB of RAM and a fast SSD with enough free space, since the full model on disk is close to 370GB. That's not nothing. You'll want an NVMe drive, not some old spinning hard disk, or things will get painfully slow.
Now the Catch. And It's a Real One.
Okay, here's where I have to be honest with you. This is slow. Like, really slow.
On the developer's own 25GB test machine, measured speeds were somewhere around 0.05 to 0.1 tokens per second. That means one token, roughly three quarters of a word, every 10 to 20 seconds. A short 100-token reply could take you 17 to 33 minutes. If the model does any kind of long reasoning trace before answering, you could be waiting hours for one response.
On better hardware it gets more usable but still not fast. A 48GB M4 Pro Mac reportedly hits around 0.3 tokens per second. A beefy 128GB M5 Max, running with Metal acceleration and some expert-pinning tricks, gets up to about 2 tokens per second. Still nowhere close to what you'd get from a proper GPU cluster or a hosted API.
So don't go into this thinking you're getting a private, free version of a frontier model that talks back instantly. You're not. This is closer to a batch job than a conversation.
So Why Does This Even Matter?
I asked myself this same question honestly. If it's this slow, what's the actual use?
A few honest answers came up while I was reading through discussions and the project's own documentation. First, cost. Once it's set up, there's zero marginal cost per token. If you have some kind of task where speed genuinely doesn't matter, like batch processing a huge folder of legal documents overnight, or summarizing a big pile of research papers while you sleep, slow-but-free beats fast-but-billed.
Second, privacy. If you're working with medical records or confidential company data that legally cannot leave your machine, running everything locally, even slowly, might be worth more than speed.
Third, and this is the one I actually find most interesting, it's proof that the whole "you need a massive GPU cluster to touch a frontier-scale model" assumption isn't as absolute as the industry likes to say. Colibri doesn't make Nvidia irrelevant, not even close, GPU clusters will always crush this in raw throughput. But it shows that a 744B parameter MoE model doesn't have to sit entirely in VRAM before it can produce a single token. That's a bigger deal conceptually than it sounds.
Vincenzo isn't alone in exploring this space either. Antirez, the guy who created Redis, has his own project called ds4 doing something similar with a GLM 5.2 branch, using SSD streaming techniques, and getting decent results on a 128GB M5 MacBook Pro. There's clearly a small but growing group of developers poking at the same idea from different angles: exploit MoE sparsity, treat disk as memory, skip the GPU tax entirely.
What Vincenzo Says About It
From what he's posted, this whole project started basically as a personal challenge, not a startup pitch or anything commercial. On the original Hacker News thread he mentioned something like, the important part was the journey of getting there, not necessarily ending up with something production ready. He's also upfront about the limits of his own testing, saying he doesn't have access to more powerful machines himself, so a lot of the higher-end numbers people quote (like that 2 tokens per second on the M5 Max) actually come from the community testing it, not from him.
That kind of honesty is refreshing, I think. A lot of AI tooling gets marketed with a straight face like it's already production ready, when really it's a proof of concept someone built in their spare time. Colibri doesn't pretend to be more than that.
Should You Actually Try It?
Depends what you're after. If you want to experiment with a genuinely frontier-scale open model without spending anything on GPU rental or API credits, and you don't mind waiting a long time per response, sure, go for it. If you're a researcher comparing quantization approaches on consumer hardware, the layer-mapping logic in Colibri is a decent baseline to study.
But if you need something that responds in real time, or you're building a product that needs to serve multiple users, this isn't it. Stick to a proper GPU setup or a hosted API for that. Colibri was never trying to compete there anyway.
What I keep coming back to is just, this is a solo developer, working with a single C file, proving something that a lot of well funded labs probably assumed wasn't really worth exploring. Sometimes the interesting stuff in AI isn't the biggest model or the fastest chip. It's someone figuring out a clever way around the constraint everyone else just accepted as fixed.
The repo is public, the code is short enough that you can actually read the whole thing in an afternoon, and if you've got a spare SSD and some patience, you can watch it happen yourself, one very slow token at a time.