A friend sent me a screenshot last month. His 16GB MacBook Air was running a model that typed roughly one word every two seconds. He’d downloaded a 14GB file and figured 14 fits inside 16, easy. It doesn’t.
macOS keeps a big slice of unified memory for itself, and on a 16GB Mac only about 12GB is usable by the model, going by the numbers in OpenClaw DC’s Mac mini guide. The context window needs its own space on top of that, and it grows as the chat gets longer. Once the model spills over, the Mac starts using the SSD like memory and everything crawls. The file size itself was fine in his case. The leftover memory was the problem. I think this one mistake is behind most of the “local LLMs are useless” posts I see.
And the fix is boring. Take the file size, add 3 to 4GB for context, and check that the total stays under what your Mac really gives the GPU. That’s my own rule of thumb, not an official number. Models also change fast, and I’m writing this on 4 November 2026, so a lot of older guides still recommend models that newer ones clearly beat. Qwen 3.6 came out in April, Qwen 3.8 is already showing up in recent guides, and OpenAI’s gpt-oss is still listed by several of them as one of the best for tool calling.
A few terms before the tiers
Every model page throws these words at you, so here they are in plain English. Quantization means storing the model’s numbers with fewer bits. Q8 is close to the original quality, Q6 is a hair below that, Q4 is about half the size of Q8 with a small drop, and Q3 starts hurting on harder tasks. Unsloth’s page for Qwen 3.6 shows the pattern well: the 27B model needs roughly 15GB at 3-bit, 18GB at 4-bit, 24GB at 6-bit and 30GB at 8-bit. For most people Q4 to Q6 is where I’d stay.
Next is dense versus mixture-of-experts, usually written MoE. A dense 27B model uses all 27 billion parameters for every word it writes. An MoE like Qwen 3.6 35B-A3B has 35 billion in total but only uses about 3 billion per word, which makes it much faster. The catch is that the whole model still has to sit in memory, so you save speed, not space. That’s mostly true, though I’m oversimplifying: some hobby tools can stream the unused experts from the SSD, at a big cost in speed.
The last term is the KV cache, which is the memory the model uses to remember the conversation so far. Long chats and long documents make it bigger. Gemma 4 31B advertises 256K context and gpt-oss advertises 128K, but on a laptop you’ll rarely have room to use all of it.
Speed is a separate question, and it depends mostly on memory bandwidth, not core count. The specs I found list 153GB/s for the base M6 at 16GB and 170GB/s at 24 or 32GB, 307GB/s for the M5 Pro and 614GB/s for the M5 Max. A rough estimate of tokens per second is bandwidth divided by the size of the active model. A GitHub user called nabe2030 tested this on an NVIDIA DGX Spark: 273GB/s divided by a 17GB model predicts about 16 tokens per second, and the measured result was 11.85. So real speed lands at around three quarters of the theory. It also means an older Max chip can beat a newer base chip, even when the base chip has the better name.
16GB MacBooks
Plan for about 12GB. At this size you’re choosing between 9 and 12 billion parameter models, and the one I’d start with is Qwen 3.5 9B. ModelFit lists it at around 11GB loaded at Q8, which is already too tight by my rule, so I’d download Q5 or Q6 and keep the extra space for context. Gemma 4 12B at 4-bit is the other one to try, mostly to compare how the writing feels, since that part is taste.
I’d skip gpt-oss-20b here, and that’s me disagreeing with a few guides. Analytics Vidhya’s Mac mini list from September says 16GB machines are fine for it. The model is about 14GB though, and 14 plus context is nowhere near 12. Maybe it runs with a short chat and every other app closed. Chrome with twenty tabs will probably push it over the edge.
But 12GB is still enough for real work. A 16GB Mac handles rewriting an email, summarising a few pages, explaining a stack trace, translating a paragraph, and answering questions on a flight with no wifi. What it can’t do is anything with five steps. Ask a 9B model to plan a change across several files and it loses the thread around step three. I spent a while confused about this at first, because the benchmark charts make small models look much closer to the big ones than they feel in actual use.
Side note, the MacBook Air has no fan, so a long generation will throttle after a few minutes. The Pro has active cooling and holds its speed better, which ModelFit also points out.
There are tricks to stream a bigger model from the SSD. A Hugging Face repo called mlx-expert-sniper reports 4.15 tokens per second for Gemma 4 26B on a 16GB M4 Mac mini, which is a great name for a project. It’s a demo, not a daily tool.
24GB MacBooks
This is where it gets useful, with one awkward detail. By default macOS gives the GPU somewhere around two thirds to three quarters of memory, so on a 24GB Mac that’s maybe 16 to 18GB. Unsloth’s table puts Qwen 3.6 27B at 18GB for 4-bit and the 35B-A3B at about 22GB, which means neither fits comfortably under the default limit. You have two choices. Download the 3-bit versions, which are 15GB and 17GB, or raise the limit yourself with a terminal command like sudo sysctl iogpu.wired_limit_mb=20480. As far as I know that works on recent macOS versions, it resets when you reboot, and I'd leave at least 4GB for the system. If you set it too high, the Mac gets unstable and you'll find out the annoying way.
Nobody mentions this on the model pages.
My pick for daily use is Qwen 3.6 35B-A3B. It’s multimodal, so it takes images, and it’s quick because only 3 billion parameters are active per word. On an M5 Max with 128GB, a benchmark by a GitHub user called stared measured about 85 tokens per second for it on MLX, against 17 for the dense 27B. Your 24GB machine will be slower than that, probably a lot slower on a base chip, but the gap between the two models is what matters.
The 27B is what I’d switch to when an answer has to be right. The same write-up quotes the Artificial Analysis index at 37 for the 27B and 33 for the 35B-A3B, so the dense model is a bit smarter and a lot slower.
gpt-oss-20b fits comfortably here at about 14GB. It’s what I’d use if you want a local agent that calls tools, since a couple of guides describe its tool-call output as the cleanest of the open models. For coding, Qwen3-Coder 30B is good in coding and not much else, and Analytics Vidhya recommends 24GB or more for it. Gemma 4 26B-A4B comes to about 15.6GB at 4-bit on MLX and makes a fair second opinion.
With 24GB you can paste in a 20 or 30 page report and ask questions, summarise meeting notes, pull structured data out of messy text, and get decent coding help inside an editor. Keep the documents short enough that the cache doesn’t eat the rest of your memory.
One honest limit. Stared’s write-up says these local models sit around where cloud models were in mid-2025 to early 2026, which means they’re still a generation behind the best ones on hard problems. That’s fine for most daily work. It’s not fine for a tricky bug in a big codebase.
32GB and above
At 32GB the machine starts to feel comfortable. Qwen 3.6 27B at Q6 is about 22GB, which leaves real room for context, and OpenClaw DC’s 32GB guide picks it as the best all-round model for this size. The 35B-A3B at Q5 is around 24 to 26GB if you’d rather have speed. Gemma 4 31B at Q6 is about 25GB and worth having for a second opinion, though one test on a DGX Spark found its prompt processing dropped 12 to 13 percent as prompts grew from 512 to 16,000 tokens, where Qwen only dropped 3 to 6 percent. So for long documents I’d lean Qwen. At 48GB you can run the 27B at Q8, about 30GB, and still keep a browser and an editor open.
At 64GB the newest guide I found, from OpenClaw DC last week, picks Qwen 3.8 27B at Q8, a 27GB file, with Qwen 3.6 35B-A3B at Q8 right behind at 34GB. I haven’t seen much independent testing of Qwen 3.8 yet, so treat that pick as a good bet and not a settled answer.
Which brings up the Llama question. Llama 3.3 70B at 4-bit is the biggest dense model that fits in 64GB, and Analytics Vidhya says ollama run llama3.3:70b works on a 64GB M5 Pro. It's an older model though, and I'd expect single digit tokens per second on a Pro chip. The M4 Pro's 273GB/s divided by a roughly 40GB model gives about 7, and a hardware roundup on fungies.io claiming 15 to 25 for 70B on a 48GB M4 Pro looks too high to me. The bigger surprise is that gpt-oss-120b does not fit. Its file is about 59GB, and a 64GB Mac gives the GPU roughly 52GB by default. A lot of older "best for 64GB" posts still recommend it.
Only an M4 Max or M5 Max gets you to 96 or 128GB, and 128GB is the one laptop size that holds the big MoE models. gpt-oss-120b has 117 billion parameters, uses 5.1 billion per word, ships under Apache 2.0 and needs about 66 to 74GB in its native 4-bit format. Qwen 3.5 122B-A10B is the broader-knowledge option at roughly 70 to 86GB depending on the quant. Speeds for gpt-oss-120b are all over the place. willitrunai estimates 67 tokens per second on an M4 Max, OpenClaw DC says 14 to 20 for a larger Q6 version on the same chip, and another of their posts claims 64 to 88 on an M5 Max at Q8. Those can’t all be right, so I’d plan for the low end until you’ve tested it yourself. There’s also a tool-calling bug in Ollama (issue 14493) that OpenClaw DC reported with the Qwen 122B model earlier this year, and I couldn’t confirm whether it’s fixed. If you’re building an agent on it, check that first.
DeepSeek V4 Flash is the ceiling. It has 284 billion parameters with 13 billion active, comes under an MIT license, and a 3-bit version needs 110 to 135GB. A project called Mference claims 4.8 tokens per second for it on a 24GB Mac by streaming experts from the SSD. It’s experimental and I wouldn’t build anything on it yet.
Cheat sheet
Sizes below are approximate and come from the guides mentioned above, so check the exact file on Hugging Face before you download.
+--------+----------------+----------------------+-----------------+--------------------------------------+
| Memory | Start with | Quant | Rough size | Good for |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 16GB | Qwen 3.5 9B | Q5 or Q6 | about 9GB | Rewriting, short summaries, |
| | | | | simple Q&A |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 24GB | Qwen 3.6 35B- | 3-bit, or 4-bit with | 17 to 23GB | Daily chat, images, |
| | A3B | a raised limit | | documents |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 24GB | gpt-oss-20b | Native 4-bit | about 14GB | Tool calling and agents |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 24GB | Qwen3-Coder | 4-bit | roughly 17 to | Coding in an editor |
| | 30B | | 19GB | |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 32GB | Qwen 3.6 27B | Q6 | about 22GB | Hard questions, longer code |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 64GB | Qwen 3.8 27B | Q8 | about 27GB | Best quality with long |
| | | | | context |
+--------+----------------+----------------------+-----------------+--------------------------------------+
| 128GB | gpt-oss-120b | Native 4-bit | 66 to 74GB | Agents and heavy coding |
+--------+----------------+----------------------+-----------------+--------------------------------------+For images, Qwen 3.6 and Gemma 4 both take them as input. For writing, I’d try two models and keep the one that sounds less like a manual.
Before you download anything
Install Ollama or LM Studio first. Ollama runs from the command line and LM Studio is a desktop app, and either will only take two minutes to set up. MLX is Apple’s own library for running models fast on these chips, and llama.cpp is the older open engine that most of the other tools sit on. The usual advice is that MLX is faster on a Mac, but stared’s benchmark found llama.cpp ahead by 10 to 24 percent on an M5 Max for Qwen 3.6. That was one machine and two models, so test on yours.
To check a model fits, open Activity Monitor, go to the Memory tab, and watch memory pressure while the model answers a long question. Green or yellow is fine. Red means it’s too big, and you should drop to a smaller quant.
If it’s too slow, the fixes are usually small. Close Chrome first, because browsers tend to be the biggest memory user on a laptop. Then shorten the context length in your tool’s settings, since a 32K window costs far more memory than an 8K one. Only after that would I go to a smaller quant. I’d rather lose a bit of quality at Q4 than run Q6 with a context so short that the model forgets the start of my own question.
For long PDFs on a 16GB or 24GB machine, split them into chunks of about 10 pages and ask the same question on each one, then ask for a combined answer at the end. It’s slower, but one giant paste would fill the cache and send you straight back to the crawling problem from the start.
If you’re buying right now, the M6 MacBook Pro 14 goes on sale on 6 November, with pre-orders open since 28 October according to aicybr. The base has 16GB and you can configure 24GB or 32GB, at 153 to 170GB/s. Nobody has published real LLM benchmarks for it yet, so any speed number you see this week is an estimate from the bandwidth. If I had to pick one configuration for local models, it’d be 32GB. The 16GB version is for small stuff only, and 24GB makes you do the sysctl dance from earlier.
People will say just use ChatGPT, and for hard problems that’s fair. It doesn’t help on a plane, or with files you can’t upload.
Tonight, download Qwen 3.5 9B or the 35B-A3B, whichever fits your Mac. Ask it five real questions from your own work, the kind you’d normally paste into a chat window. If three of the answers are usable, keep it. If not, try the next size up.