A 304 billion parameter model should not look like something you run on a box that is 150 mm wide and weighs 1.2 kg.
Yet here we are. NVIDIA added DeepSeek V4 Flash 0731 to NGC in August 2026 and lists DGX Spark among the tested hardware. DeepSeek’s current 0731 release is listed at 304B total parameters with only 13B active for each token. The model can also handle up to one million tokens of context. On paper, that combination sounds almost silly for a desktop computer.
Then you look at the memory. DGX Spark has 128GB of unified LPDDR5X memory. NVIDIA’s current NGC package for DeepSeek V4 Flash is 155.44GB compressed, and NVIDIA labels Spark support as a validated multi node profile. So the official FP8 package does not magically squeeze into one 128GB box just because the product page says DGX Spark is supported.
That is where this story gets good. One Spark can run DeepSeek V4 Flash. Two Sparks can run a much less compromised version. Four Sparks can push huge context and several users at once. But every step changes what the phrase “runs locally” really means.
304B sounds impossible until you understand the 13B number
DeepSeek V4 Flash 0731 is a sparse Mixture of Experts model. It has 304B parameters in total, but around 13B are active for each token according to NVIDIA’s current NGC listing.
That does not mean you only need enough memory for a 13B model.
This confused me for a bit because the headline numbers look like they should solve the whole problem. They don’t. A Mixture of Experts model keeps a large set of experts available and routes each token through only some of them. You save compute because the full 304B is not being used for every token, but the weights still have to exist somewhere that the system can access.
So 13B active helps with compute. It does not turn the model into a 13B download.
The exact parameter number also changed across DeepSeek V4 releases, which makes old articles messy. The earlier V4 Flash pages listed smaller totals. The current DeepSeek V4 Flash 0731 page on Hugging Face is tagged as 304B, and NVIDIA’s August NGC release also says 304B total and 13B active. If you see 284B or 291B in an older post, you are probably looking at an earlier V4 Flash build rather than the 0731 release this article is about.
This matters because local AI discussions love one clean number. “304B on your desk” makes a great headline. The hardware does not care about the headline. It cares about bytes.
NVIDIA’s NGC artifact is marked FP8 and shows a compressed size of 155.44GB. Compressed download size is not the same thing as runtime memory use, and you also need room for the inference engine, KV cache, operating system, and other allocations. That makes a stock 128GB Spark a bad fit for the official artifact as a straightforward one box load.
And NVIDIA’s own wording confirms it. The current NGC page calls DGX Spark a “validated multi node profile.”
That tiny phrase is doing a lot of work.
The 128GB wall is real
DGX Spark is strange hardware in a good way. It has 128GB of coherent unified memory shared across the Grace CPU and Blackwell GPU. Memory bandwidth is 273GB/s. NVIDIA rates the GB10 chip at 140W TDP and advertises up to 1 PFLOP of FP4 compute with sparsity.
For local LLM work, I care more about the 128GB than the 1 PFLOP sticker.
A normal high end GPU can be much faster on a model that fits in its VRAM. But the moment a model needs 60GB, 90GB or 110GB, a 32GB card has a different problem. It is fast memory that cannot hold the thing you want to run. Spark trades some raw bandwidth for a much larger pool.
DeepSeek V4 Flash pushes that idea until it almost breaks.
The official NVIDIA package is larger than the memory pool before runtime overhead enters the picture. Even if you somehow got every byte packed perfectly, you would have no useful space left for long context. That is why the clean FP8 route moves toward two boxes.
Two DGX Sparks give you 256GB of combined unified memory and a 200Gbps ConnectX link between systems. NVIDIA itself says two Sparks can work with models up to 405B parameters. The company sells a two unit Spark bundle for $9,449 in the US, including the connecting cable.
That price is where the cute little desktop story starts to feel different.
You are no longer buying a fancy mini PC. You are building a tiny distributed inference system.
One Spark can run it, but there is a catch
This is the part that makes the simple answer annoying.
Yes, people are running DeepSeek V4 Flash 0731 on one DGX Spark right now.
They are not doing it by loading NVIDIA’s official 155GB FP8 NGC package in the obvious way. Community projects are using much heavier compression and custom serving code designed specifically around GB10’s memory limits.
One public project called ds4 on Spark uses a heavily compressed IQ2XXS style model path and downloads roughly 91GiB of model data. That is small enough to leave room inside the 128GB system for the server and a usable KV cache. The developer behind it reported around 35 tokens per second in earlier single stream testing and later around 59 tokens per second aggregate with 12 concurrent requests on the 0731 release. Community users have reported closer to 27 to 30 tokens per second for a single low effort chat stream, with high or max reasoning closer to 22 in one tested setup.
Read those numbers carefully. The 59 tokens per second figure is aggregate multi agent throughput. It is not one conversation blasting out 59 tokens every second.
Still, 25 to 30 tokens per second from a 304B class model on one tiny desktop is not bad at all. I would call that very usable for coding, private research, document work, and local agent experiments.
But compression has a price.
A model squeezed down near two bits per weight is not the same thing as the official FP8 artifact. One user on NVIDIA’s forum compared the local heavy quant with the API on a technical problem and said the API version produced much better results in that small test. They specifically noticed wrong line numbers and less precise numeric answers from the local quant. That is one person’s test, not a proper model evaluation, but it points at the exact tradeoff people tend to skip.
You can make the model fit. The harder question is what you gave up to make it fit.
There are rough edges too. A forum user reported about 112GB of memory in use with a 200K context configuration, then saw memory slowly rise during long runs. Another user described direct loading failures until a virtual memory based weight server was used. Later fixes addressed one long running memory growth issue, but this is still community tuned software rather than a boring one click desktop app.
That matters to me more than another screenshot saying “304B running locally.”
If your idea of local AI is downloading LM Studio, clicking a model, and chatting ten minutes later, this setup is not that. You are dealing with custom engines, context limits, server flags, quant choices, and memory floors. Sometimes it works great. Sometimes one setting eats the last few gigabytes and the box starts saying no.
Two Sparks are the cleaner answer, and the bill gets ugly
The more normal route for the official model is two GB10 systems.
NVIDIA forum users have documented DeepSeek V4 Flash running across two DGX Spark class machines using tensor parallelism over the ConnectX network. One published setup used two 128GB Sparks with a direct 200G connection and ran the official FP8 model with a 200K context. Short context single stream generation landed around the high 30s in tokens per second in that community test, with performance changing as context and concurrency increased.
Another community run pushed the same general setup to a 500K context window. The reported generation rate stayed in a usable range for a single stream at several tested context depths, though long prompt processing took more time as context grew.
These are community measurements. They are not NVIDIA product benchmarks, and software changes every few weeks. I would not buy $9,449 of hardware because somebody posted one nice number on a forum.
But the bigger point holds. Two Sparks move DeepSeek V4 Flash from “clever compression experiment” toward “serious local serving setup.”
You get far more memory headroom. You can keep a less compressed model. You get room for larger KV caches. You also get NVIDIA networking built for this sort of job.
And you now own two computers because one small computer was not enough.
This is where local AI economics gets funny. Cloud DeepSeek is easy. You pay for usage and go home. A dual Spark system asks for nearly $9,500 before electricity, storage, time, and whatever evening you lose because NCCL decided today is a good day to teach you networking.
The local version gives you privacy and control. It can run without sending your private code or documents to an external API. For people handling proprietary repositories or sensitive internal data, that alone can justify hardware.
For somebody who asks an LLM twenty questions after dinner, buying two Sparks to avoid API fees would be a pretty bad financial plan.
The 1 PFLOP number does not rescue everything
There is another number on the DGX Spark page that can confuse this discussion: 1 PFLOP of FP4 AI compute.
It sounds enormous. It is also a theoretical FP4 figure that uses sparsity. You should not read it as “this tiny box is faster than every desktop GPU.” Local LLM inference does not care about one compute number in isolation. Once a model is large enough, moving weights through memory can become a major limit.
Spark has 273GB/s of memory bandwidth. That is plenty for a compact system with 128GB of shared memory, but it is far below the bandwidth available on some large discrete GPUs. A high end GPU with less memory can therefore generate faster on a model that fits entirely inside its VRAM. Then it hits the capacity wall and the situation flips.
DeepSeek V4 Flash makes this trade very easy to see. The model activates only 13B parameters per token, so its sparse design cuts down the amount of compute needed compared with a dense 304B model. But it still has a huge weight set to keep available, plus routing, cache, and serving overhead. You cannot fix that with a bigger TOPS number printed on the box.
This is why I would never compare Spark with an RTX 5090 by asking which one is “faster” without naming the model. If the model fits on the 5090, the discrete GPU may have a large speed advantage. If the model needs 90GB, the 5090 is suddenly solving a different problem. Spark’s strange strength is that it can keep models in memory that normal gaming cards cannot hold.
DeepSeek V4 Flash sits right at the edge where that strength starts turning into a memory fight again.
The one million token context headline comes with another bill
DeepSeek V4 Flash 0731 supports a one million token input context. That is one of the weirdest parts of this model.
A million tokens can hold huge codebases, long agent histories, piles of documents, or enough log data to make your terminal feel like a bad life choice. DeepSeek has spent a lot of engineering effort on compressed attention to make long context less painful than it would be with a normal dense attention setup.
But long context still uses memory.
On one Spark, this becomes a fight over every gigabyte. The compressed ds4 on Spark project has been tested with very large contexts, even beyond one million tokens under stripped down conditions according to its developer. Normal users have reported a more practical comfort zone closer to 192K when they want enough free memory for batching and stable service.
One forum test described 262K context leaving about 1GB available and forcing requests into slower fallback behavior. Dropping to 196K restored roughly 8GB of headroom and made the server behave better.
That is a perfect example of why model support numbers need context of their own.
“Supports 1M context” tells you what the model architecture allows.
It does not promise that your 128GB desktop will serve one million tokens with good concurrency, lots of free RAM, and no compromise.
A four GB10 community setup published on August 13 is a better picture of what happens when you throw more memory at the problem. Four systems gave the cluster 512GB of combined memory. The tester reported about 66 tokens per second for one stream, around 185 tokens per second aggregate at eight concurrent requests, and a 1,048,576 token context configuration. That is much closer to a small local inference cluster than a PC under your monitor.
The boxes are small. The system is not.
So can one tiny DGX Spark run DeepSeek V4 Flash?
Yes.
But I would put a very large asterisk beside that answer.
One DGX Spark can run DeepSeek V4 Flash 0731 when you use a heavily compressed community build and software made around the machine’s 128GB memory ceiling. Real users are getting useful single stream speeds in roughly the 20 to 30 plus token per second area depending on setup, reasoning level, context, and serving path. Multi agent aggregate throughput can go higher.
The official FP8 NGC artifact is a different story. NVIDIA lists that package at 155.44GB compressed and describes Spark support as a validated multi node profile. Two Sparks are the more sensible route if you want the official style deployment with far less compression and more context headroom.
That makes the headline both true and misleading at the same time.
A tiny Spark can run a 304B model. It cannot ignore physics. The reason it works is a mix of Mixture of Experts design, aggressive quantization, unified memory, custom kernels, speculative decoding, and developers spending a frankly ridiculous amount of time squeezing every last gigabyte out of the box.
I like that more than the marketing version.
DGX Spark is interesting because it lets people experiment with models that used to belong almost entirely in server racks. DeepSeek V4 Flash is also a perfect stress test for its limits. The model is just large enough to expose where 128GB feels huge and where 128GB suddenly feels tiny.
If I already owned one Spark, I would absolutely try the compressed single box build. It is too interesting not to.
If I were buying hardware specifically for DeepSeek V4 Flash 0731, I would not pretend one Spark is the clean answer. I would budget for two, compare that $9,449 bundle against a larger workstation and cloud usage, and decide how much local privacy is really worth to me.
Because “it runs” and “this is the machine I should buy” are two very different questions.