Local inference on DGX Spark: the recipes for DeepSeek V4 Flash, GLM-5.2 and Qwen3.6

Linux Services & Systems

Domains, hosting, PEC, email infrastructure, network services and Linux systems. Open Source infrastructure support and management.

Discover →

Cybersecurity

CISO-as-a-service consulting: posture, remediation roadmap, ongoing support.

Discover →

On 5 August Mia, who publishes as @MiaAI_lab, posted a list on X titled “What are the best models you can run on your NVIDIA DGX Spark? Aug 2026 Edition”. Ninety thousand views in a day, and underneath it a collection of GitHub recipes you can open and read line by line.

It is one of the few concrete answers to the question people ask when they weigh keeping models inside their own perimeter: what actually runs, and how fast.

The list

On a single unit the recommendation is DeepSeek V4 Flash. Two units are marked as the “sweet spot”. On three, GLM-5.2 is “still the best intelligence you can run locally if you have 3 sparks”. On four, GLM-5.2 NVFP4 spread across all of them. There is also a note of waiting: Qwen3.8-27B “should be released soon and this could change my recommendation”.

What sits underneath

The recipes are public, and that makes the list verifiable rather than arguable. The MiaAI-Lab GitHub organisation has 55 repositories, twenty of which concern the DGX Spark. The recipe for DeepSeek V4 Flash on two units has 442 stars, the monitoring dashboard 126.

Under each line of the list there is a different stack, and it is not vLLM everywhere.

On a single unit DeepSeek V4 Flash does not run on vLLM. The README says so plainly: “vLLM cannot read this repo’s asymmetric GGUF, so the repo ships its own server”. The engine is DwarfStar 4, the C and CUDA runtime written by Salvatore Sanfilippo, described in the recipe as “deliberately narrow, not a generic GGUF runner”. Mia’s repository is a wrapper around three third-party projects, among them the CUDA fork tuned for the Spark. The weights take about 110 GiB on disk and the KV cache costs 9.5 KiB per token.

On two units it moves to vLLM in tensor parallel, with the checkpoint pinned to a specific commit and the ghcr.io/anemll/dspark-vllm-gx10 image, a port of vLLM 0.25 for GB10. The two machines talk over RoCE, with the NCCL variables to be matched to your own card. The KV cache uses an experimental path, nvfp4_ds_mla.

On three units the checkpoint of the GLM-5.2 recipe is hybrid: NVFP4 for the hot weights (attention, shared experts, dense backbone) and AQLM at 2 bits for the cold MoE experts, the ones routing rarely activates. That fits 744 billion parameters into roughly 272 GB. The bottleneck is stated in the repository: the 2-bit experts “spend ~92% of their bus traffic on random 16B codebook gathers”. It needs a vLLM fork, not the official one.

Three figures combine different measurements

DeepSeek V4 Flash on one unit, 1M context and 26 tokens per second. The 26 comes from a decode benchmark on 256 output tokens, published as an image inside the repository, which at concurrency 1 measures 26.7 tokens per second. That benchmark does not state the prompt length, and the repository does not document a run at one million tokens of context. There is also a divergence between documentation and code: the README table gives 262,144 tokens as the default, while start.sh sets CTX="${CTX:-1000000}".

DeepSeek V4 Flash 0731 on two units, 1M and 82 tokens per second. The 82.4 tokens per second are the concurrency-1 row of a decode on 2048-token completions with short prompts. The README is honest and says as much: “This repo still does not claim a full 1M-token retrieval or correctness benchmark”.

GLM-5.2 on three units, 348k context and 25 tokens per second. The two figures belong to two different configurations. The path that reaches 348k measures about 21 tokens per second on structured output, between 13.6 and 19 on mixed load. The 25 or 26 tokens per second belong to the other path, which drops to 235k of context. Neither documented configuration does 348k at 25 tokens per second.

This is not an inflated list: the numbers all exist and sit in the repositories. It is that the maximum configurable context and the speed on short prompts are two quantities that are not measured together, and side by side in a table they look like the same experiment. Anyone sizing hardware on those figures is heading for a predictable disappointment: throughput falls as the context fills, which is exactly the use case a million tokens is bought for.

What it costs

A DGX Spark carries the GB10 superchip with 128 GB of unified LPDDR5x memory and 273 GB/s of bandwidth, a ConnectX-7 NIC at 200 Gbps to link two units, and it draws 240 W in a 1.2 kg enclosure.

The Founders Edition list price went from 3,999 to 4,699 dollars with an official announcement on 23 February 2026, justified by “industry wide memory supply constraints”, with no change to the hardware. At that price the configurations in the list cost roughly 4,700 dollars for one unit, 9,400 for the two-unit “sweet spot”, 14,100 for three and 18,800 for four, before cables, duties and VAT.

The 273 GB/s is the number to keep in mind more than the TOPS: on models of this size memory bandwidth is what sets how many tokens per second you get, and it is the same reason approaches like hybrid CPU and GPU inference and KV cache quantisation work.

The dashboard that powers off the fleet

Among the repositories there is sparkDash, the panel that watches several Sparks in one window: unified memory, bandwidth, vLLM queues, tokens per second per backend and power controls.

The README states something the list on X does not mention: “There is no authentication on the HTTP/WebSocket API. Run sparkDash only on a trusted network”, adding that this “includes power APIs (shutdown / Wake-on-LAN)”. Anyone who can reach that port can switch the cluster off.

The warning is written down and the author deserves credit for it. The question is what happens next: a convenient dashboard installed on a machine in the office, a firewall rule written in a hurry, and the trusted perimeter becomes the whole corporate network. It is the kind of detail that separates an experiment from a service, and it moves the question from running models to governing them, as in Open Intelligence, Secure Governance.

What we think

The list is useful because the practical piece is almost always missing from the open weights debate: not whether a model can be downloaded, but what happens when you switch it on using hardware you can buy. Here the models are all open weight, the hardware has a list price, the recipes carry an MIT licence and the numbers can be argued with because they are written down.

The numbers are self-produced: they are published by whoever wrote the recipes, measured on their own machine. The linked Hugging Face account declares three 128 GB GB10 units and one RTX 5090, which is more disclosure about the measuring hardware than most circulating benchmarks offer.

They can be compared, though, and that is the difference from a marketing figure. Sapid Labs collects DGX Spark measurements submitted by the people who run them at howtospark.com, “crowdsourced from real Sparks, with the recipes to reproduce every number”. On GitHub there is at least one other independent measurement of DeepSeek V4 Flash on two units, with a different configuration (FP8 in tensor parallel rather than NVFP4 with speculative decoding) and a different result, around 41 tokens per second single-stream.

The recipes depend on forks. The vLLM for GB10 is not the official one, the engine for DeepSeek is a fork of a fork, and the two-unit recipe drives somebody else’s repository. Putting one of these configurations into production means inheriting a chain of dependencies nobody has committed to maintaining.

The stated context is not the usable context. A million configurable tokens does not mean a million tokens usable at the published speed, and before signing a ten-thousand-dollar order that test is worth running on your own documents, with your own prompts.

Sources

• Mia’s post on X, 5 August 2026
• MiaAI-Lab on GitHub, the recipes
• DeepSeek V4 Flash recipe on one DGX Spark
• DeepSeek V4 Flash 0731 recipe on two DGX Sparks
• GLM-5.2 NVFP4 and AQLM recipe on three DGX Sparks
• sparkDash, the multi-Spark dashboard
• How To Spark, DGX Spark benchmarks submitted by the people running them
• elsung, an independent measurement of DeepSeek V4 Flash on two DGX Sparks
• NVIDIA DGX Spark, product specifications
• NVIDIA, DGX Spark price change announcement