Analysis · 10 min · August 27, 2026

DwarfStar, and the case for an engine that runs one model

Salvatore Sanfilippo published ds4 — DwarfStar — on GitHub on May 7, 2026, written in C and Metal over roughly a week of long days. It is MIT licensed and it has since collected about 21,900 stars. The README describes it in one sentence: “a small native inference engine optimized first for DeepSeek V4 Flash.” It also runs GLM 5.2, and DeepSeek V4 PRO on machines with enough memory to be called infrastructure. That is the whole model list.

Sanfilippo named it after what a dwarf star is: most of the mass of the original, in a fraction of the volume. The name is doing real work here, because the compression is the entire technical argument.

The quantization is asymmetric on purpose

Every general-purpose runtime offers you a quantization level and applies it more or less uniformly. Pick Q4, and the model gets squeezed to roughly four bits throughout. ds4 refuses that framing. From the README: “Only the routed MoE experts are quantized, up/gate at IQ2_XXS, down at Q2_K. They are the majority of all the model space: the other components (shared experts, projections, routing) are left untouched to guarantee quality.”

That sentence is the design. In a large mixture-of-experts model, the routed experts hold most of the parameters but any given token touches only a few of them. The components that fire on every single token — attention projections, the shared experts, the routing network — are comparatively tiny. Crushing the first group to two bits buys almost all of the memory saving. Leaving the second group alone costs almost nothing in space and preserves the parts where damage would compound across every token.

Sanfilippo has described the result as an extremely asymmetric 2/8 bit recipe. It is not a new idea in the abstract — the quantization scene has been arguing about mixed-precision layouts for years, and imatrix builds already vary precision by tensor. What is different is targeting: because ds4 only has to be correct for one architecture, the recipe can be tuned to that architecture instead of being a default that has to be safe for a thousand models it has never seen.

What it costs in hardware

This is where the enthusiasm needs a floor under it. The README is specific: the Q2 build targets “96/128 GB RAM machines,” the Q4 build wants 256 GB or more, and PRO at Q2 wants 512 GB.

A 128 GB Mac is a five-figure machine in most configurations. This is not the hardware class the rest of this site is written for, and it would be dishonest to present ds4 as though it democratises anything for a reader with a 24 GB card. What it does is move the boundary: work that previously required a rack now fits in one expensive but purchasable computer. That is a real change, and it is a change for a specific and fairly small group of people.

There is an escape hatch, and the author is candid about its cost. On Metal, ds4 can stream from SSD — routed experts live in an in-memory cache and get loaded from the GGUF file on a miss — with the flat caveat that “streaming is not as fast as fitting the full model in RAM.” Useful for trying the thing before committing to the machine. Not a substitute for the machine.

The numbers, from the project's own table

On an M5 Max with 128 GB running the Metal backend, the README reports 790.18 tokens/s prefill and 39.35 tokens/s generation at a 2,048-token context, falling to 398.50 and 27.64 at 65,536 tokens. On an NVIDIA DGX Spark GB10 with 128 GB under CUDA: 825.76 prefill and 18.05 generation at 2,048 tokens, and 822.98 and 13.84 at 65,536.

Two things in that table are worth sitting with. The first is that generation on the Mac beats the DGX Spark by a wide margin while prefill goes the other way, which is a memory-bandwidth story rather than a compute story and matches what we said about unified memory versus VRAM. The second is that the DGX Spark barely slows down on prefill between 2k and 64k context while the Mac drops by half. If your workload is long documents read once, those are different machines than the headline generation figure suggests.

These are the project's own published benchmarks, run by the author and contributors on their own hardware. We have not reproduced them.

What the author admits, which is most of what you need

The README carries its own warnings, and they are better than most third-party reviews. “The software is currently very fast changing. Consider it beta quality.” On the CPU path: “Do not treat the CPU path as the production target.”

The most useful admission concerns distributed inference. ds4 can split a model across machines — pipeline parallelism over the network, tensor parallelism over RDMA between two Macs on Thunderbolt 5, enough to fit PRO at Q4 across a pair of 512 GB Mac Studios. But generation stays strictly autoregressive, so, in the author's words, distributed inference “is therefore mainly for fitting larger models and speeding up long prefills, not for making decode faster.” Anyone who has watched a distributed-inference demo and assumed two machines means twice the speed should read that sentence twice.

What adding it to the catalogue cost us

ds4 is now in the tools page, and getting it there meant widening what that page is for. The catalogue used to assume software a reader could install on hardware the rest of the site addresses, which put a 128 GB floor outside its scope by definition. We changed the rule rather than bend the entry: machines in that class are no longer exotic, and a catalogue that pretends the DGX Spark and the large Mac Studio do not exist is describing 2024. The entry leads with the memory requirement so nobody reaches the install instructions before finding out they cannot use them.

DeepSeek V4 Flash — the model ds4 exists to run — was missing from the directory for the same reason when this post went up. That half has since been fixed: it now has an entry and a hardware profile, with memory figures measured from the unsloth GGUF builds. Worth noting where those figures land against the numbers above, because they do not agree. The smallest stock GGUF in that repository is about 91 GB and the Q2 build is about 97 GB, so a 96 GB machine misses both once you leave room for context. The 96 GB floor quoted here is a property of ds4's own quantization, not of the model, and that is precisely the point the engine is making.

The broader point about specialisation

We have argued before that the local inference engines solve different problems and that choosing between them is mostly a question of what you need, not which is fastest. ds4 adds a category that piece did not anticipate: the engine that treats generality as a cost rather than a feature.

That trade is only available to someone willing to rewrite when the target moves, and it is the reason a project like this comes from a systems programmer with a track record of maintaining one thing for a decade rather than from a startup that needs a roadmap. Whether the approach survives depends on something outside the code: how long DeepSeek keeps shipping models whose architecture the recipe still fits. The bet is legible, the author has made it explicitly, and the code is MIT licensed either way.