How Much Does a Local LLM Actually Cost to Run? I Measured Every Watt on Apple Silicon

The conventional wisdom about running LLMs on your own hardware is that it’s effectively free. You already bought the machine, electricity costs are a rounding error, and every token after that is pure profit. A recent article from Towards Data Science, How Much Does It Actually Cost to Run a Local LLM?, challenged that assumption by measuring the power draw of an RTX 3090 and pricing it against real electricity rates. Its most striking finding stayed with me: the most expensive model to run wasn't the largest one. Cost per token tracked throughput, not parameter count. I don't have a 3090. I run everything on an M3 Ultra Mac Studio with 96 GB of unified memory—no discrete GPU, no VRAM, just a single memory pool shared between CPU and GPU. That's a different enough architecture that I wanted to test whether the same counterintuitive result holds on Apple Silicon, and by how much. So I measured it. Five models, sustained generation, real wall-socket energy, priced at my actual utility rate. The short version: not only does the surprise hold, but it hits even harder. On Apple Silicon, my 120-billion-parameter model costs about five times less per token than a model one-quarter its size. Here's how I got the numbers and why they land where they do. ## How the Measurement Works The instrument is a small tool I built called TokenWatt. It's an OpenAI-compatible proxy designed specifically for Apple Silicon: you point it at whatever local inference server you already run, and it forwards every request byte-for-byte while bracketing it with an energy measurement. It reads the whole-SoC power rails through Apple's IOReport interface—the same counters used by Activity Monitor's energy tab—so there's no need for sudo, no external instrumentation, just the numbers the chip reports about itself. It subtracts a rolling idle baseline to measure only the marginal cost of the request (not the machine sitting idle), then prices the result at your electricity rate. Mine is a flat $0.31/kWh. The honesty problem with any such measurement is that on-die counters can drift, and there's no way for a reader to know whether to trust them. So the one thing I insisted on—and the reason the tool exists at all—is calibration against ground truth. I ran every measurement in this article with the Mac plugged into a Shelly Plug US Gen4, which meters actual energy at the wall, and TokenWatt's built-in calibration fit the SoC counters to it—no hand-tuning, and the same procedure anyone with a metering plug can repeat. Every number below carries a real error band from that fit, landing between ±2.6% and ±4.5%. When you see "$0.109 per million tokens," that's a wall-verified figure with a stated uncertainty, not a hopeful reading off an internal register. For the headline numbers, I ran each model in a sustained generation loop—the machine doing nothing but decoding tokens as fast as it can—at three durations (120, 360, and 720 seconds), three passes each, with a 15-minute idle baseline between runs. That yields the cleanest, most reproducible signal: a saturated GPU, no gaps, no cold starts. It's a best-case scenario, and I'll return to what happens in messy real-world traffic later. ## The Five Models Everything here fits within 96 GB of unified memory, which turns out to matter (more on that later). The models span from a 4-billion-parameter dense model up to a 120-billion-parameter mixture-of-experts across different quantizations. "Footprint" indicates what the weights occupy in unified memory, and "tok/s" is sustained decode throughput. Cost is per million output (generated) tokens, at $0.31/kWh. | Model | Params | Type | Footprint | tok/s | $/1M out | |---|---|---|---|---|---| | Qwen3.5-4B | 4B | dense | 2.9 GB | 133.8 | $0.063 | | Qwen3.6-35B-A3B | 35B | MoE (3B) | 35 GB | 76.0 | $0.087 | | Qwen3-Coder-Next | ~80B | MoE (3B) | 60 GB | 65.0 | $0.103 | | gpt-oss-120b | 120B | MoE (5B) | 59 GB | 74.0 | $0.109 | | Qwen3.6-27B | 27B | dense | 28 GB | 21.5 | $0.554 | By 2026 standards, these models and measurement methods remain relevant as more users adopt local AI inference with Apple Silicon. The key insight—cost per token depends more on throughput than raw parameter count—becomes even more critical as models grow in size and complexity. For Apple Silicon users, the combination of unified memory and high memory bandwidth can make larger, more efficient models (particularly those using Mixture of Experts) surprisingly cost-effective compared to smaller dense models.

via Towards Data Science