Meet qwenGPU: Qwen3.5-2B With 45 Compiled GPU Layers That Run CUDA Inside the Model
qwenGPU is an experimental model that compiles a SIMT GPU directly into the weights of Qwen3.5-2B. Developer Vixhal (@TheVixhal) released it on GitHub under the MIT license. The model runs real CUDA, compiled to NVIDIA PTX, inside its own forward passes. For developers, it tests one idea: exact computation inside a pretrained LLM, rather than beside it.
TL;DR
- Size: Qwen3.5-2B (24 layers) plus 45 compiled GPU layers, 69 in total. Context: 262,144 tokens (base model).
- Runs on: Apple silicon, an NVIDIA GPU, or a CPU, with roughly 16 GB of free memory. No quantized build.
- Performance: Every launch in the developer's runs was exact; text drift measured a KL divergence of about 7e-4.
- Best result: The reference interpreter matched a real NVIDIA Tesla T4 on 49 of 49 kernels.
- Worst result: About 10 to 15 tokens per second on Apple MPS; Qwen-written CUDA passed 4 of 5 tests.
What Is qwenGPU?
qwenGPU is Qwen3.5-2B with 45 compiled transformer layers placed in front of Qwen's own 24. None of those layers are trained β a compiler computes every weight.
via MarkTechPost