New LoRA Adapter Boosts Qwen3.8 Coding Success from 39% to 65%

New LoRA Adapter Boosts Qwen3.8 Coding Success from 39% to 65%


Hugging Face's harness-hub team has released Qwen3.8-27B-pi-adapted, a LoRA adapter for Qwen3.8-27B. Trained with reinforcement learning, the adapter is designed to operate inside the pi coding agent. Rather than optimizing for general chat, Qwen3.8-27B-pi-adapted targets a single harness. For developers running local coding agents, that focus is the point: you keep the open base model and swap in a small, harness-specific adapter.


TL;DR


  • Size: LoRA rank 32 (alpha 64) on Qwen3.8-27B, a 27B dense hybrid with 262,144 native context. The merged BF16 checkpoint is about 54 GB.
  • Runs on: vLLM with LoRA enabled, or llama.cpp with a GGUF adapter on a Q8_0 base. Inference VRAM is not disclosed. Training used 2x NVIDIA B200.
  • Performance: Large pass@1 gains inside pi with thinking off, plus far shorter sessions.
  • Best: 64.7% on SWE-bench Pro (hard 51), up from 39.2%.
  • Worst: 55.8% on Terminal-Bench 2.1. The base model with thinking already reaches 64.7% on SWE-bench Pro.
  • Bottom line: Best case is thinking-level accuracy at about 4.5x fewer tokens. Caveats: small eval sets, self-reported results, pi only.

What Is Qwen3.8-27B-pi-adapted?


Qwen3.8-27B-pi-adapted is a LoRA adapter trained via reinforcement learning to work inside the pi coding agent. It is not a general-purpose chat fine-tune. The adapter is designed to be layered onto the open Qwen3.8-27B base model, giving developers a lightweight way to specialize a local coding agent without retraining or replacing the base model.


Performance Highlights


Inside pi with thinking disabled, the adapter delivers large pass@1 gains and substantially shorter sessions. Its strongest result is 64.7% on SWE-bench Pro (hard 51), up from 39.2% for the baseline. Its weakest result is 55.8% on Terminal-Bench 2.1. Notably, the base model with thinking enabled already reaches 64.7% on SWE-bench Pro, so the adapter's advantage is efficiency: it approaches thinking-level accuracy while using roughly 4.5x fewer tokens.


Caveats


Results are self-reported and based on small evaluation sets. The adapter is scoped to the pi coding agent, so gains may not transfer to other harnesses.

via MarkTechPost

Related