BlockLock: Reproducible LLM Inference with Fixed Batch Geometry

Authors: Xiuyuan Qi, Pedro Reviriego, Javier Conde, Shanshan Liu

Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Cite as: arXiv:2611.06977 [cs.AI] — https://doi.org/10.48550/arXiv.2611.06977

Abstract

Reproducible LLM inference requires that a request produce the same output regardless of the other requests in its batch. Existing solutions pay for this guarantee in different ways: batch-invariant kernels constrain every operator, while isolating prefill with fixed-shape decode (the CoRun-style baseline) sacrifices shared prefill compute and pads the decode phase.

We present BlockLock, which retains shared batched prefill and pins only the numerical path of the protected request. Specifically, it fixes the geometry that the request observes, operating in one of two modes:

• BlockLock-Rows uses fixed-width rows and retains batched decode across all requests.
• BlockLock-Pack uses terminal-anchored packing with an isolated dummy segment, then extracts the protected request's KV state for target-only decode.

Under a stated per-deployment contract, both modes preserve the target's output bitwise when its companion requests change.

Evaluation

We evaluate BlockLock across five dense models—Qwen3-8B/14B, Llama-3.1-8B-Instruct, and Gemma-3-4B/12B-IT—and three single-GPU deployments (NVIDIA A6000, A100, and H100), using both tinyBenchmarks and BFCL. We also include initial tensor-parallel experiments on two NVIDIA A6000 GPUs. In the reported tests, both modes preserve target sequences and Top-20 logit bits.

Results

At equal delivered work, BlockLock-Rows reduces latency by 53.5–55.0% and improves throughput by 115.2–122.2% relative to vLLM Batch-Invariant. Compared with batch-invariant kernels, it requires no custom kernel implementations and constrains only the protected requests.

Key Takeaways

• Reproducibility without global constraints: BlockLock guarantees bitwise-identical output for a protected request even as its batch companions change, without forcing batch invariance on every operator.
• Two flexible modes: BlockLock-Rows preserves full batched decode; BlockLock-Pack isolates the protected request through terminal-anchored packing and KV-state extraction.
• Substantial efficiency gains: Relative to vLLM Batch-Invariant, BlockLock-Rows cuts latency by more than half and more than doubles throughput at equal delivered work.
• No custom kernels required: Unlike batch-invariant kernel approaches, BlockLock constrains only the protected requests and needs no bespoke kernel implementations.

Metadata

• Submission history: Submitted on 4 November 2026
• arXiv identifier: arXiv:2611.06977v1 [cs.AI]
• DOI: https://doi.org/10.48550/arXiv.2611.06977 (arXiv-issued DOI via DataCite)

via ArXiv AI