On the Depth Scalability of Logic Gate Networks
Authors: Taegun An, Dohun Kim, Haebeom Lee, Changhee Joo
Submitted: 19 July 2026
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
arXiv ID: 2607.21633 (v1)
DOI: https://doi.org/10.48550/arXiv.2607.21633
Abstract
Logic Gate Networks (LGNs) implement computation through compositions of Boolean operations. However, unlike classical Boolean circuits, existing LGNs do not reliably benefit from increased depth. This paper identifies two distinct causes for this limitation: optimization collapse in deep relaxed LGNs and a topology-induced limitation that persists even when skip-biased initialization and straight-through estimation stabilize training. These findings indicate that trainability alone is insufficient; deeper layers must also receive information that supports useful computation.
To address this, we introduce Input-Anchored Logic Gate Networks (IALGNs), where each gate combines an evolving hidden feature with a direct input anchor. This topology preserves a computational spine while conditioning every layer on the original input. We demonstrate that a depth-\(D\) path can depend on up to \(D+1\) input bits, establishing a strict path-wise depth hierarchy. A Random-k anchor relaxation further improves anchor selection without relaxing the spine.
Across MNIST, CIFAR-10, and CIFAR-100 benchmarks, IALGNs achieve consistent fixed-width depth–accuracy improvements beyond 100 layers. In contrast, alternative LGN topologies either saturate or degrade in performance at comparable depths. Layer-wise probes, topology ablations, and effective-depth analysis reveal that input anchoring produces progressively more informative representations and preserves longer computational paths. These results demonstrate that scalable depth in LGNs requires both stable optimization and an information-access pattern that supports input-conditioned refinement.
Key Contributions
- Identification of root causes for depth scaling limitations in LGNs, distinguishing between optimization collapse and topology-induced constraints.
- Introduction of IALGNs, a novel architecture that ensures every layer retains direct access to input features, enabling deeper networks.
- Theoretical proof of a strict path-wise depth hierarchy, showing that depth-\(D\) paths can access up to \(D+1\) input bits.
- Empirical validation across three standard datasets, demonstrating consistent improvements beyond 100 layers.
Relevance to 2026 AI Landscape
As of 2026, the drive toward deeper and more efficient neural architectures remains central to advancements in edge AI, neuromorphic computing, and explainable models. Input-anchored designs offer a promising direction for scaling Boolean-style networks without sacrificing interpretability or computational efficiency—key considerations for next-generation hardware and low-power applications.
Paper Details
- Main text: 7 pages
- Figures: 4
- Tables: 2
For citation, please use the arXiv ID and DOI above.
via ArXiv LG