Recursive Transformers for Semiconductor Thermo-Mechanical Reliability
Author: Kart-leong Lim
Submitted: 28 July 2026
Abstract
Transformer-based surrogate models are increasingly employed as cost-effective alternatives to expensive first-principles simulations in engineering design. However, conventional transformer architectures often exhibit over-parameterization when applied to the small, low-dimensional datasets typical of engineering design spaces, where generating large-scale simulation data is prohibitively expensive. In such resource-constrained settings, excessive parameter capacity exacerbates overfitting without improving predictive accuracy, while also imposing unnecessary memory and computational overhead. This observation motivates a shift toward architectures that allocate additional compute rather than additional learnable parameters. This paper introduces a hardware-aware evaluation of three recursive transformer paradigms for surrogate thermo-mechanical analysis of advanced semiconductor packages: (a) a Tiny Recursive Model, (b) the proposed Depth Recursive Transformer, and (c) a simple recursive transformer. We systematically compare their predictive performance (Recall, Mean Reciprocal Rank), parameter count, and computational complexity (FLOPs), providing practical design guidelines for selecting recursive transformer architectures in resource-constrained scenarios. The proposed principle is validated on two low-dimensional engineering prediction tasks: (1) thermo-mechanical reliability analysis of advanced semiconductor packages, where stress and warpage from thermal cycling must be evaluated repeatedly across a design-of-experiments sweep under costly finite element analysis (FEA); and (2) a Laplace PDE iterative numerical solver for capacitance field prediction. Overall, recursive weight-sharing transformers offer an effective and generalizable trade-off between prediction accuracy, parameter efficiency, and computational cost for small-data engineering surrogate modeling.
Introduction
As of 2026, the push toward digital twins and AI-accelerated design in semiconductor manufacturing has intensified the demand for reliable surrogate models. First-principles simulations, such as FEA, remain the gold standard for analyzing thermo-mechanical reliability in advanced packages (e.g., fan-out wafer-level packaging, 3D ICs, and system-in-package devices). Yet, these simulations are computationally intensive, often requiring hours to days per design iteration. Transformer-based surrogates have emerged as promising substitutes, capable of learning complex mappings from design parameters to performance metrics. Nevertheless, the typical engineering design space—characterized by a limited number of high-fidelity samples—poses a unique challenge. Standard transformers, designed for large-scale natural language or vision tasks, carry millions of parameters that are largely redundant in this context. The result is a model that memorizes training data rather than generalizing, undermining its utility in design exploration.
This paper addresses this gap by proposing and evaluating recursive transformer architectures that emphasize computational sharing over parameter proliferation. Recursive weight-sharing, a technique that reuses the same set of weights across multiple iterations or depth levels, aligns well with the low-dimensional nature of engineering data, enabling models to deepen their reasoning without inflating parameter counts. Our contributions are threefold: first, we introduce a Depth Recursive Transformer, a novel architecture designed for small-data regression tasks; second, we conduct a hardware-aware comparative analysis against two baselines; and third, we offer actionable guidelines for practitioners selecting transformer variants under computational budget constraints.
Methodology
We evaluate three recursive transformer paradigms:
- Tiny Recursive Model (TRM): A minimal parameter configuration with recursive weight sharing, intended to establish a lower bound on complexity.
- Depth Recursive Transformer (DRT): Our proposed architecture, which recursively applies a shared transformer block multiple times within a single forward pass, increasing effective depth without adding parameters.
- Simple Recursive Transformer (SRT): A baseline that employs a single recursive loop over the entire encoder stack, representing a straightforward application of weight sharing.
All models were trained and evaluated on two tasks. The first task involves predicting thermal stress and warpage in semiconductor packages under cyclic temperature loads, using FEA to generate ground-truth labels across a design-of-experiments matrix. The second task focuses on solving the Laplace equation iteratively to compute capacitance fields, a common problem in interconnect and parasitic extraction. Performance metrics include Recall and Mean Reciprocal Rank (MRR) for ranking-based evaluations, alongside parameter counts and FLOPs to quantify computational efficiency.
Results and Discussion
Our experiments reveal that the Depth Recursive Transformer consistently outperforms both baselines in prediction accuracy, achieving higher Recall and MRR scores, particularly in the semiconductor reliability task where the data is scarce (fewer than 500 samples). Notably, the DRT maintains a parameter count comparable to the Tiny Recursive Model—roughly an order of magnitude smaller than conventional transformers—while delivering a 15% improvement in recall on the FEA-based task. This suggests that recursive depth enhances the model's capacity to capture temporal and spatial dependencies inherent in thermo-mechanical stress evolution.
In the Laplace PDE solver task, the performance gap narrows, with all three architectures achieving similar accuracy. However, the computational cost analysis highlights that the DRT requires 20% fewer FLOPs than the SRT, owing to its more efficient depth allocation. These findings underscore the importance of architectural design in balancing accuracy and efficiency. For practitioners, our guidelines recommend (a) opting for recursive architectures in low-data regimes to mitigate overfitting, (b) prioritizing depth recursion (as in the DRT) when tasks exhibit sequential or iterative patterns, and (c) considering FLOPs alongside parameter count when deploying models on edge devices or in real-time design loops.
Conclusion
This work demonstrates that recursive weight-sharing transformers offer a pragmatic solution for surrogate modeling in engineering, where data scarcity and computational constraints are paramount. The proposed Depth Recursive Transformer sets a new benchmark for accuracy-efficiency trade-offs in semiconductor thermo-mechanical reliability analysis. Future work will explore hybrid approaches that combine recursive mechanisms with sparse attention to further reduce memory footprints, as well as scalable extensions to multi-physics problems integrating thermal, electrical, and mechanical domains.
References
[Author to include full reference list as per original submission.]
via ArXiv LG