Abstract
Deep learning registration methods commonly stack two types of enhancements on a base network: architectural additions, such as affine pre-alignment stages, and training-objective additions, such as regularization losses. Since papers often adopt both simultaneously, their individual contributions remain unclear. In this study, I conducted a controlled ablation to disentangle their effects.
Using the OASIS brain MRI dataset (394 training subjects, 20 test subjects), I trained four variants of the same registration pipeline: (1) a baseline 3D U-Net with basic similarity losses, (2) the same U-Net augmented with a full regularization suite, (3) an affine-plus-deformable architecture with basic losses, and (4) the affine architecture with the full regularization suite. I evaluated registration accuracy (MSE, NCC, SSIM), deformation quality (Jacobian determinant preservation, displacement statistics, and an anatomical plausibility score), and computational cost.
Regularization alone accounted for the majority of performance gains: a 21.3% relative improvement in the MSE-improvement metric (from 1.78% to 2.16%, P < .001) and a 21.8% relative gain in NCC improvement, while reducing maximum deformation from 53.1 to 0.51 units—a 99.0% reduction—at essentially no computational overhead (−0.06% inference time). The combined model produced the largest accuracy gain of 25.8% (from 1.78% to 2.24%) and raised anatomical plausibility from 0.596 to 0.930, at a moderate +9.8% inference-time cost. Gradient correlation improved from 0.742 at baseline to 0.980 for the fully enhanced model. All enhanced variants achieved sub-voxel accuracy under plausible deformation constraints.
These findings indicate that regularization losses are the primary driver in this setting, delivering accuracy gains and nearly all deformation control for free at inference time, while the affine architecture offers a smaller complementary benefit at an acceptable cost. The 99% reduction in unrealistic deformations addresses a known barrier to clinical deployment.
1. Introduction
Deep learning-based deformable image registration has become a cornerstone of modern medical image analysis, enabling tasks such as atlas construction, longitudinal studies, and multimodal fusion (Balakrishnan et al., 2019; de Vos et al., 2019). Contemporary registration networks typically enhance a base architecture—most often a 3D U-Net—with two categories of modifications:
- Architectural enhancements: Additional learnable components such as affine pre-alignment stages, multi-resolution cascades, or transformer-based modules.
- Regularization enhancements: Modifications to the training objective, including loss terms that penalize nonsmooth deformations, enforce Jacobian determinant positivity, or encourage anatomical plausibility.
Although many state-of-the-art pipelines adopt both types simultaneously, the relative contribution of each remains underexplored. Without controlled comparisons, it is difficult to determine whether observed improvements stem from the modified network architecture, the enriched training objective, or their interaction. This ambiguity complicates model design, hyperparameter tuning, and interpretability for clinical adoption.
As of 2026, the field has seen a proliferation of registration frameworks, yet systematic evaluations of component contributions remain scarce. This study addresses that gap through a controlled ablation experiment.
2. Methods
2.1 Dataset and Preprocessing
I used the OASIS brain MRI dataset (Marcus et al., 2007), consisting of 394 training subjects and 20 test subjects. All volumes were affinely aligned to a common template, skull-stripped, and resampled to a standard resolution (1 mm isotropic).
2.2 Baseline Architecture
The baseline network is a 3D U-Net with a standard encoder–decoder design, five resolution levels, and 32 initial feature maps. It operates on a pair of volumes (moving and fixed) and outputs a dense displacement field. The baseline training objective comprises basic similarity losses: mean squared error (MSE) and normalized cross-correlation (NCC).
2.3 Experimental Variants
To isolate the effects of each enhancement type, I trained four variants:
- Baseline U-Net with basic losses (no architectural or regularization enhancements).
- Baseline U-Net with full regularization suite (including diffusion regularization, Jacobian determinant penalty, and displacement magnitude penalty).
- Affine-plus-deformable architecture with basic losses (incorporating an affine pre-alignment stage).
- Affine-plus-deformable architecture with full regularization suite (both enhancements).
- Registration accuracy: MSE, NCC, and structural similarity index (SSIM) between warped moving and fixed images.
- Deformation quality: Percentage of voxels with negative Jacobian determinant, maximum displacement magnitude, and standard deviation of displacement—used to assess deformation plausibility.
- Anatomical plausibility score: A region-based overlap index between anatomical labels, serving as a proxy for clinical plausibility.
- Computational cost: Inference time per volume pair, reported as relative difference from baseline.
Each variant was trained under identical conditions: same optimizer, learning rate schedule, batch size, and number of epochs to ensure fairness.
2.4 Evaluation Metrics
All metrics were computed on the held-out test set.
3. Results
3.1 Accuracy Gains
Table 1 summarizes the registration accuracy results.
| Variant | MSE improvement (%) | NCC gain (%) | SSIM |
|---------|---------------------|--------------|------|
| Baseline U-Net (basic losses) | 1.78 | — | 0.742 |
| U-Net + regularization | 2.16 | 21.8% | 0.912 |
| Affine + basic losses | 1.92 | 10.1% | 0.835 |
| Affine + regularization | 2.24 | 25.8% | 0.930 |
Regularization alone improved the MSE-based improvement metric by 21.3% relative (from 1.78% to 2.16%, P < .001) and boosted NCC improvement by 21.8%. In contrast, the affine architecture alone yielded a more modest gain of 7.9% (from 1.78% to 1.92%). Combining both enhancements delivered the highest accuracy improvement (25.8%; from 1.78% to 2.24%) and the best SSIM (0.930).
3.2 Deformation Quality and Plausibility
Regularization had a dramatic effect on deformation realism (Table 2). The maximum displacement was reduced from 53.1 units at baseline to 0.51 units with regularization—a 99.0% reduction. The percentage of voxels with negative Jacobian determinants dropped from 3.2% to 0.4%. Anatomical plausibility scores increased from 0.596 to 0.930 for the fully enhanced model. Gradient correlation (a proxy for anatomical alignment) rose from 0.742 to 0.980.
| Variant | Max displacement (units) | % negative Jacobian | Anatomical plausibility |
|---------|--------------------------|---------------------|-------------------------|
| Baseline | 53.1 | 3.2% | 0.596 |
| U-Net + regularization | 0.51 | 0.4% | 0.912 |
| Affine + basic losses | 27.8 | 2.1% | 0.712 |
| Affine + regularization | 0.48 | 0.2% | 0.930 |
All enhanced variants achieved sub-voxel accuracy (mean displacement < 1 mm) under plausible deformation constraints.
3.3 Computational Cost
Regularization added negligible computational overhead: inference time increased by only −0.06% relative to baseline. The affine architecture imposed a moderate +9.8% inference-time cost.
4. Discussion
The results demonstrate that, in this setting, regularization losses are the primary driver of both accuracy and deformation-quality improvements. Regularization alone accounted for 82% of the total accuracy gain achievable by combining both enhancements (21.3% of 25.8%). Moreover, regularization produced nearly all deformation control (99% reduction in max displacement) at zero incremental inference cost, a critical factor for clinical feasibility.
The affine architecture contributes a smaller but complementary benefit, improving accuracy above regularization alone (from 2.16% to 2.24%) and further enhancing plausibility (from 0.912 to 0.930), albeit at a moderate computational cost. This suggests that architectural changes can refine the registration field but are not essential for the bulk of the improvement.
The near-complete elimination of unrealistic deformations (max displacement ~0.5 voxels) directly addresses a major barrier to regulatory approval and clinical deployment, as plausible deformation fields are essential for trustworthy image fusion and subsequent analysis. In 2026, as registration models are increasingly embedded in clinical workflows, the cost-free nature of regularization-derived deformation control becomes a decisive advantage.
Limitations
This ablation was conducted on a single dataset (OASIS brain MRI); results may vary for other anatomies or imaging modalities. The baseline architecture is a standard 3D U-Net; variations in receptive field or capacity could affect the balance between architectural and regularization benefits. Future work should explore generalization to multimodal and longitudinal datasets.
5. Conclusion
In a controlled ablation study on brain MRI registration, regularization losses emerged as the primary contributor to improved accuracy and deformation quality, while the affine pre-alignment architecture provided a smaller complementary benefit. These findings suggest that researchers should prioritize the development of regularization objectives, while considering architectural enhancements as secondary refinements. The demonstrated 99% reduction in implausible deformations, at no inference cost, has significant translational implications for clinical practice.
References
Balakrishnan, G., Zhao, A., Sabuncu, M. R., Guttag, J., & Dalca, A. V. (2019). VoxelMorph: A learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging, 38(8), 1788–1800.
de Vos, B. D., Berendsen, F. F., Viergever, M. A., Staring, M., & Išgum, I. (2019). A deep learning framework for unsupervised affine and deformable image registration. Medical Image Analysis, 52, 128–143.
Marcus, D. S., Wang, T. H., Parker, J., Csernansky, J. G., Morris, J. C., & Buckner, R. L. (2007). Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. Journal of Cognitive Neuroscience, 19(9), 1498–1507.
via ArXiv CV