Open-Set Cattle Muzzle Identification: A Leakage-Controlled Benchmark and Evaluation Protocol
Submitted on 22 Aug 2026
Authors: Lalit BC, Dharmendra Singh Chaudhary, Shovit Nepal
Abstract
Reliable individual cattle identification is essential for disease surveillance, vaccination record-keeping, breeding management, and livestock insurance. While the bovine muzzle offers a stable and non-contact biometric, existing recognition systems predominantly assume a closed set of enrolled animals, which restricts their practical deployment. In this work, we reformulate cattle muzzle biometrics as an open-set, gallery-based identification problem that can reject previously unseen animals and support incremental enrollment—without requiring model retraining.
We introduce a leakage-controlled evaluation protocol based on identity-disjoint splits, per-fold retraining, held-out threshold calibration, verified duplicate removal, and bootstrap confidence intervals. The framework is assessed using two contrasting embedding configurations: a hybrid CNN-ViT metric-learning model and the MegaDescriptor-L foundation model. Under oracle threshold selection, the hybrid model achieves detection-and-identification rates of 98.3%, 96.4%, and 93.6% at target false-acceptance rates of 10⁻¹, 10⁻², and 10⁻³, respectively. MegaDescriptor-L attains corresponding rates of 99.3%, 98.1%, and 96.1%.
However, deployable threshold calibration reveals a significant gap between oracle and calibrated performance: the hybrid model records a false-acceptance rate of 1.03% at a 1% target, whereas MegaDescriptor-L reaches 2.44%. Incremental enrollment achieves Rank-1 accuracy above 91% with a single reference image and up to 97.3% with eight reference images—without retraining the model or degrading the existing gallery. These findings underscore that threshold calibration, leakage control, and embedding quality are critical for reliable open-set cattle identification, and we provide a practical evaluation framework for deployment-oriented animal biometric systems.
1. Introduction
The ability to accurately identify individual cattle underpins modern livestock management, from disease tracking to insurance claims. Traditional methods such as ear tags or RFID chips are invasive, prone to loss, and can cause discomfort. Biometric identification, using the unique pattern of a cow’s muzzle, offers a non-contact alternative that remains stable over time. However, most existing systems assume a closed set of enrolled animals, meaning they cannot handle new individuals that appear after deployment—a common scenario in real-world farms where animals are added or removed frequently.
2. Problem Definition
We reframe cattle muzzle identification as an open-set, gallery-based problem. In this paradigm, a system must (1) recognize animals that are present in its gallery (the enrolled set), and (2) reject animals that are not enrolled, flagging them as unknown. Additionally, the system should support incremental enrollment, allowing new animals to be added to the gallery without retraining the underlying model. This is a more realistic and scalable approach for commercial and agricultural settings.
3. Evaluation Protocol and Benchmark
To ensure fair and replicable evaluation, we propose a leakage-controlled protocol that addresses common pitfalls in biometric benchmarking:
- Identity-Disjoint Splits: Train, validation, and test sets contain disjoint identities, preventing information leakage that could inflate accuracy.
- Per-Fold Retraining: Each cross-validation fold retrains the model from scratch, ensuring no accidental data reuse.
- Held-Out Threshold Calibration: Thresholds for acceptance/rejection are tuned on a separate validation set, never on the test set.
- Verified Duplicate Removal: Duplicate images of the same animal are removed across splits to avoid bias.
- Bootstrap Confidence Intervals: Reported metrics include confidence intervals derived from bootstrap resampling for statistical robustness.
This protocol is designed to be a benchmark for future research in animal biometrics, promoting transparency and reproducibility.
4. Method
We explore two embedding configurations:
- Hybrid CNN-ViT Metric Learning: A combination of convolutional neural networks (CNNs) for local feature extraction and vision transformers (ViTs) for global context, trained with a metric-learning objective (e.g., triplet loss or ArcFace).
- MegaDescriptor-L Foundation Model: A large-scale pretrained model, adapted for cattle muzzle embeddings via fine-tuning on the available data.
- Hybrid model: 1.03% false-acceptance rate (FAR) at a 1% target FAR.
- MegaDescriptor-L: 2.44% FAR at the same target.
- 91%+ with a single reference image.
- 97.3% with eight reference images.
- Open-set recognition is feasible for livestock biometrics, with high detection and identification rates.
- Threshold calibration is crucial—oracle performance is not representative of deployment.
- Embedding quality significantly impacts generalization and robustness.
- Incremental enrollment works effectively, making the system practical for dynamic farm environments.
Both models generate embeddings that are compared using a similarity metric (e.g., cosine distance), with a decision threshold determining accept/reject.
5. Results
5.1 Oracle vs. Calibrated Performance
Under ideal (oracle) threshold selection, both models perform remarkably well. However, in practice, thresholds must be set using only validation data, which leads to a performance drop:
This discrepancy highlights the importance of robust threshold calibration methods, as naive oracle tuning can overestimate real-world performance.
5.2 Incremental Enrollment
Adding new animals to the gallery without retraining yields Rank-1 accuracies of:
Notably, this does not degrade performance on previously enrolled animals, ensuring long-term system stability.
6. Discussion and Conclusion
Our work introduces a rigorous, leakage-controlled evaluation framework for open-set cattle muzzle identification. The results demonstrate that:
As 2026 approaches, the livestock industry increasingly adopts AI-driven tools, and our framework provides a solid foundation for building trustworthy identification systems. Future work may extend this protocol to other animal species, incorporate temperature or pose variation, and explore self-supervised learning to reduce reliance on labeled data.
Cite as: arXiv:2608.28663 [cs.CV]
DOI: https://doi.org/10.48550/arXiv.2608.28663
via ArXiv CV