Automated Driving System Classification: A Sensitivity Analysis of GRU, LSTM, and Transformer Encoders
Authors: Bidhya Shrestha, Christos Papadopoulos
Submitted: July 25, 2026
Abstract
Automated driving systems (ADSs) are becoming increasingly prevalent. Future software-defined vehicles (SDVs) are expected to run multiple ADSs, both native and aftermarket, such as Comma.ai's Openpilot. Monitoring systems that independently verify which ADS is active are essential for safety monitoring, regulatory compliance, insurance assessment, and anomaly detection. In this paper, we evaluate the effectiveness of three sequence-based classification models—Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) networks, and a Transformer encoder—for identifying Level 2 automated driving systems using vehicle telematics data alone. The systems classified include Comma Openpilot, Tesla Autopilot, and Cadillac Super Cruise, alongside manual driving.
All three models achieve strong performance on clean data, with macro F1-scores of 0.92 (GRU), 0.90 (LSTM), and 0.93 (Transformer encoder). Threat-matched training yields macro F1-scores between 0.904 and 0.916, with only a modest penalty on clean-data performance. We also introduce a modular robustness evaluation framework that simulates realistic telematics degradation through five corruption families at five severity levels (L1–L5). Continuous channels are perturbed using additive white Gaussian noise with cumulative drift, correlated cross-channel noise, and temporal jitter. Binary event signals are subjected to burst loss, delayed transitions, spurious toggles, and cross-feature inconsistencies, inspired by communication errors.
Robustness is measured using macro-F1, which assigns equal weight to each class and is suitable for imbalanced multiclass evaluation. Our evaluation reveals a sharp failure-mode split: event-level corruptions reduce macro-F1 only slightly (≥0.87 at L5), while temporal jitter collapses macro-F1 to 0.44–0.50 across all three model architectures.
Keywords: automated driving systems; machine learning classification; GRU; LSTM; Transformer; sensitivity analysis; robustness evaluation; telematics data; temporal jitter
via ArXiv LG