Securing Multimodal AI through Internal Information Decomposition

Authors: Jehyeok Yeon, Hyeonjeong Ha, Qiusi Zhan, Heng Ji

Submitted: May 3, 2026 (arXiv:2607.21600)

Accepted as Spotlight Paper at ICML 2026

Abstract

Multimodal large language models introduce attack surfaces that are absent in unimodal systems. Adversaries can distribute malicious intent across modalities to bypass safeguards that inspect each modality in isolation. This motivates the use of cross-modal consistency as a detection signal, rather than relying on isolated modality inspection.

Our key observation: benign inputs induce compatible predictive behavior from text-only and vision-only reasoning, which stabilizes when fused. In contrast, adversarial manipulation disrupts this consistency, resulting in abnormal multimodal behavior. Existing defenses that examine raw inputs or outputs overlook this internal fusion process, making them both brittle and computationally expensive.

We propose FlowGuard, a lightweight inference-time framework that detects harmful inputs by monitoring internal multimodal consistency. Unlike approaches that rely on scalar confidence metrics, FlowGuard derives FlowVectors—inspired by Partial Information Decomposition—to quantify cross-modal redundancy, synergy, and modality-specific dominance. These vectors capture whether fused multimodal predictions remain aligned with unimodal semantic evidence.

In a one-class classification setting trained solely on benign data, FlowGuard reduces Attack Success Rates from >90% to <15% on unseen attacks, with <3% utility loss and up to a 6× latency reduction. Our results demonstrate that monitoring cross-modal consistency offers an efficient and effective defense for multimodal reasoning—particularly relevant as multimodal AI systems become widespread in 2026 and beyond.

Keywords

Multimodal large language models, cross-modal consistency, adversarial defense, Partial Information Decomposition, FlowGuard, inference-time detection, 2026 AI security

Published on arXiv under arXiv:2607.21600 (cs.AI).

via ArXiv AI