Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B

LLMs 📅 2026-07-28 ⏱ 2 min read 👁 8 views ⭐ 7/10

arxiv 2607.22545 deberta-v3 financial services large language models machine learning prompt injection detection regulatory compliance safety classifier

Overview

Deploying large language models (LLMs) in financial-services and agentic environments demands safety classifiers that can simultaneously address prompt injection, regulatory compliance, and general harm. As of 2026, no existing open-source guardrail handles all three axes in a single inference pass. Semalith v1.4 fills this gap.

Model Architecture and Training

Semalith v1.4 is a compact 184M-parameter safety classifier built on DeBERTa-v3-base. It performs simultaneous three-axis classification—prompt injection, general harm, and financial-services regulatory compliance—in a single forward pass. The model features:

A 22-class head with labels: BENIGN, nine prompt-injection sub-types, general harm, and eleven BFSI (Banking, Financial Services, and Insurance) labels.
An auxiliary 4-class super-category head, trained under a jointly weighted loss.
Training data: a 76,204-row corpus mined from 49 public sources, with SHA-1 deduplication against every held-out evaluation set. Of 22 benchmarks, 21 show zero contamination; the maximum contamination is 0.22%.

Performance Benchmarks

Semalith v1.4 delivers state-of-the-art results against the much larger Llama-Guard-3-8B model (8B parameters, roughly 44× larger):

Prompt injection detection: Wins all 7 out of 7 evaluations.
Overall benchmarks: Wins 11 out of 18 evaluations.
False positive rate (FPR): Achieves FPR = 0.000 on 208 benign agentic prompts, compared to 0.063 for Llama-Guard-3-8B.

On general-harm benchmarks (WildGuardMix, HEx-PHI, HarmBench), Llama-Guard-3-8B leads; this complementary behavior is documented in Section 4 of the paper. Six measured weak spots are disclosed in Section 6.

Deployment Guidance

v1.3: Recommended for conversational moderation deployments (ToxicChat F1 score: 0.624).
v1.4: Recommended when BFSI label coverage or zero-FPR on benign agentic prompts is the priority.

Additional Information

Comments: 16 pages, 8 tables, no figures.
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR).
Cite as: arXiv:2607.22545 [cs.LG].
DOI: https://doi.org/10.48550/arXiv.2607.22545

via ArXiv LG