Home / Current Issue / Paper 1722808
Detecting Prompt Injection in Retrieval-Augmented Generation Pipelines Using Lightweight Classifiers
Subject area: Science,Engineering and Technology · Area of research: Detecting Prompt Injection
Abstract
Retrieval-Augmented Generation (RAG) grounds large language model (LLM) outputs in externally retrieved documents, but this same mechanism creates a novel attack surface: adversarial instructions embedded inside retrieved content can hijack the downstream model's behavior, a threat known as indirect prompt injection. Unlike classical adversarial machine learning, which perturbs input features to evade a classifier, indirect prompt injection exploits the inability of an LLM to structurally distinguish trusted developer instructions from untrusted retrieved data. This paper proposes and evaluates a lightweight, model-agnostic defense: a supervised classifier that screens retrieved passages for injected instructions before they reach the generation model's context window. We construct a synthetic, multi-domain RAG corpus spanning banking, healthcare, life sciences, general technology, and customer-support retrieval scenarios, and train three lightweight classifiers — logistic regression, a calibrated linear support vector machine, and a random forest — on TF-IDF features. On a held-out test set the best classifier achieves near-perfect detection (F1 = 1.000, AUC = 1.000). Critically, we then evaluate generalization against an adaptive-obfuscation test set using homoglyph substitution, payload splitting, and unseen synonym paraphrasing not present in training; recall degrades to 89.5–92.0% while precision remains at 100%, revealing an asymmetric robustness gap between naive and adaptive attackers. We further quantify the false-positive-rate/usability tradeoff via a decision-threshold sweep, showing how operators can tune the detector for high-recall security-critical deployments (e.g., banking transaction assistants) versus low-friction deployments (e.g., customer support). We discuss deployment implications for banking, healthcare, and life-sciences RAG systems, where both missed injections and false blocks carry asymmetric operational costs, and outline future work on adversarially robust training and cross-lingual obfuscation.
Keywords
Prompt injection, retrieval-augmented generation, large language models, adversarial machine learning, intrusion detection, TF-IDF, lightweight classifiers, LLM security, AI safety, banking security, healthcare AI, life sciences AI.
How to cite this paper
@article{1722808,
author = {Srikumar Nayak, Tanya Jane},
title = {Detecting Prompt Injection in Retrieval-Augmented Generation Pipelines Using Lightweight Classifiers},
journal = {Iconic Research And Engineering Journals},
year = {2024},
volume = {8},
number = {2},
pages = {1428-1437},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1722808.pdf},
abstract = {Retrieval-Augmented Generation (RAG) grounds large language model (LLM) outputs in externally retrieved documents, but this same mechanism creates a novel attack surface: adversarial instructions embedded inside retrieved content can hijack the downstream model's behavior, a threat known as indirect prompt injection. Unlike classical adversarial machine learning, which perturbs input features to evade a classifier, indirect prompt injection exploits the inability of an LLM to structurally distinguish trusted developer instructions from untrusted retrieved data. This paper proposes and evaluates a lightweight, model-agnostic defense: a supervised classifier that screens retrieved passages for injected instructions before they reach the generation model's context window. We construct a synthetic, multi-domain RAG corpus spanning banking, healthcare, life sciences, general technology, and customer-support retrieval scenarios, and train three lightweight classifiers — logistic regression, a calibrated linear support vector machine, and a random forest — on TF-IDF features. On a held-out test set the best classifier achieves near-perfect detection (F1 = 1.000, AUC = 1.000). Critically, we then evaluate generalization against an adaptive-obfuscation test set using homoglyph substitution, payload splitting, and unseen synonym paraphrasing not present in training; recall degrades to 89.5–92.0% while precision remains at 100%, revealing an asymmetric robustness gap between naive and adaptive attackers. We further quantify the false-positive-rate/usability tradeoff via a decision-threshold sweep, showing how operators can tune the detector for high-recall security-critical deployments (e.g., banking transaction assistants) versus low-friction deployments (e.g., customer support). We discuss deployment implications for banking, healthcare, and life-sciences RAG systems, where both missed injections and false blocks carry asymmetric operational costs, and outline future work on adversarially robust training and cross-lingual obfuscation.},
keywords = {Prompt injection, retrieval-augmented generation, large language models, adversarial machine learning, intrusion detection, TF-IDF, lightweight classifiers, LLM security, AI safety, banking security, healthcare AI, life sciences AI.},
month = {August},
}