LLM Guardrails Evaluation Toolkit
By Ahmet Göksu, Ayodele Daniel Abraham, Kisejjere Rashid, Muhammad Abdullah Said, Niamh Maher, Onurcan Genç, Parivrudh Sharma, Shalina Prakash, Suha Abualoul
October 2026
Testing whether content-safety guardrails actually prevent RAG-specific failures, or merely flag the content around them.
Advisor & Editor
Authors
Ahmet Göksu
Research Fellow
Ayodele Daniel Abraham
Research Fellow
Kisejjere Rashid
Research Fellow
Muhammad Abdullah Said, M.Sc.
Research Fellow
Niamh Maher, Ph.D.
Research Fellow
Onurcan Genç
Research Fellow
Parivrudh Sharma, M.Sc.
Research Fellow
Shalina Prakash
Research Fellow
Suha Abualoul
Research Fellow
Advisors & Mentors
Advisor & Editor
Originally Published
OCTOBER 2026
Research Overview
Can a guardrail prevent the failure—or only flag the content around it?
As organisations connect large language models to internal documents and knowledge bases, general-purpose safety guardrails are increasingly expected to inspect user prompts, retrieved content and generated answers.
Yet a “safe” or “unsafe” verdict does not, on its own, demonstrate that an attack was prevented. A guardrail may react to unrelated content, miss the actual failure mechanism or reduce the quality of legitimate answers.
This study introduces GuardEval: a shared evaluation harness that records retrieval behaviour, guardrail verdicts and final responses under controlled conditions. Rather than collapsing different experiments into a single benchmark score, each failure mode is evaluated within its own dataset, guard placement, truth source and outcome measures.
Research at a Glance
A structured evaluation of RAG-specific guardrail behaviour
Research Streams
Semantic jamming, sensitive disclosure, prompt injection, and gender-paired verdict stability.
Risk Frameworks
Mapped to OWASP’s Top 10 for LLM Applications, the NIST AI RMF and the MIT AI Risk Repository.
Headline Metrics
Attack success, over-blocking, refusal correctness and paired flip rate.
Shared Harness
GuardEval brings retrieval evidence, guard verdicts and model responses into one workflow.
Key Findings
What the experiments revealed
The results should be interpreted within each research stream rather than combined into a single performance score.
Semantic Jamming
The adversarial blocker entered the top-five retrieved passages in 70 of 100 cases. None was flagged by the tested chunk policies, and the jam rate remained 21/100 in every condition.
Sensitive Disclosure
Unguarded leakage increased from 2/60 generic prompts to 8/12 prompts in each named-question framing. A custom category reduced leakage; tested access control prevented unauthorised retrieval.
Direct Prompt Injection
The strongest tested query-placement configuration reduced attack success from 32/50 to 21/50, while also blocking 2/25 benign hard negatives.
Indirect Prompt Injection
Literal matching recorded no successful format variants, but manual review found five encoded-format compliances. Chunk filtering also reduced clean-task accuracy from 63/81 to 58/81.
Gender-Paired Stability
Six gender-swapped pairs received different guard verdicts. The result indicates instability in this sample, but does not establish a systematic directional gender gap.
Cross-Cutting Finding
A safety flag does not prove that the relevant failure was detected or prevented. Filtering can also impose a measurable accuracy cost on legitimate tasks.
How GuardEval Works
From a failure mode to actionable evidence
The shared workflow keeps the evidence traceable while allowing each stream to use the placement and truth source appropriate to its research question.
Define the Failure Mode
Select the risk and establish the relevant adversarial cases, clean twins or benign controls.
Establish the Baseline
Measure how the RAG pipeline behaves before a guardrail is introduced.
Select Guard Placement
Test the query, retrieved-chunk or generated-output stage according to the mechanism under study.
Record the Full Case
Capture retrieved documents, guard decisions, filtered content and final model responses.
Score Security and Utility
Measure attack outcomes alongside over-blocking, refusals, answer accuracy and paired changes.
Map Evidence to Controls
Translate the findings into preventive, detective and corrective actions with accountable owners.
Interactive Toolkit
Move from reading the findings to evaluating your own system.
GuardEval is designed as a deployment-specific starting point. Teams can adapt the test cases, substitute their own prompts and corpora, compare unguarded and guarded conditions, and inspect retrieved evidence alongside final answers.
- Compare guard placements and policies
- Evaluate security and utility together
- Inspect cases automated judges may miss
- Test custom guard categories
- Generate governance evidence
- Track tested and proposed controls
Research Notes
Scope, limitations and citation
Use the expandable sections below for responsible interpretation and attribution.
Scope and Limitations
This is an exploratory, stream-specific evaluation rather than a universal benchmark. The experiments primarily use Llama Guard 3 and Qwen3.5:9b, limited datasets and configurations, and generally single experimental runs.
The indirect prompt-injection variants derive from 16 underlying items, and the gender-paired evaluation contains 50 matched pairs. The findings should not be generalised to other guards, generators, corpora or production environments without further testing.
GuardEval provides a structured starting point for deployment-specific evaluation; it is not a certification that a guardrail or RAG system is universally safe.
Suggested Citation and License
Licensed under the Creative Commons Attribution–NonCommercial 4.0 International License (CC BY-NC 4.0).