Works · Framework Demo

Trustworthy Deployment of LLM-Based RAG Systems for Small Businesses

Treats trustworthiness as a system-level property that emerges from the interaction between security filtering, confidence handling, and downstream response control — not just a better model. A lightweight k3s-based pipeline layers prompt-injection defenses with calibrated confidence scores so resource-constrained SMEs can deploy customer-support RAG safely.

Preprint MIT code · CC BY 4.0 paper 3 LLM backends · 6 injection benchmarks
0.994
Best F1 (layered defense)
≈0.01
Calibration error (ECE)
Guard + GenTel-Shield
Layered injection defense

How it works

A five-stage pipeline, guarded at both ends

01

User query

Routed through a lightweight k3s edge cluster into the pipeline.

02

Injection guard

Structured prompt filtering plus a pre-trained GenTel-Shield detector screen the query.

03

RAG retrieval

Relevant business documents are retrieved for grounded generation.

04

Generation + calibration

The LLM answers with a structured JSON output carrying a calibrated confidence score.

05

Response control

Low-confidence or blocked queries are escalated or rejected instead of answered.

Worked example

Three requests through the pipeline

Illustrative example using representative data from the paper — not a live model call. Pick a scenario to see how the pipeline responds at each stage.

Incoming query
01–02 · Guard verdict

03 · Retrieved context


04 · Generated answer

Calibrated confidence

05 · Downstream action: