PromptSentinel: responsible AI prompt management

Status: presented at an IEEE Student Event at Panimalar Engineering College. The system is open source — the reference implementation has since been consolidated into LLM Guardian, which carries the keyword-policy, PII, toxicity and compliance layers this paper describes. Exact event date, certificate, and any proceedings link get added once I pull the files from my vault.

Every LLM application has the same quiet assumption baked in: the prompt that reaches the model is safe, appropriate, and belongs to the person who sent it. None of those are guaranteed.

The problem

Organizations adopting LLMs inherit whatever their users paste into the prompt box — employee PII, toxic content, injection attempts, and prompts that violate internal policy. Most applications treat this as the model's problem to handle gracefully, which is backwards: there is no centralized layer that screens a prompt before it reaches the model, redacts what must not leave, and keeps an auditable record of every decision.

PromptSentinel addresses that with a middleware service for responsible AI prompt management: real-time prompt compliance, PII redaction, toxicity detection, and responsible output validation — with full auditability.

The design

The middleware sits between users and the LLM endpoint and classifies every prompt into one of three verdicts:

• Safe — forwarded to the model
• Flagged — quarantined or redacted for review
• Blocked — rejected outright

The moderation pipeline behind that decision runs several checks in sequence:

1. Keyword policy checks — configurable lists of flagged and blocked terms, driven by policy rules
2. PII detection & redaction — Presidio NER finds names, emails, phones, and identifiers, and redacts them before anything leaves the perimeter
3. Toxicity scoring — Detoxify scores with configurable thresholds decide prompt acceptance
4. Audit logging — every decision, verdict, and redaction is written to SQLite with indexed lookups, exposed through a web dashboard for human review

Safe prompts are then forwarded to the model (an LLM API in the reference implementation) for the actual response.

User prompt
  → keyword policy checks
  → PII detection + redaction (Presidio NER)
  → toxicity scoring (Detoxify, configurable threshold)
  → classify: Safe / Flagged / Blocked
  → audit log (SQLite) + review dashboard
  → forward safe prompts to the LLM

Responsible AI, operationally

"Responsible AI" often dies at a principles document. PromptSentinel's contribution is making the principles operational and inspectable:

• Transparency — no silent decisions: every verdict and redaction is logged with a reason
• Human oversight — the dashboard turns the audit log into a review workflow, so edge cases are adjudicated by people, not assumed away
• Configurable thresholds — stricter policy profiles can be applied to sensitive deployments without code changes

It also carried the security posture of the platform itself: security headers, trusted-host checks, admin protection on sensitive endpoints, and tests that lock in the detection behavior.

Where it sits in my work

PromptSentinel is the research face of a line that runs through several builds: the LLM Security & Data Privacy Compliance Middleware (Round-2 project, Shentinelix), the PEC Techathon 3.0 Prompt Compliance Automation system, and the later MCP Guardian work (shipped since as LLM Guardian) — each iteration moving one step closer to the agent pipeline. Its PII + toxicity + policy screening is the same trio that guards user prompts in every one of them.

Status & next steps

Presented at the IEEE Student Event, Panimalar Engineering College. I'll expand this post with the presentation details, evaluation numbers, and the responsible-AI framework from the full paper once I recover it from the vault.