Guardrails
Also called AI guardrails, LLM guardrails
AI guardrails are the controls placed around an AI model to keep its inputs, outputs and actions within defined limits. They include filters that block harmful prompts, checks that stop personal data or made-up facts reaching users, and limits on what an AI agent can do without human approval. Guardrails sit outside the model itself.
How it works
A guardrail layer checks traffic in both directions. Input guardrails screen user prompts and retrieved documents for prompt injection, banned topics and personal data before they reach the model. Output guardrails check responses against policy: they block toxic or discriminatory content, redact customer data, verify that answers are grounded in approved sources and stop the model giving regulated advice it is not permitted to give. Failed checks return a safe fallback or route to a person.
For AI agents, guardrails also restrict actions. The agent can call only approved tools, within set limits, and needs human approval for payments, account changes or anything irreversible. Every blocked or flagged event is logged for monitoring and audit. In a bank, guardrails are part of the wider model risk framework: they are designed with the use case, tested through red teaming before launch and reviewed as the model, prompts and threats change.
The rules
Requires high-risk AI systems to achieve appropriate accuracy, robustness and cybersecurity, including resilience against attempts to manipulate inputs or exploit vulnerabilities.
Providers of high-risk AI systems, including credit scoring systemsRequires model evaluation including adversarial testing, assessment and mitigation of systemic risks, serious incident reporting and adequate cybersecurity protection.
Providers of general-purpose AI models with systemic riskIdentifies risks specific to generative AI, such as confabulation, harmful content and information security, and sets out actions to govern, map, measure and manage them.
Voluntary, for any organisation developing or using generative AICommon questions
What is the difference between guardrails and alignment?
Alignment shapes how a model behaves during training. Guardrails are separate controls applied at run time around the model. Firms using third-party models cannot change their training, so guardrails are the main control they own.
Do guardrails stop hallucinations?
They reduce the risk. Grounding checks compare an answer with approved source documents and block or flag claims that are not supported. They do not make a model accurate on their own, so human review stays in place for high-impact uses.
Are AI guardrails required by law?
No law uses the word. The EU AI Act requires robustness, accuracy and human oversight for high-risk systems and risk mitigation for powerful general-purpose models, and guardrails are a common way firms meet those duties.
