Explainability
Also called explainable AI, XAI, interpretability
Explainability is the ability to show, in terms a person can understand, why an AI or statistical model produced a given output. In financial services it lets firms justify credit decisions, alerts and risk scores to customers, auditors and supervisors. Several data protection, credit and AI laws require explanations of automated decisions.
How it works
Firms apply explainability at two levels. Global explanations describe which inputs drive a model overall. Local explanations show why a single customer was declined or a single transaction was flagged. Techniques include feature importance scores, reason codes and simpler surrogate models. Model validation teams test whether explanations are stable and match how the model actually behaves.
Explainability matters because supervisors expect firms to understand the models they use. A financial crime analyst needs to know why an alert fired before filing a report. A lender in the US must state the principal reasons for an adverse decision. Under the EU AI Act, people affected by high-risk AI decisions gain a right to an explanation from 2 December 2027.
The rules
Requires instructions that let deployers interpret outputs, and gives affected people a right to an explanation of decisions based on high-risk AI.
Providers and deployers of Annex III high-risk AI systemsGives individuals the right not to be subject to solely automated decisions with legal or similarly significant effects, subject to safeguards including human intervention.
Controllers processing personal data of people in the EURequires creditors to give applicants the specific principal reasons for adverse credit decisions, including decisions made with complex models.
US creditorsSets five principles for model identification, governance, development, independent validation and risk mitigants, covering AI and machine learning models.
UK banks, building societies and PRA-designated investment firms with internal model approvalCommon questions
Is explainability a legal requirement?
Several rules require it in effect. GDPR requires meaningful information about the logic of automated decisions. US Regulation B requires specific reasons for adverse credit decisions. The EU AI Act requires high-risk systems to be interpretable by deployers.
What is the difference between explainability and interpretability?
Interpretability usually describes models whose workings are understandable by design, such as decision trees or scorecards. Explainability is broader and includes techniques that explain outputs of complex models after the fact.
Why does explainability matter for AML alerts?
Analysts must assess why a transaction or customer was flagged before deciding whether to escalate or report it. Clear alert reasons speed up triage and give auditors and supervisors a record of how decisions were made.
