Large language model
LLMAlso called foundation model, general-purpose AI model
A large language model (LLM) is an AI model trained on very large amounts of text to predict the next word in a sequence. That lets it summarise, classify, translate and draft text. Compliance teams use LLMs to read regulations, screen news and draft reports. Under the EU AI Act, most LLMs count as general-purpose AI models.
How it works
An LLM turns text into tokens and processes them through billions of learned parameters to produce a response. Firms rarely train their own. They use a provider's model through an interface, often adding retrieval from internal documents so answers draw on approved sources. Prompts, outputs and sources are logged so reviewers can check how an answer was produced.
LLMs can produce confident but wrong answers, known as hallucinations, and their outputs can vary between runs. That matters when the output feeds a regulatory decision. Firms treat LLM use cases through model risk and third-party risk processes, test accuracy on their own data and keep humans responsible for final decisions. EU AI Act duties for general-purpose AI model providers have applied since 2 August 2025.
The rules
Requires technical documentation, copyright policies and training data summaries, with extra duties for models posing systemic risk.
Providers of general-purpose AI models placed on the EU marketVoluntary profile applying the NIST AI RMF to generative AI risks such as confabulation, data privacy and information integrity.
Organisations developing or using generative AI, on a voluntary basisCommon questions
What is an LLM used for in compliance?
Common uses include summarising new regulations, mapping rules to policies, screening adverse media, drafting suspicious activity narratives, reviewing communications and answering staff questions from internal policy documents. Most firms keep a person responsible for checking outputs.
What is an LLM hallucination?
A hallucination is an answer that sounds plausible but is false or unsupported, such as an invented citation. It happens because the model generates likely text rather than looking up facts. Grounding the model in approved sources and testing on firm data reduces the rate.
Does the EU AI Act regulate LLMs?
Yes. Most LLMs are general-purpose AI models under the Act. Their providers have had documentation and transparency duties since 2 August 2025. Firms that use an LLM for a high-risk purpose, such as credit scoring, also carry deployer duties for that system.
