Large Language Models can explain concepts, summarize documents, generate code, and operate tools through an application. These capabilities create value, but they also create risks when output is wrong, private data is mishandled, or a model triggers real-world actions.

AI Safety is the ongoing work of identifying those risks and reducing their probability and impact. It is not a single filter, prompt, or model setting.

What Is LLM Safety?

LLM Safety is the practice of designing, evaluating, deploying, and operating language-model systems so they remain useful while respecting defined boundaries for people, data, organizations, and society. Requirements depend on the use case and the possible impact of failure.

Begin with a Threat Model

A threat model identifies assets, users, attackers, data flows, tools, failure modes, and consequences before controls are selected. High-impact failures need stronger prevention, detection, approval, and recovery.

System: Internal document assistant
Assets: Employee identity and private documents
Untrusted inputs: Questions and document text
Critical risks:
- Cross-user document disclosure
- Prompt injection inside a document
- Unsupported claims without citations
- Unauthorized external actions
Fallback: No answer or human review when evidence is insufficient

Common Safety Risks

• Hallucination: false facts, citations, calculations, or code presented confidently.
• Harmful content: dangerous, abusive, discriminatory, or otherwise unsuitable responses.
• Bias: unfair quality or treatment across people, languages, or groups.
• Privacy leakage: exposure of personal, confidential, or regulated data.
• Prompt injection: untrusted text attempts to override instructions or misuse tools.
• Unsafe tool use: generated requests create unauthorized real-world effects.
• Automation bias: users trust fluent output more than the evidence supports.

Defense in Depth

Defense in depth uses independent layers so one failure does not expose the whole system. Prompts can guide behavior, but access controls, schemas, network isolation, and approval gates belong in application and infrastructure code.

Authentication and authorization
  -> Input limits and validation
  -> Scoped instructions and context
  -> Model safety behavior
  -> Tool allowlists and approval gates
  -> Output validation and source checks
  -> Human review
  -> Monitoring, incident response, and rollback

Input and Prompt Safety

• Authenticate users and enforce request-size and rate limits.
• Validate files, formats, and dynamic prompt values.
• Collect only data required for the task.
• Separate untrusted user or document content from trusted instructions.
• Avoid placing secrets in model-visible context.
• Define task boundaries and what to do when evidence is missing.
• Apply product, age, region, and risk policies before execution.

Python Validation Example

This example provides basic validation before an LLM call. It is one layer, not a complete content-safety system.

from dataclasses import dataclass

MAX_INPUT_CHARS = 4_000
ALLOWED_TASKS = {"explain", "summarize", "rewrite"}

@dataclass
class Request:
    task: str
    text: str

def validate_request(request: Request) -> str:
    task = request.task.strip().lower()
    text = request.text.strip()
    if task not in ALLOWED_TASKS:
        raise ValueError("Unsupported task")
    if not text:
        raise ValueError("Text is required")
    if len(text) > MAX_INPUT_CHARS:
        raise ValueError("Input is too long")
    return text

request = Request("explain", "Explain Python functions.")
print(validate_request(request))

Output Safety

• Validate structured output against an allowlisted schema.
• Verify claims and citations using trusted sources.
• Detect sensitive data or policy violations where appropriate.
• Sanitize generated HTML, SQL, shell commands, and executable content.
• Run generated code only in an isolated, resource-limited environment.
• Do not automatically convert generated text into external actions.
• Require qualified review for high-impact results.

RAG Safety

Retrieved documents can be malicious, stale, low quality, or incorrectly permissioned. Enforce access before retrieval, track provenance and freshness, isolate instructions found in documents, require citations, and return insufficient evidence when sources do not support an answer.

Tool and Agent Safety

• Give each workflow the minimum tools and permissions it needs.
• Validate tool names and arguments against an allowlist and schema.
• Keep credentials outside model-visible context.
• Use sandboxing and network restrictions for code execution.
• Require confirmation for sending, buying, deleting, publishing, or account changes.
• Limit retries, time, tokens, actions, and spending.
• Log tool decisions and outcomes for audit and incident review.

Privacy and Data Governance

• Document what data is collected, why, where it flows, and how long it remains.
• Minimize personal data in prompts, outputs, and logs.
• Encrypt data and separate users or tenants with access controls.
• Review provider data-use, region, retention, and deletion terms.
• Support required consent, access, correction, and deletion processes.
• Do not place production data in training or evaluation without authorization.

Human Oversight

Human review must be meaningful. Reviewers need enough time, context, expertise, authority, and interface support to reject or correct a result. Show sources and uncertainty, make escalation easy, and train reviewers to recognize automation bias.

Safety Evaluation

• Test harmful requests relevant to the product.
• Test harmless requests that resemble risky ones to measure over-refusal.
• Test prompt injection through users, retrieval, media, and tool results.
• Test privacy leakage and cross-user access.
• Measure fairness across relevant groups and languages.
• Test unauthorized tools and confirmation-bypass attempts.
• Retest after model, prompt, retrieval, policy, or tool changes.

Monitoring and Incident Response

Monitor privacy-safe signals such as validation failures, refusals, corrections, escalations, tool errors, unusual traffic, and confirmed incidents. Prepare a response process before deployment.

Detect issue
  -> Limit or disable the affected capability
  -> Preserve appropriate audit evidence
  -> Assess affected users and data
  -> Notify responsible teams
  -> Correct the control
  -> Add regression tests
  -> Restore gradually and monitor

Tradeoffs and Limitations

• No model or safeguard guarantees perfect behavior.
• Overly broad controls can block harmless and valuable requests.
• Attack methods, models, and user needs change over time.
• Safety classifiers and human reviewers have their own errors and bias.
• Additional controls can increase latency, cost, and complexity.
• Requirements differ by age, culture, law, industry, and deployment context.

Best Practices

• Define users, capabilities, assets, abuse cases, and critical failures first.
• Use layered controls across models, applications, infrastructure, and people.
• Keep authorization, business rules, and approvals outside prompts.
• Collect and expose the minimum data and privileges required.
• Test both harmful behavior and over-refusal.
• Use staged deployment, monitoring, rollback, and incident response.
• Version models, prompts, policies, tools, datasets, and safety tests.
• Communicate AI involvement, limitations, sources, and escalation paths.