Large Language Models and Foundation Model Systems

Last Updated May 10, 2026

Large language models and foundation model systems are no longer only natural-language models; they are increasingly becoming general-purpose computational interfaces that connect language, reasoning, retrieval, tools, memory, workflows, governance, and institutional decision-making. A large language model can generate text, summarize documents, write code, classify content, extract information, answer questions, translate language, support tutoring, assist research, draft reports, and coordinate tool use. But once an LLM is embedded in an application, it becomes more than a model. It becomes part of a foundation model system.

A foundation model system includes the base model, tokenizer, context window, prompt architecture, retrieval layer, tool interfaces, orchestration logic, safety filters, evaluation harness, monitoring pipeline, data-governance process, user-feedback loop, incident-response plan, cost controls, and institutional accountability structure. The model generates outputs, but the system determines how inputs are selected, what evidence is retrieved, what tools are invoked, what policies constrain the answer, what users see, what logs are retained, and how failure is detected.

The central argument is that responsible use of large language models requires a systems view. LLMs should not be evaluated only as text generators or benchmark performers. They should be evaluated as components inside sociotechnical systems that shape knowledge access, labor, communication, education, software production, research, public administration, and decision support. Their value depends on how they are trained, adapted, grounded, deployed, monitored, governed, and contested.

This article develops Large Language Models and Foundation Model Systems as an advanced article within the Artificial Intelligence Systems knowledge series. It explains transformer architecture, attention, tokenization, context windows, pretraining, instruction tuning, alignment, retrieval, grounding, tool use, agentic orchestration, evaluation, cost, latency, hallucination, prompt injection, privacy, systemic risk, monitoring, and institutional accountability. Selected Python and R examples appear here, while the full GitHub repository contains expanded computational scaffolding for LLM evaluation, system-risk scoring, prompt testing, retrieval review, governance summaries, SQL schemas, documentation templates, and reproducible notebooks.

Why Large Language Models Matter

Large language models matter because language is a general interface for human knowledge work. Institutions store policy, law, science, documentation, contracts, code, procedures, research, reports, correspondence, training materials, meeting notes, and decision records in language. A system that can operate across language can therefore touch many forms of work: writing, search, analysis, translation, explanation, summarization, education, planning, coding, and decision support.

Earlier language technologies were often narrow. A sentiment classifier classified sentiment. A translation model translated. A named-entity recognizer extracted entities. A search engine retrieved documents. Large language models changed this pattern by making natural-language prompting a flexible interface. A single model can perform many tasks when prompted, few-shot demonstrated, fine-tuned, retrieved against, or connected to tools.

This flexibility creates both leverage and risk. LLMs can make knowledge work faster, improve access to information, support technical documentation, help users understand complex material, and reduce friction in research and production. But they can also produce fluent falsehoods, amplify bias, leak private information, obscure uncertainty, generate insecure code, enable manipulation, and create overreliance. Their outputs often sound more authoritative than their evidence warrants.

LLMs also matter because they are increasingly becoming interface layers for other systems. A model may not only answer a question. It may retrieve documents, summarize evidence, call a calculator, inspect a dataset, write code, draft an email, invoke an API, create a ticket, query a database, or route a workflow. Once that happens, language generation becomes part of operational infrastructure.

The importance of LLMs therefore cannot be measured only by benchmark scores. Their social significance comes from how they are embedded into systems of knowledge, labor, education, governance, automation, software, and public communication. The same model can be a writing assistant, a research interface, a risky automation layer, or a decision-support system depending on the architecture around it.

Back to top ↑

From Language Models to Foundation Model Systems

A language model estimates patterns in sequences of tokens. A large language model does this at scale using massive datasets, deep neural architectures, extensive compute, and self-supervised training objectives. But the applications people use are rarely just a raw model. They are foundation model systems.

A foundation model system may include:

• a base model or model family;
• a tokenizer and context-management strategy;
• system prompts and task prompts;
• retrieval-augmented generation;
• external tools and APIs;
• memory or user-profile mechanisms;
• safety classifiers and policy filters;
• evaluation datasets and test suites;
• human review and escalation rules;
• logging, monitoring, and incident response;
• cost, latency, and rate-limit controls;
• governance documentation and audit trails.

This distinction is essential. A model may be capable in isolation but unsafe in a workflow. A model may perform well on a benchmark but fail in a domain with missing context, adversarial inputs, ambiguous instructions, or high-stakes consequences. A model may generate plausible answers, while the surrounding system fails to retrieve authoritative evidence or communicate uncertainty. The systems layer determines whether an LLM becomes a useful tool, a fragile demo, or an institutional liability.

Seeing LLMs as foundation model systems also clarifies why deployment is contextual. A model that is acceptable for brainstorming may be unacceptable for legal advice. A model that is useful for internal drafting may be risky for customer-facing commitments. A model that summarizes low-risk documents may require stronger controls when summarizing regulated records, private data, or scientific evidence. Capability is not the same as deployment readiness.

Back to top ↑

Transformers, Attention, and Sequence Modeling

The modern LLM ecosystem is built largely on transformer architecture. Transformers use attention mechanisms to model relationships among tokens in a sequence. Instead of processing language strictly step by step, attention allows the model to weigh relationships across the context. This makes transformers powerful for language modeling, translation, summarization, code generation, retrieval, multimodal alignment, and many other tasks.

Self-attention allows each token representation to be influenced by other tokens. Multi-head attention allows the model to learn different relationship patterns in parallel. Positional information allows the model to distinguish order. Feedforward layers transform representations. Stacking many layers allows the model to build increasingly complex internal representations of language, context, and task structure.

However, architecture alone does not explain LLM behavior. Data, scale, training objective, optimization, instruction tuning, reinforcement learning from feedback, system prompts, retrieval, tool use, safety policies, and deployment context all shape behavior. An LLM is not merely a transformer. It is a trained artifact embedded in a system.

It is also important not to overinterpret internal mechanisms. Attention patterns can be inspected, but they do not always provide faithful explanations of model behavior. A transformer can generate coherent text without grounding every statement in evidence. Architecture enables capability; governance determines whether capability is used responsibly.

Back to top ↑

Tokens, Context Windows, and Memory

LLMs process tokens, not human concepts directly. Tokenization converts text into units that may represent words, subwords, punctuation, spaces, code fragments, or other text pieces. The model predicts token sequences. This matters because tokenization affects cost, context length, multilingual performance, code behavior, and how efficiently information is represented.

The context window defines how much information the model can condition on at once. A longer context window allows the system to include more instructions, documents, history, examples, and tool outputs. But longer context does not automatically mean better reasoning or perfect memory. The model may still ignore relevant material, overfocus on recent tokens, confuse instructions, or fail to integrate evidence correctly.

Memory is a system design choice. Some LLM applications retain conversation history. Others retrieve prior user preferences, documents, or records. Some memory is explicit and user-controlled. Some is session-level. Some is stored in vector databases or external state. Memory can improve continuity, but it also creates privacy, consent, retention, and governance obligations.

A well-designed LLM system should distinguish context from memory and memory from authority. Context may be temporary. Memory may be retained. Authority comes from governance: source quality, permissions, user intent, and review. A model should not treat remembered information as automatically current, correct, or appropriate for a new use.

Back to top ↑

Pretraining, Instruction Tuning, and Alignment

Pretraining teaches a model broad statistical structure from large corpora. In autoregressive language modeling, the system learns to predict tokens from context. This training can produce broad language competence, factual associations, coding patterns, style imitation, reasoning-like behavior, and task flexibility. But pretraining alone does not guarantee helpfulness, truthfulness, safety, or alignment with user intent.

Instruction tuning adapts a pretrained model to follow natural-language instructions. The model is trained on examples of tasks and desired responses. This can make the model more usable as an assistant, analyst, tutor, or interface. Reinforcement learning from human feedback and related preference-optimization methods can further shape output style, refusal behavior, helpfulness, and safety constraints.

Alignment is not a finished property. A model may behave well in common cases but fail under adversarial prompts, ambiguous tasks, unusual domains, or hidden conflicts among instructions. Alignment also depends on the system layer: system prompts, tool permissions, retrieval sources, policy filters, monitoring, and human oversight. A model aligned for one deployment may be inappropriate for another.

Alignment should therefore be treated as ongoing system work. It requires evaluation, red teaming, incident review, monitoring, documentation, and deployment boundaries. A model may become safer for general assistance while still requiring strict controls when used for legal, medical, financial, infrastructure, educational, or public-sector workflows.

Back to top ↑

Retrieval, Grounding, and Knowledge Systems

LLMs generate from learned patterns and current context. They do not automatically know whether a statement is true, current, legally valid, scientifically supported, or institutionally authoritative. Retrieval-augmented generation addresses this by bringing external evidence into the context window.

A retrieval-augmented LLM system typically includes:

• document ingestion;
• chunking and metadata creation;
• embedding generation;
• vector or hybrid search;
• authority and freshness filtering;
• passage selection;
• prompt construction;
• answer generation;
• citation or source display;
• evaluation of retrieval and answer quality.

Grounding is not merely adding documents. Retrieved evidence must be relevant, authoritative, current, and correctly used. If retrieval fails, the model may answer from incomplete or misleading context. If chunking is poor, relevant evidence may be fragmented. If metadata is weak, the system may retrieve semantically similar but low-quality sources. If the prompt does not force source use, the model may ignore evidence. A grounded LLM system is therefore a knowledge architecture problem.

Grounding also affects user trust. A sourced answer may look more reliable than an unsourced answer, even when the source is weak or misapplied. Responsible LLM systems should avoid citation laundering: the practice of attaching sources to claims they do not actually support. Strong grounding makes evidence visible; weak grounding makes unsupported claims look institutional.

Back to top ↑

Tools, Agents, and Orchestration Layers

LLMs can be connected to tools: calculators, search engines, databases, code interpreters, calendars, email systems, workflow engines, data-analysis libraries, document stores, APIs, and domain-specific software. Tool use can make LLM systems more capable by allowing them to retrieve, compute, write, transform, execute, and act.

But tool use changes the risk profile. A generated paragraph may be wrong, but a tool-using system may send an email, update a record, run code, query private data, modify a calendar, or trigger an external workflow. This requires permissioning, sandboxing, logging, user confirmation, access control, input validation, and rollback mechanisms.

Agentic systems add additional complexity. They may decompose tasks, plan steps, call tools, observe results, revise plans, and continue until a goal is met. This can be useful for research, automation, coding, and operations. It also introduces risks of runaway loops, tool misuse, hidden state, prompt injection, overbroad permissions, and poor error recovery. In governed environments, autonomy must be bounded by policy and monitoring.

Orchestration design should distinguish drafting from execution. A model may draft an email without sending it, generate a database query without running it, propose a code patch without deploying it, or prepare a workflow update without applying it. These boundaries preserve human judgment and reduce the risk of unintended action.

Back to top ↑

Evaluation: Capability, Reliability, Safety, and Use-Case Fit

LLM evaluation should be multidimensional. A model can be strong at summarization but weak at citation fidelity. It can write fluent explanations while producing false details. It can pass a benchmark while failing in a specialized institutional workflow. It can be safe in general chat but unsafe when connected to sensitive tools.

Evaluation should be scenario-specific. A legal assistant, coding assistant, research assistant, customer-support bot, tutoring system, medical summarizer, or public-sector decision-support tool requires different evaluation data and different failure thresholds. General model performance is not a substitute for system-level evaluation.

Evaluation should also include negative cases. A system should be tested when the answer is absent, when the user request is ambiguous, when retrieved sources conflict, when a tool fails, when prompts are adversarial, when the system should refuse, and when human review is required. A system that performs well only under ideal prompts is not deployment-ready.

Back to top ↑

Cost, Latency, Context, and Infrastructure Constraints

LLM systems are constrained by cost, latency, throughput, memory, context length, and infrastructure reliability. Every request consumes tokens. Longer prompts cost more. Retrieved context adds tokens. Tool calls add latency. Multi-step agentic workflows can multiply cost. High-volume deployments require monitoring, caching, rate limiting, fallback behavior, and service-level planning.

Cost and latency also shape design. A system may use a smaller model for routing and a larger model for final synthesis. It may cache common answers, retrieve only top evidence, summarize context, compress memory, or use structured outputs to reduce retries. It may use batch evaluation offline and stricter controls online. Efficient design is not merely financial; it affects reliability, sustainability, and user trust.

Infrastructure design should include fallback paths. If retrieval fails, the system should not pretend evidence exists. If a tool times out, the system should report the failure. If the model returns unsafe or malformed output, the system should retry, escalate, or stop. LLM systems need observability for prompts, retrieval, tools, outputs, errors, refusals, user feedback, and cost.

Operational constraints also shape equity. Expensive systems may be available only to some users. Low-latency tools may be prioritized for high-value workflows. Cost-cutting may reduce retrieval depth, human review, or monitoring. Responsible design should make these tradeoffs explicit rather than hiding them inside system behavior.

Back to top ↑

Hallucination, Prompt Injection, Data Leakage, and Systemic Risk

LLM risks are not limited to hallucination, although hallucination is central. A model may generate unsupported claims, fabricate citations, misstate policy, invent code behavior, or overgeneralize from weak evidence. The danger is amplified because language can sound confident even when it is wrong.

Prompt injection is a system-level security risk. Malicious or untrusted content can instruct the model to ignore rules, reveal data, manipulate tool calls, or alter behavior. Retrieval-augmented systems are especially exposed because retrieved documents may contain adversarial instructions. Tool-using systems must treat untrusted text as data, not authority.

Data leakage can occur through prompts, logs, training data, memory systems, or tool integrations. Sensitive user information, proprietary documents, credentials, internal records, or regulated data may enter an LLM workflow. Governance requires access controls, retention policies, redaction, logging rules, privacy review, and clear boundaries around training and storage.

Systemic risk arises when many downstream systems depend on similar foundation models, evaluation assumptions, or vendor infrastructure. If many applications share the same base model, defects can propagate widely. Homogenization creates efficiency, but it can also create correlated failure.

Risk management should focus not only on what the model says, but on what users and systems do with the output. A low-quality answer in a brainstorming context may be low risk. The same answer in a medical, legal, financial, public-sector, infrastructure, or educational context may be high risk. Use context determines consequence.

Back to top ↑

Governance, Monitoring, and Institutional Accountability

LLM governance should address both model behavior and system behavior. Model-level documentation is necessary but insufficient. Institutions must also document the application layer, retrieval sources, tool permissions, prompt policies, evaluation plan, monitoring signals, escalation procedures, data-retention practices, and human-review responsibilities.

A responsible LLM system should document:

• base model and version;
• intended use and prohibited use;
• system prompt and policy constraints;
• retrieval sources and authority rules;
• tool permissions and confirmation rules;
• data classification and privacy controls;
• evaluation datasets and metrics;
• known limitations and failure modes;
• monitoring signals and incident thresholds;
• human review and escalation paths;
• rollback or model-switching procedures;
• audit logging and retention policy.

Monitoring should include quality, safety, security, cost, latency, retrieval quality, tool failures, refusal patterns, user feedback, and incident reports. The institution deploying the system is responsible for how the model is used. It cannot treat model output as external magic. Once an LLM is embedded in a workflow, it becomes part of institutional action.

Institutional accountability also requires contestability. Users and affected parties should have appropriate ways to question, appeal, correct, or challenge outputs where LLM systems influence decisions, access, benefits, services, evaluation, or official communication. Human oversight should be meaningful, not ceremonial. Reviewers need context, evidence, authority, and time.

Governance should also distinguish between model cards and system cards. A model card can describe model training, evaluation, intended use, and limitations. A system card should describe the deployed application: prompts, retrieval, tools, data flows, monitoring, privacy controls, escalation, and operational boundaries. LLM risk often lives in the system layer.

Back to top ↑

Common Failure Modes

Large language model systems often fail when fluency is mistaken for reliability. A polished response can conceal weak evidence, ambiguous instructions, missing sources, unsupported synthesis, or tool failure. Foundation model systems also fail when organizations treat deployment as a model-selection problem rather than a systems-governance problem.

These failure modes reinforce the systems argument. LLM deployment is not only about choosing a capable model. It is about designing the surrounding architecture so that evidence, authority, uncertainty, permissions, monitoring, and accountability remain visible.

Back to top ↑

Limits and Open Problems

Large language models and foundation model systems have important limits. Fluency is not truth: language quality can mask weak evidence or false claims. Context is not comprehension: adding more text to the prompt does not guarantee correct use of evidence. Retrieval can fail silently: a system may retrieve irrelevant, stale, or low-authority sources. Prompt injection is a system risk: untrusted text can attempt to manipulate model behavior or tool use.

Tool use increases consequences. An LLM connected to tools can act, not only answer. Memory creates privacy obligations because retained context and user data require consent, retention controls, and access rules. Evaluation can be too generic: benchmark performance may not predict domain-specific system reliability. Governance cannot be outsourced to the model: institutions remain responsible for the workflows they deploy.

Several open problems remain difficult. How should institutions evaluate open-ended outputs at scale without reducing quality to simplistic scores? How should systems represent uncertainty in ways users actually understand? How should governance handle long-context models that process large amounts of sensitive material? How should systems detect prompt injection hidden in retrieved documents, emails, or webpages? How should organizations evaluate downstream labor, educational, civic, and institutional effects?

Another open problem is dependency. Many organizations may build different products on similar foundation models, cloud infrastructures, and evaluation assumptions. This creates efficiency and standardization, but it also creates correlated risk. If a base model has a systematic failure, or if a vendor changes behavior, many downstream systems can be affected at once.

The goal is not to dismiss LLMs. They are among the most important technologies in modern AI. The goal is to understand them as components within foundation model systems: powerful, flexible, uncertain, and deeply dependent on design choices around retrieval, tools, prompts, evaluation, monitoring, and governance. Responsible deployment requires building systems that make evidence visible, uncertainty reviewable, permissions constrained, failures detectable, and accountability explicit.

Back to top ↑

Mathematical Lens

A language model estimates the probability of a token sequence by decomposing it into next-token predictions.

Training often minimizes negative log-likelihood over a corpus.

Scaled dot-product attention computes relationships among tokens.

Generation samples or selects the next token from a probability distribution.

Retrieval-augmented generation conditions the model on external evidence.

Tool-using systems transform generated decisions into actions.

System-level risk depends on model behavior, task context, evidence quality, user reliance, and institutional consequences.

A review rule can route high-risk outputs or actions to human oversight.

Back to top ↑

Variables and System Interpretation

Back to top ↑

Worked Example: Building a Governed LLM Knowledge Assistant

Consider an organization building an internal knowledge assistant for policy, technical documentation, project records, and research materials. The goal is not only to answer questions, but to answer from approved sources, show evidence, respect permissions, identify uncertainty, and route high-risk requests to human review.

A responsible design would include:

1. Define the approved use cases: search, summarization, drafting, explanation, and document navigation.
2. Classify documents by sensitivity, authority, freshness, and access permissions.
3. Build a retrieval system with metadata, chunking, hybrid search, and source ranking.
4. Use prompts that require evidence-grounded answers and uncertainty disclosure.
5. Block the model from answering restricted legal, medical, financial, or policy questions without review.
6. Evaluate retrieval quality and answer quality on realistic internal queries.
7. Test for prompt injection in retrieved documents.
8. Log prompts, retrieval results, model outputs, failures, costs, and user feedback according to privacy policy.
9. Provide human escalation for sensitive, ambiguous, or high-impact cases.
10. Review incidents and update retrieval, prompts, permissions, and evaluation sets over time.

This example shows why LLM deployment is a systems problem. The answer quality depends on the model, but also on data governance, retrieval design, prompt control, permissions, evaluation, monitoring, and institutional accountability.

Suppose the assistant is asked for the organization’s current data-retention policy. It retrieves an archived document, a draft revision, and the currently approved policy. A weak system may summarize all three as if they are equally authoritative. A governed system would identify the approved policy as controlling, label the archived and draft sources, cite only the current policy for operational guidance, and disclose uncertainty if the source record is incomplete.

Back to top ↑

Computational Modeling

Computational modeling can make LLM system governance concrete. An evaluation workflow can score task quality, grounding, factuality, citation fidelity, safety, prompt-injection resistance, privacy controls, latency, cost, tool behavior, and review requirements. A system-risk workflow can identify which use cases require additional controls before deployment. A monitoring workflow can track quality, incidents, refusals, tool failures, retrieval failures, costs, and user feedback over time.

The examples below are intentionally lightweight and educational. They do not replace production evaluation harnesses, red-team procedures, retrieval logs, model cards, system cards, or human review. Their purpose is to show how LLM systems can be evaluated as foundation model systems rather than as isolated text generators.

A mature production system would connect these workflows to real evaluation datasets, prompt versions, retrieval traces, tool-call logs, monitoring pipelines, privacy reviews, user feedback, incident records, and governance documentation. The goal is not merely to score responses. The goal is to determine whether the system can be trusted for a defined use case under defined controls.

Back to top ↑

Python Workflow: LLM System Evaluation and Governance Review

The following Python workflow simulates an LLM application evaluation portfolio. It scores responses across task quality, grounding, factuality, safety, security, latency, cost, and governance risk. The workflow is dependency-light and designed to be adapted to real evaluation logs.

"""
Large Language Models and Foundation Model Systems

Python workflow:
- Simulate LLM application evaluation records.
- Score quality, grounding, factuality, citation fidelity, safety, and security.
- Estimate cost, latency, retrieval, and tool-use risk.
- Produce governance review flags and summary tables.

This is a simplified workflow. Production LLM systems should connect these
records to actual evaluation datasets, retrieval logs, prompt versions,
tool-call logs, human reviews, privacy reviews, and incident records.
"""

from __future__ import annotations

from pathlib import Path

import numpy as np
import pandas as pd


RANDOM_SEED = 42
rng = np.random.default_rng(RANDOM_SEED)

OUTPUT_DIR = Path("outputs")
OUTPUT_DIR.mkdir(exist_ok=True)


def simulate_llm_evaluations(n: int = 220) -> pd.DataFrame:
    """Create synthetic evaluation records for LLM system responses."""
    use_cases = [
        "knowledge_search",
        "document_summary",
        "code_assistance",
        "policy_explanation",
        "research_synthesis",
        "customer_support",
        "decision_support",
    ]

    risk_levels = ["low", "medium", "high"]

    rows = []

    for i in range(n):
        use_case = rng.choice(use_cases)
        risk_level = rng.choice(risk_levels, p=[0.50, 0.35, 0.15])

        task_quality = rng.uniform(0.55, 0.98)
        grounding_score = rng.uniform(0.35, 0.98)
        factuality_score = rng.uniform(0.45, 0.99)
        citation_fidelity = rng.uniform(0.35, 0.98)
        safety_score = rng.uniform(0.55, 1.00)
        prompt_injection_resistance = rng.uniform(0.40, 0.98)
        privacy_control_score = rng.uniform(0.55, 1.00)
        human_review_score = rng.uniform(0.40, 1.00)
        observability_score = rng.uniform(0.45, 1.00)

        retrieved_sources = int(rng.integers(0, 8))
        tool_calls = int(rng.integers(0, 5))
        input_tokens = int(rng.integers(400, 9000))
        output_tokens = int(rng.integers(100, 1800))
        latency_seconds = float(rng.gamma(shape=2.5, scale=1.2) + 0.15 * tool_calls)

        if use_case in ["decision_support", "policy_explanation"]:
            grounding_score = min(grounding_score, rng.uniform(0.35, 0.90))
            citation_fidelity = min(citation_fidelity, rng.uniform(0.35, 0.92))

        if tool_calls >= 3:
            prompt_injection_resistance = min(
                prompt_injection_resistance,
                rng.uniform(0.40, 0.88),
            )

        rows.append(
            {
                "eval_id": f"LLM-EVAL-{i:03d}",
                "use_case": use_case,
                "risk_level": risk_level,
                "task_quality": float(task_quality),
                "grounding_score": float(grounding_score),
                "factuality_score": float(factuality_score),
                "citation_fidelity": float(citation_fidelity),
                "safety_score": float(safety_score),
                "prompt_injection_resistance": float(prompt_injection_resistance),
                "privacy_control_score": float(privacy_control_score),
                "human_review_score": float(human_review_score),
                "observability_score": float(observability_score),
                "retrieved_sources": retrieved_sources,
                "tool_calls": tool_calls,
                "input_tokens": input_tokens,
                "output_tokens": output_tokens,
                "latency_seconds": latency_seconds,
            }
        )

    return pd.DataFrame(rows)


def score_llm_system(records: pd.DataFrame) -> pd.DataFrame:
    """Score LLM evaluation records for quality and governance risk."""
    scored = records.copy()

    scored["total_tokens"] = scored["input_tokens"] + scored["output_tokens"]

    scored["quality_score"] = (
        0.22 * scored["task_quality"]
        + 0.26 * scored["grounding_score"]
        + 0.26 * scored["factuality_score"]
        + 0.26 * scored["citation_fidelity"]
    )

    scored["security_and_safety_score"] = (
        0.35 * scored["safety_score"]
        + 0.25 * scored["prompt_injection_resistance"]
        + 0.25 * scored["privacy_control_score"]
        + 0.15 * scored["human_review_score"]
    )

    scored["governance_readiness_score"] = (
        0.30 * scored["observability_score"]
        + 0.25 * scored["human_review_score"]
        + 0.20 * scored["privacy_control_score"]
        + 0.15 * scored["citation_fidelity"]
        + 0.10 * scored["grounding_score"]
    )

    scored["operational_cost_index"] = np.clip(
        (scored["total_tokens"] / 10000)
        + (scored["latency_seconds"] / 20)
        + (scored["tool_calls"] / 10),
        0,
        1.5,
    )

    scored["tool_use_risk"] = np.clip(scored["tool_calls"] / 5, 0, 1)

    scored["retrieval_gap_risk"] = np.where(
        scored["retrieved_sources"] == 0,
        0.35,
        np.clip(0.20 - (scored["retrieved_sources"] / 30), 0, 0.20),
    )

    risk_weight = scored["risk_level"].map(
        {
            "low": 0.10,
            "medium": 0.25,
            "high": 0.45,
        }
    )

    scored["llm_system_risk"] = (
        0.22 * (1 - scored["quality_score"])
        + 0.22 * (1 - scored["security_and_safety_score"])
        + 0.18 * (1 - scored["grounding_score"])
        + 0.14 * (1 - scored["governance_readiness_score"])
        + 0.10 * scored["operational_cost_index"]
        + 0.07 * scored["tool_use_risk"]
        + 0.07 * risk_weight
    )

    scored["review_required"] = (
        (scored["llm_system_risk"] > 0.40)
        | (scored["risk_level"].eq("high"))
        | (scored["grounding_score"] < 0.60)
        | (scored["factuality_score"] < 0.65)
        | (scored["citation_fidelity"] < 0.60)
        | (scored["prompt_injection_resistance"] < 0.60)
        | (scored["privacy_control_score"] < 0.70)
        | (scored["human_review_score"] < 0.60)
        | ((scored["tool_calls"] >= 3) & (scored["safety_score"] < 0.80))
    )

    scored["deployment_recommendation"] = np.select(
        [
            scored["llm_system_risk"] > 0.55,
            scored["privacy_control_score"] < 0.70,
            scored["prompt_injection_resistance"] < 0.60,
            scored["citation_fidelity"] < 0.60,
            scored["review_required"],
            scored["quality_score"] > 0.82,
        ],
        [
            "pause_for_system_risk_review",
            "fix_privacy_controls_before_deployment",
            "run_prompt_injection_and_tool_security_review",
            "run_citation_and_grounding_review",
            "approve_only_after_human_review",
            "candidate_for_controlled_deployment",
        ],
        default="continue_evaluation",
    )

    return scored.sort_values("llm_system_risk", ascending=False)


def summarize_by_use_case(scored: pd.DataFrame) -> pd.DataFrame:
    """Summarize quality and risk by use case."""
    return (
        scored.groupby("use_case")
        .agg(
            evaluations=("eval_id", "count"),
            mean_quality_score=("quality_score", "mean"),
            mean_grounding_score=("grounding_score", "mean"),
            mean_factuality_score=("factuality_score", "mean"),
            mean_security_and_safety=("security_and_safety_score", "mean"),
            mean_governance_readiness=("governance_readiness_score", "mean"),
            mean_llm_system_risk=("llm_system_risk", "mean"),
            review_rate=("review_required", "mean"),
            mean_tool_calls=("tool_calls", "mean"),
            mean_total_tokens=("total_tokens", "mean"),
            mean_latency_seconds=("latency_seconds", "mean"),
        )
        .reset_index()
        .sort_values("mean_llm_system_risk", ascending=False)
    )


def summarize_by_risk_level(scored: pd.DataFrame) -> pd.DataFrame:
    """Summarize LLM system quality and review requirements by risk level."""
    return (
        scored.groupby("risk_level")
        .agg(
            evaluations=("eval_id", "count"),
            mean_quality_score=("quality_score", "mean"),
            mean_grounding_score=("grounding_score", "mean"),
            mean_security_and_safety=("security_and_safety_score", "mean"),
            mean_governance_readiness=("governance_readiness_score", "mean"),
            mean_llm_system_risk=("llm_system_risk", "mean"),
            review_rate=("review_required", "mean"),
        )
        .reset_index()
        .sort_values("mean_llm_system_risk", ascending=False)
    )


def main() -> None:
    """Run LLM system evaluation and governance review."""
    records = simulate_llm_evaluations()
    scored = score_llm_system(records)
    use_case_summary = summarize_by_use_case(scored)
    risk_summary = summarize_by_risk_level(scored)

    governance_summary = pd.DataFrame(
        [
            {
                "evaluations_reviewed": len(scored),
                "review_required": int(scored["review_required"].sum()),
                "high_risk_cases": int(scored["risk_level"].eq("high").sum()),
                "low_grounding_cases": int((scored["grounding_score"] < 0.60).sum()),
                "low_citation_fidelity_cases": int(
                    (scored["citation_fidelity"] < 0.60).sum()
                ),
                "prompt_injection_review_cases": int(
                    (scored["prompt_injection_resistance"] < 0.60).sum()
                ),
                "privacy_review_cases": int(
                    (scored["privacy_control_score"] < 0.70).sum()
                ),
                "mean_quality_score": scored["quality_score"].mean(),
                "mean_grounding_score": scored["grounding_score"].mean(),
                "mean_security_and_safety": scored["security_and_safety_score"].mean(),
                "mean_governance_readiness": scored["governance_readiness_score"].mean(),
                "mean_system_risk": scored["llm_system_risk"].mean(),
                "mean_total_tokens": scored["total_tokens"].mean(),
                "mean_latency_seconds": scored["latency_seconds"].mean(),
            }
        ]
    )

    records.to_csv(OUTPUT_DIR / "python_llm_evaluation_records.csv", index=False)
    scored.to_csv(OUTPUT_DIR / "python_llm_system_risk_scores.csv", index=False)

    use_case_summary.to_csv(
        OUTPUT_DIR / "python_llm_use_case_summary.csv",
        index=False,
    )

    risk_summary.to_csv(
        OUTPUT_DIR / "python_llm_risk_summary.csv",
        index=False,
    )

    governance_summary.to_csv(
        OUTPUT_DIR / "python_llm_governance_summary.csv",
        index=False,
    )

    memo = f"""# LLM Foundation Model System Governance Memo

Evaluations reviewed: {int(governance_summary.loc[0, "evaluations_reviewed"])}
Review required: {int(governance_summary.loc[0, "review_required"])}
High-risk cases: {int(governance_summary.loc[0, "high_risk_cases"])}
Low-grounding cases: {int(governance_summary.loc[0, "low_grounding_cases"])}
Low citation-fidelity cases: {int(governance_summary.loc[0, "low_citation_fidelity_cases"])}
Prompt-injection review cases: {int(governance_summary.loc[0, "prompt_injection_review_cases"])}
Privacy review cases: {int(governance_summary.loc[0, "privacy_review_cases"])}
Mean quality score: {governance_summary.loc[0, "mean_quality_score"]:.4f}
Mean grounding score: {governance_summary.loc[0, "mean_grounding_score"]:.4f}
Mean security and safety score: {governance_summary.loc[0, "mean_security_and_safety"]:.4f}
Mean governance readiness: {governance_summary.loc[0, "mean_governance_readiness"]:.4f}
Mean system risk: {governance_summary.loc[0, "mean_system_risk"]:.4f}
Mean total tokens: {governance_summary.loc[0, "mean_total_tokens"]:.2f}
Mean latency seconds: {governance_summary.loc[0, "mean_latency_seconds"]:.2f}

Interpretation:
- LLM systems should be evaluated by use case, not only by general benchmarks.
- Grounding, factuality, citation fidelity, safety, security, and privacy should be reviewed together.
- High-risk use cases require human review and explicit deployment boundaries.
- Token cost, latency, retrieval quality, and tool behavior are system-level governance concerns.
"""

    (OUTPUT_DIR / "python_llm_governance_memo.md").write_text(memo)

    print(governance_summary.T)
    print(use_case_summary)
    print(risk_summary)
    print(scored.head(10))
    print(memo)


if __name__ == "__main__":
    main()

This workflow treats LLM evaluation as foundation-model-system governance. It does not rank responses only by task quality. It also examines grounding, factuality, citation fidelity, safety, prompt-injection resistance, privacy controls, human review, observability, tool use, cost, latency, and system risk. That mirrors the central argument of the article: large language models should be evaluated as deployed systems, not isolated text engines.

Back to top ↑

R Workflow: LLM Evaluation Summary and Risk Review

The following R workflow summarizes LLM evaluation records by use case, risk level, quality score, grounding score, safety score, and review status. It is designed as a lightweight statistical review layer for LLM system governance.

# Large Language Models and Foundation Model Systems
# R workflow: LLM evaluation summary and risk review.

set.seed(42)

n <- 220

use_cases <- c(
  "knowledge_search",
  "document_summary",
  "code_assistance",
  "policy_explanation",
  "research_synthesis",
  "customer_support",
  "decision_support"
)

risk_levels <- c("low", "medium", "high")

records <- data.frame(
  eval_id = paste0("LLM-EVAL-", sprintf("%03d", 1:n)),
  use_case = sample(use_cases, size = n, replace = TRUE),
  risk_level = sample(
    risk_levels,
    size = n,
    replace = TRUE,
    prob = c(0.50, 0.35, 0.15)
  ),
  task_quality = runif(n, min = 0.55, max = 0.98),
  grounding_score = runif(n, min = 0.35, max = 0.98),
  factuality_score = runif(n, min = 0.45, max = 0.99),
  citation_fidelity = runif(n, min = 0.35, max = 0.98),
  safety_score = runif(n, min = 0.55, max = 1.00),
  prompt_injection_resistance = runif(n, min = 0.40, max = 0.98),
  privacy_control_score = runif(n, min = 0.55, max = 1.00),
  human_review_score = runif(n, min = 0.40, max = 1.00),
  observability_score = runif(n, min = 0.45, max = 1.00),
  input_tokens = sample(400:9000, size = n, replace = TRUE),
  output_tokens = sample(100:1800, size = n, replace = TRUE),
  tool_calls = sample(0:4, size = n, replace = TRUE),
  latency_seconds = rgamma(n, shape = 2.5, scale = 1.2)
)

records$total_tokens <- records$input_tokens + records$output_tokens

records$quality_score <- 0.22 * records$task_quality +
  0.26 * records$grounding_score +
  0.26 * records$factuality_score +
  0.26 * records$citation_fidelity

records$security_and_safety_score <- 0.35 * records$safety_score +
  0.25 * records$prompt_injection_resistance +
  0.25 * records$privacy_control_score +
  0.15 * records$human_review_score

records$governance_readiness_score <- 0.30 * records$observability_score +
  0.25 * records$human_review_score +
  0.20 * records$privacy_control_score +
  0.15 * records$citation_fidelity +
  0.10 * records$grounding_score

records$operational_cost_index <- pmin(
  (records$total_tokens / 10000) +
    (records$latency_seconds / 20) +
    (records$tool_calls / 10),
  1.5
)

records$tool_use_risk <- pmin(records$tool_calls / 5, 1)

records$risk_weight <- ifelse(
  records$risk_level == "low",
  0.10,
  ifelse(records$risk_level == "medium", 0.25, 0.45)
)

records$llm_system_risk <- 0.22 * (1 - records$quality_score) +
  0.22 * (1 - records$security_and_safety_score) +
  0.18 * (1 - records$grounding_score) +
  0.14 * (1 - records$governance_readiness_score) +
  0.10 * records$operational_cost_index +
  0.07 * records$tool_use_risk +
  0.07 * records$risk_weight

records$review_required <- records$llm_system_risk > 0.40 |
  records$risk_level == "high" |
  records$grounding_score < 0.60 |
  records$factuality_score < 0.65 |
  records$citation_fidelity < 0.60 |
  records$prompt_injection_resistance < 0.60 |
  records$privacy_control_score < 0.70 |
  records$human_review_score < 0.60 |
  (records$tool_calls >= 3 & records$safety_score < 0.80)

use_case_summary <- aggregate(
  cbind(
    quality_score,
    grounding_score,
    factuality_score,
    security_and_safety_score,
    governance_readiness_score,
    llm_system_risk,
    review_required,
    total_tokens,
    latency_seconds,
    tool_calls
  ) ~ use_case,
  data = records,
  FUN = mean
)

risk_summary <- aggregate(
  cbind(
    quality_score,
    grounding_score,
    security_and_safety_score,
    governance_readiness_score,
    llm_system_risk,
    review_required
  ) ~ risk_level,
  data = records,
  FUN = mean
)

governance_summary <- data.frame(
  evaluations_reviewed = nrow(records),
  review_required = sum(records$review_required),
  high_risk_cases = sum(records$risk_level == "high"),
  low_grounding_cases = sum(records$grounding_score < 0.60),
  low_citation_fidelity_cases = sum(records$citation_fidelity < 0.60),
  prompt_injection_review_cases = sum(records$prompt_injection_resistance < 0.60),
  privacy_review_cases = sum(records$privacy_control_score < 0.70),
  mean_quality_score = mean(records$quality_score),
  mean_grounding_score = mean(records$grounding_score),
  mean_security_and_safety = mean(records$security_and_safety_score),
  mean_governance_readiness = mean(records$governance_readiness_score),
  mean_system_risk = mean(records$llm_system_risk),
  mean_total_tokens = mean(records$total_tokens),
  mean_latency_seconds = mean(records$latency_seconds)
)

dir.create("outputs", recursive = TRUE, showWarnings = FALSE)

write.csv(records, "outputs/r_llm_evaluation_records.csv", row.names = FALSE)
write.csv(use_case_summary, "outputs/r_llm_use_case_summary.csv", row.names = FALSE)
write.csv(risk_summary, "outputs/r_llm_risk_summary.csv", row.names = FALSE)
write.csv(governance_summary, "outputs/r_llm_governance_summary.csv", row.names = FALSE)

print("Use-case summary")
print(use_case_summary)

print("Risk summary")
print(risk_summary)

print("Governance summary")
print(governance_summary)

This R workflow mirrors the LLM-governance structure in a compact form. It summarizes use-case-level and risk-level patterns so task quality, grounding, factuality, citation fidelity, safety, prompt-injection resistance, privacy controls, human review, observability, tool use, latency, cost, and review status can be interpreted together.

Back to top ↑

GitHub Repository

The article body includes selected computational examples so the conceptual and mathematical argument remains readable. The full repository can hold expanded workflows for prompt evaluation, retrieval testing, tool-use logs, system-risk scoring, prompt-injection simulation, privacy review, cost analysis, governance dashboards, model cards, system cards, and lifecycle monitoring.

Back to top ↑

From Language Generation to Accountable Systems

Large language models mark a major shift in artificial intelligence because they turn language into a general interface for computation, knowledge, writing, coding, search, explanation, and workflow support. They can compress enormous amounts of learned structure into a flexible conversational system. But the same flexibility makes them difficult to govern when they are treated as stand-alone intelligence rather than components in designed systems.

The central lesson is that LLMs become trustworthy only through system architecture. A base model generates outputs. A foundation model system decides what context enters the prompt, what evidence is retrieved, what tools are available, what memory is retained, what actions require approval, what outputs are blocked, what incidents are logged, and what humans can contest. Responsibility lives in that larger architecture.

This article also shows why evidence and authority matter. An LLM can produce language that sounds plausible, but institutions need answers that are grounded, current, permitted, reviewable, and accountable. Retrieval systems, citation checks, source registries, monitoring, and human review are not optional extras. They are part of the infrastructure that turns model capability into responsible knowledge work.

The strongest foundation model systems will not be those that simply maximize autonomy or output volume. They will be those that make uncertainty visible, preserve source context, constrain tool authority, protect sensitive data, monitor failures, support contestability, and assign accountability. LLMs are powerful technologies, but their legitimacy depends on the systems built around them.

Within the Artificial Intelligence Systems knowledge series, this article belongs near Self-Supervised Learning and Foundation Models, Representation Learning and Embedding Spaces, Transfer Learning, Fine-Tuning, and Model Adaptation, Retrieval-Augmented Generation and AI Knowledge Systems, AI Agents, Tool Use, and Workflow Automation, Natural Language Processing and Computational Language Systems, Data Governance, Provenance, and Lineage in AI Systems, and AI Governance and Regulatory Systems. It provides the foundation-model-system layer for understanding how language models become applications, workflows, institutions, and risks.

Back to top ↑

Related Articles

• Artificial Intelligence Systems
• Self-Supervised Learning and Foundation Models
• Representation Learning and Embedding Spaces
• Transfer Learning, Fine-Tuning, and Model Adaptation
• Retrieval-Augmented Generation and AI Knowledge Systems
• Generative AI and Synthetic Content Systems
• Natural Language Processing and Computational Language Systems
• Explainable AI and Model Interpretability
• Data Governance, Provenance, and Lineage in AI Systems
• AI Safety and System Reliability
• Model Validation, Benchmarking, and Generalization Theory

Further Reading

• Vaswani, A. et al. (2017) ‘Attention Is All You Need’. Available at: https://arxiv.org/abs/1706.03762
• Devlin, J., Chang, M.W., Lee, K. and Toutanova, K. (2019) ‘BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding’. Available at: https://arxiv.org/abs/1810.04805
• Brown, T.B. et al. (2020) ‘Language Models are Few-Shot Learners’, Advances in Neural Information Processing Systems. Available at: https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
• Bommasani, R. et al. (2021) ‘On the Opportunities and Risks of Foundation Models’. Available at: https://arxiv.org/abs/2108.07258
• Ouyang, L. et al. (2022) ‘Training language models to follow instructions with human feedback’. Available at: https://arxiv.org/abs/2203.02155
• Liang, P. et al. (2022) ‘Holistic Evaluation of Language Models’. Available at: https://arxiv.org/abs/2211.09110
• Lewis, P. et al. (2020) ‘Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks’. Available at: https://arxiv.org/abs/2005.11401
• NIST (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). Available at: https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10

References

• Bommasani, R. et al. (2021) ‘On the Opportunities and Risks of Foundation Models’. Available at: https://arxiv.org/abs/2108.07258
• Brown, T.B. et al. (2020) ‘Language Models are Few-Shot Learners’, Advances in Neural Information Processing Systems. Available at: https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html
• Devlin, J., Chang, M.W., Lee, K. and Toutanova, K. (2019) ‘BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding’. Available at: https://arxiv.org/abs/1810.04805
• Lewis, P. et al. (2020) ‘Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks’. Available at: https://arxiv.org/abs/2005.11401
• Liang, P. et al. (2022) ‘Holistic Evaluation of Language Models’. Available at: https://arxiv.org/abs/2211.09110
• NIST (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). Available at: https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10
• OpenAI (2023) ‘GPT-4 Technical Report’. Available at: https://arxiv.org/abs/2303.08774
• Ouyang, L. et al. (2022) ‘Training language models to follow instructions with human feedback’. Available at: https://arxiv.org/abs/2203.02155
• Perez, F. and Ribeiro, I. (2022) ‘Ignore Previous Prompt: Attack Techniques For Language Models’. Available at: https://arxiv.org/abs/2211.09527
• Vaswani, A. et al. (2017) ‘Attention Is All You Need’. Available at: https://arxiv.org/abs/1706.03762
• Wei, J. et al. (2022) ‘Chain-of-Thought Prompting Elicits Reasoning in Large Language Models’. Available at: https://arxiv.org/abs/2201.11903