Orthogonal Decomposition and Structured Approximation: How Linear Algebra Separates Signal, Residuals, and Stable Structure

Last Updated July 3, 2026

Orthogonal decomposition and structured approximation explain how complex systems can be separated into meaningful components without losing mathematical discipline. When data, signals, models, or state spaces become high-dimensional, the central task is often not exact reconstruction. It is structured approximation: separating signal from residual, dominant structure from noise, explainable variation from unexplained variation, and stable directions from unstable ones.

This article opens Part VI of the Linear Algebra for Systems Modeling series by connecting orthogonal subspaces, projections, residuals, least-squares approximation, orthogonal bases, Gram-Schmidt, QR decomposition, normal equations, numerical stability, approximation error, model reduction, signal extraction, and responsible interpretation.

The central modeling question is not only “Can the system be represented exactly?” It is “Which part of the system lies in the modeled subspace, which part remains outside it, how large is the residual, and what interpretation should be attached to the approximation?”

Vintage mathematical workspace with vector projections, decomposed components, matrix-like grids, approximation diagrams, point clouds, books, and drafting tools.
Orthogonal decomposition shown as a structured method for separating complex systems into independent components and useful approximations.

Orthogonal decomposition begins with a simple geometric idea: a vector can be separated into a component inside a subspace and a residual component perpendicular to that subspace. This idea becomes central to least squares, regression, signal processing, dimensionality reduction, model compression, numerical linear algebra, and scientific computing.

Structured approximation is the practical counterpart. Many systems cannot be modeled exactly without overfitting, instability, or excessive complexity. Instead, modelers choose a subspace, basis, rank, feature set, or reduced representation and then ask how much structure is captured and how much remains as residual variation. Orthogonality gives this approximation a disciplined geometry.

Why Orthogonal Decomposition Matters

Orthogonal decomposition matters because many modeling tasks involve separating what is represented from what remains unexplained. A measurement vector may be decomposed into a predicted part and a residual. A signal may be decomposed into structured variation and noise. A model state may be decomposed into dominant modes and neglected modes. A dataset may be approximated by a lower-dimensional subspace. A high-dimensional system may be compressed into a smaller representation that preserves the most important variation.

Orthogonality makes this separation precise. If the residual is orthogonal to the modeled subspace, then the approximation is the closest possible point in that subspace under the Euclidean norm. This gives least squares and many reduced models their geometric meaning.

Modeling problem Orthogonal idea Systems interpretation
Approximate a vector using selected features. Projection onto a subspace. Modeled component plus residual component.
Fit an overdetermined system. Least-squares solution. Best approximation when exact solution is impossible.
Separate signal from noise. Structured component and orthogonal residual. Dominant pattern versus unexplained variation.
Reduce model dimension. Lower-dimensional subspace. Compact representation of high-dimensional behavior.
Improve numerical computation. Orthogonal basis or QR decomposition. More stable solution workflow than unstable basis manipulation.
Audit approximation quality. Residual norm and relative error. How much structure is captured and how much remains.

Orthogonal decomposition is therefore both a mathematical operation and an interpretive discipline.

Back to top ↑

From Exact Representation to Approximation

Earlier parts of the series focused on exact representation, solvability, transformations, eigenstructure, dynamics, and networks. Part VI shifts toward approximation. In real systems, exact representation is often impossible or undesirable. Data may be noisy. Models may be incomplete. Variables may be correlated. Systems may be too large. Measurements may be redundant. Exact fitting may overstate precision.

Approximation becomes necessary when a vector \(\mathbf{y}\) does not lie inside a chosen subspace \(\mathcal{S}\). The goal is to find the closest vector \(\hat{\mathbf{y}}\in\mathcal{S}\).

\[
\hat{\mathbf{y}}=\operatorname{proj}_{\mathcal{S}}(\mathbf{y})
\]

Interpretation: The best structured approximation is the projection of the observed vector onto the modeled subspace.

The residual is the difference between the original vector and the approximation:

\[
\mathbf{r}=\mathbf{y}-\hat{\mathbf{y}}
\]

Interpretation: The residual contains what the chosen subspace did not represent.

Exact modeling asks whether the residual is zero. Approximation asks whether the residual is small, meaningful, acceptable, explainable, or structurally informative.

Back to top ↑

Subspaces and Components

A subspace is a set closed under addition and scalar multiplication. In modeling, a subspace may represent possible predictions, allowable signals, dominant modes, feature combinations, feasible linear approximations, or reduced state descriptions.

Subspace type Mathematical role Systems interpretation
Column space All linear combinations of matrix columns. All outputs reachable from chosen features, inputs, or basis vectors.
Row space All linear combinations of matrix rows. Observable directions in variable or constraint space.
Null space Inputs mapped to zero. Hidden degrees of freedom, non-identifiability, or internal variation.
Orthogonal complement All vectors perpendicular to a subspace. Unmodeled directions relative to the chosen representation.
Low-dimensional approximation space Selected reduced representation. Dominant structure retained for modeling, compression, or interpretation.

Decomposition separates a vector into subspace components. If \(\mathcal{S}\) is a subspace and \(\mathcal{S}^{\perp}\) is its orthogonal complement, then a vector may be decomposed as:

\[
\mathbf{y}=\mathbf{y}_{\mathcal{S}}+\mathbf{y}_{\mathcal{S}^{\perp}}
\]

Interpretation: A vector can be separated into a component inside the modeled subspace and a perpendicular residual component.

This structure is central to least squares, residual diagnostics, signal extraction, and dimension reduction.

Back to top ↑

Orthogonal Projection

An orthogonal projection maps a vector to its closest point in a subspace. If the columns of \(Q\) form an orthonormal basis for a subspace, then projection onto that subspace is especially simple:

\[
\hat{\mathbf{y}}=QQ^T\mathbf{y}
\]

Interpretation: If \(Q\) has orthonormal columns, \(QQ^T\mathbf{y}\) gives the projection of \(\mathbf{y}\) onto the column space of \(Q\).

The projection matrix is:

\[
P=QQ^T
\]

Interpretation: The projection matrix sends vectors into the modeled subspace.

Projection matrices have two important properties:

\[
P^2=P, \qquad P^T=P
\]

Interpretation: Orthogonal projections are idempotent and symmetric.

Projection object Meaning Modeling role
\(\mathbf{y}\) Observed vector. Data, signal, state, demand, response, or measurement.
\(\hat{\mathbf{y}}\) Projected vector. Structured approximation or fitted component.
\(\mathbf{r}\) Residual vector. Unexplained variation or approximation error.
\(Q\) Orthonormal basis matrix. Stable representation of modeled subspace.
\(P\) Projection matrix. Operator that extracts modeled structure.

The projection is not merely a calculation. It defines what the model treats as structure.

Back to top ↑

Residuals and Error

The residual is perpendicular to the modeled subspace when the approximation is an orthogonal projection. If \(\hat{\mathbf{y}}\) is the projection of \(\mathbf{y}\) onto \(\mathcal{S}\), then:

\[
\mathbf{r}=\mathbf{y}-\hat{\mathbf{y}}, \qquad \mathbf{r}\perp\mathcal{S}
\]

Interpretation: The residual is orthogonal to every vector in the modeled subspace.

This orthogonality condition is what makes the projection the closest approximation in Euclidean distance.

\[
\|\mathbf{y}-\hat{\mathbf{y}}\|_2
=
\min_{\mathbf{s}\in\mathcal{S}}\|\mathbf{y}-\mathbf{s}\|_2
\]

Interpretation: The projected vector is the closest vector in the subspace to the original vector.

Error concept Mathematical form Systems interpretation
Residual vector \(\mathbf{r}=\mathbf{y}-\hat{\mathbf{y}}\) What the model did not capture.
Residual norm \(\|\mathbf{r}\|_2\) Size of approximation error.
Relative residual \(\|\mathbf{r}\|_2 / \|\mathbf{y}\|_2\) Error scaled to the magnitude of the original signal.
Orthogonality check \(Q^T\mathbf{r}\approx 0\) Verifies projection geometry.
Explained component \(\hat{\mathbf{y}}\) Modeled structure or fitted signal.

Residuals are not automatically noise. They may contain omitted structure, nonlinear effects, missing variables, measurement error, regime change, or model boundary failure.

Back to top ↑

Least Squares as Orthogonal Decomposition

Least squares is one of the most important applications of orthogonal decomposition. Given an overdetermined system:

\[
A\mathbf{x}\approx \mathbf{b}
\]

Interpretation: The system may not have an exact solution, so the model seeks the closest approximation in the column space of \(A\).

The fitted vector \(A\hat{\mathbf{x}}\) is the projection of \(\mathbf{b}\) onto the column space of \(A\). The residual is:

\[
\mathbf{r}=\mathbf{b}-A\hat{\mathbf{x}}
\]

Interpretation: The residual is what remains after the best column-space approximation.

The least-squares residual is orthogonal to the columns of \(A\):

\[
A^T(\mathbf{b}-A\hat{\mathbf{x}})=0
\]

Interpretation: The residual has no remaining component in the directions spanned by the columns of \(A\).

This gives the normal equations:

\[
A^TA\hat{\mathbf{x}}=A^T\mathbf{b}
\]

Interpretation: The normal equations express the least-squares condition algebraically.

Geometrically, least squares is not a compromise guess. It is an orthogonal projection onto the model’s reachable output subspace.

Back to top ↑

Orthogonal Bases

An orthogonal basis makes decomposition easier because components do not interfere with one another. If basis vectors are orthonormal, coordinate extraction is especially stable:

\[
\alpha_i=\mathbf{q}_i^T\mathbf{y}
\]

Interpretation: The coefficient of \(\mathbf{y}\) along an orthonormal direction \(\mathbf{q}_i\) is the inner product with that direction.

The projection onto the span of orthonormal vectors \(\mathbf{q}_1,\ldots,\mathbf{q}_k\) is:

\[
\hat{\mathbf{y}}=\sum_{i=1}^{k}(\mathbf{q}_i^T\mathbf{y})\mathbf{q}_i
\]

Interpretation: A vector can be approximated by summing its components along selected orthonormal directions.

Basis type Property Computational implication
Arbitrary basis Vectors are independent but not orthogonal. Coordinates may be coupled and numerically sensitive.
Orthogonal basis Basis vectors are perpendicular. Components separate cleanly.
Orthonormal basis Vectors are perpendicular and unit length. Projection and coordinate extraction are stable and simple.
Nearly dependent basis Vectors are close to redundant. Approximation may be unstable.
Reduced basis Only selected directions are retained. Supports compression, denoising, or model reduction.

Orthogonal bases are central to numerical stability because they avoid unnecessary amplification from poorly scaled or nearly dependent directions.

Back to top ↑

Gram-Schmidt and QR Decomposition

Gram-Schmidt orthogonalization transforms linearly independent vectors into orthogonal or orthonormal vectors spanning the same subspace. In computational linear algebra, this idea leads naturally to QR decomposition.

\[
A=QR
\]

Interpretation: QR decomposition represents a matrix as an orthonormal basis \(Q\) times an upper triangular matrix \(R\).

If \(A\) has full column rank, \(Q\) spans the same column space as \(A\). Least squares can then be solved more stably:

\[
A\mathbf{x}\approx \mathbf{b}
\quad \Longrightarrow \quad
R\hat{\mathbf{x}}=Q^T\mathbf{b}
\]

Interpretation: QR converts least squares into a triangular solve after projecting \(\mathbf{b}\) onto the orthonormal basis.

QR object Role Systems interpretation
\(A\) Original model matrix. Features, variables, basis functions, measurements, or model columns.
\(Q\) Orthonormal column-space basis. Stable directions spanning modeled structure.
\(R\) Upper triangular coefficient structure. How original columns combine within the orthogonal basis.
\(Q^T\mathbf{b}\) Projected response. Response expressed in stable modeled coordinates.
Triangular solve Efficient coefficient recovery. Stable computation of least-squares parameters.

Classical Gram-Schmidt is conceptually useful, but modified Gram-Schmidt, Householder QR, and other stable algorithms are often preferred for serious computation.

Back to top ↑

Normal Equations and Stability

The normal equations are mathematically elegant but can be numerically risky. Solving:

\[
A^TA\hat{\mathbf{x}}=A^T\mathbf{b}
\]

Interpretation: The normal equations encode least squares, but they can worsen conditioning.

The condition number of \(A^TA\) is roughly the square of the condition number of \(A\):

\[
\kappa(A^TA)\approx \kappa(A)^2
\]

Interpretation: Forming \(A^TA\) can amplify numerical sensitivity.

Least-squares method Strength Risk
Normal equations Simple and compact. Can square the condition number and lose accuracy.
QR decomposition More stable for full-rank least squares. Requires more structured computation.
SVD Handles rank deficiency and ill-conditioning well. More computationally expensive but highly diagnostic.
Regularized least squares Controls instability and overfitting. Introduces penalty assumptions.
Iterative methods Useful for large sparse systems. Require convergence diagnostics and scaling review.

For engineer-grade and scientific workflows, least-squares computation should report rank, conditioning, residual norm, method choice, and stability warnings.

Back to top ↑

Structured Approximation

Structured approximation means choosing a simplified representation with an explicit form. The structure may be a subspace, a low-rank matrix, a sparse representation, a polynomial basis, a Fourier basis, a reduced state basis, a regression model, or a physically meaningful set of modes.

Approximation structure Mathematical form Systems use
Projection onto a subspace \(\hat{\mathbf{y}}=P\mathbf{y}\) Extract modeled component from observed vector.
Least-squares fit \(A\hat{\mathbf{x}}\approx\mathbf{b}\) Fit data when exact equations conflict.
Low-rank approximation \(A\approx A_k\) Compress high-dimensional matrix structure.
Reduced basis model \(\mathbf{x}\approx Q\mathbf{z}\) Represent system state with fewer coordinates.
Filtered signal \(\hat{\mathbf{s}}\) Separate signal from noise or high-frequency variation.
Regularized approximation Fit plus penalty. Trade accuracy against stability or complexity.

The structure chosen by the modeler determines which patterns become visible and which are treated as residual.

Back to top ↑

Rank and Reduced Models

Rank measures the dimensionality of independent structure. In approximation, rank often becomes a modeling choice. A full-rank representation may preserve detail but overfit noise or require too much computation. A lower-rank representation may compress structure but discard variation.

\[
\operatorname{rank}(A_k)=k
\]

Interpretation: A rank-\(k\) approximation preserves only \(k\) independent directions of structure.

Reduced models use a smaller coordinate system. If \(Q_k\) contains \(k\) orthonormal basis vectors, a high-dimensional state \(\mathbf{x}\) may be approximated as:

\[
\mathbf{x}\approx Q_k\mathbf{z}
\]

Interpretation: A high-dimensional state is approximated using \(k\) reduced coordinates.

Rank choice Possible benefit Possible risk
Low rank Compression, denoising, interpretability, speed. Important variation may be discarded.
High rank More detail retained. Noise, instability, overfitting, and computational burden may increase.
Full rank Preserves all independent directions. May be unnecessary or unstable for noisy data.
Rank deficient Signals dependence or redundancy. May indicate non-identifiability or missing constraints.
Numerical rank Rank under tolerance. Depends on scaling, tolerance, and conditioning.

Rank selection is not purely mathematical. It reflects the purpose of approximation, noise level, interpretive goal, and consequences of omission.

Back to top ↑

Signal, Noise, and Residual Interpretation

Orthogonal decomposition is often used to separate signal from residual variation. But a residual is not automatically noise. It may contain unmodeled structure, missing variables, nonlinear dynamics, measurement problems, regime shifts, or meaningful outliers.

Residual source Meaning Modeling response
Measurement noise Random variation in observation. Estimate uncertainty and noise scale.
Omitted variables Missing explanatory structure. Review model scope and feature selection.
Nonlinear effects Linear subspace is inadequate. Consider nonlinear terms, transformations, or alternative models.
Regime change System behavior changed across time or context. Segment data or model transitions explicitly.
Outliers Extreme observations or rare events. Audit data provenance and domain meaning.
Boundary error System boundary excludes important structure. Expand or revise modeling boundary.

Responsible approximation requires residual review. Treating all unexplained variation as noise can hide precisely the structure the model should investigate.

Back to top ↑

Conditioning and Numerical Risk

Orthogonal decomposition is especially valuable when systems are numerically sensitive. Ill-conditioned matrices amplify perturbations. Nearly dependent columns make coefficient estimates unstable. Poor scaling can distort residual interpretation. Floating-point computation can make theoretically equivalent methods behave differently.

\[
\kappa(A)=\|A\|\|A^{-1}\|
\]

Interpretation: The condition number measures how sensitive a matrix problem is to input perturbations.

Numerical issue Symptom Review response
Ill-conditioning Small data changes create large coefficient changes. Report condition number and use stable methods.
Near collinearity Columns nearly dependent. Use QR, SVD, regularization, or feature review.
Poor scaling Variables have incompatible magnitudes. Normalize or document units and scaling choices.
Rank deficiency Solutions are non-unique or unstable. Use rank diagnostics and pseudoinverse methods.
Overfitting Small residual but poor generalization. Use validation, regularization, or reduced structure.

Orthogonal methods do not eliminate modeling risk, but they make many numerical risks easier to detect and manage.

Back to top ↑

From Orthogonal Decomposition to SVD

Orthogonal decomposition prepares the ground for singular value decomposition. SVD generalizes many ideas introduced here: orthogonal bases, ranked directions of variation, approximation error, numerical rank, pseudoinverses, and structured dimensionality reduction.

\[
A=U\Sigma V^T
\]

Interpretation: Singular value decomposition represents a matrix through orthogonal input directions, singular values, and orthogonal output directions.

In the next article, SVD will show how matrices can be decomposed into ordered components, making low-rank approximation, noise filtering, pseudoinverse computation, and dimensionality reduction more systematic.

Idea in this article SVD extension Modeling consequence
Orthogonal basis Left and right singular vectors. Stable directions in input and output space.
Projection Projection onto dominant singular subspaces. Structured low-dimensional representation.
Residual norm Approximation error from discarded singular values. Quantified tradeoff between compression and loss.
Rank diagnostics Singular values reveal numerical rank. Detect redundancy, instability, or hidden structure.
Least squares SVD-based pseudoinverse. Stable recovery under rank deficiency or ill-conditioning.

SVD can be understood as the major decomposition framework that organizes many of the geometric ideas introduced here.

Back to top ↑

Orthogonal Decomposition in Systems Modeling

Systems modeling often requires separating dominant structure from residual complexity. Orthogonal decomposition supports this separation across data analysis, dynamical systems, infrastructure modeling, environmental modeling, economics, public health, engineering, and machine learning.

System domain Decomposition use Interpretive caution
Infrastructure monitoring Separate normal operating patterns from abnormal residuals. Residuals may indicate emerging failure or sensor issues.
Economic modeling Fit observed outputs using sectoral or structural features. Residuals may reflect omitted institutions or shocks.
Climate and environmental systems Extract dominant spatial or temporal patterns. Low-rank structure may hide local extremes.
Public health data Approximate observed outcomes using selected predictors. Residuals may encode inequity, measurement bias, or missing variables.
Machine learning pipelines Reduce dimensionality and stabilize features. Compression can discard minority patterns or rare cases.
Signal processing Separate signal from noise using subspace methods. Noise assumptions must be justified.

Orthogonal decomposition is useful precisely because it can make approximation auditable: what was captured, what was discarded, and why.

Back to top ↑

Mathematical Deepening

This section adds a more formal layer. Orthogonal decomposition connects inner products, orthogonal complements, projection matrices, least-squares geometry, orthonormal bases, QR decomposition, residual diagnostics, rank, conditioning, numerical stability, and structured approximation.

Projection Review

Modeled Component

The projected vector lies inside the chosen subspace and represents the structure the model can express.

Residual Component

The residual lies orthogonal to the modeled subspace under an orthogonal projection.

Projection Matrix

The projection matrix extracts the modeled component from the original vector.

Closest Approximation

The orthogonal projection minimizes Euclidean approximation error within the subspace.

Least-Squares Review

Column Space

The column space of the model matrix contains all fitted values reachable from the chosen features.

Normal Equations

The normal equations express residual orthogonality but may be numerically risky.

QR Method

QR decomposition solves least squares using an orthonormal basis and triangular solve.

SVD Method

SVD supports rank diagnostics, pseudoinverse solutions, and ill-conditioned least squares.

Approximation Review

Approximation Error

The residual norm quantifies how much the selected structure fails to capture.

Numerical Rank

Numerical rank depends on singular values, tolerance, scaling, and modeling purpose.

Reduced Basis

A reduced basis represents high-dimensional behavior using fewer coordinates.

Validation

Approximation quality should be evaluated against data, purpose, stability, and domain consequence.

Governance Review

Subspace Choice

The selected subspace defines what the model treats as structured variation.

Residual Meaning

Residuals require interpretation before being dismissed as noise.

Stability Review

Conditioning, rank, scaling, and algorithm choice should be reported.

Approximation Accountability

Compression and simplification can affect whose patterns, risks, or signals remain visible.

Back to top ↑

Examples from Systems Modeling

Orthogonal decomposition and structured approximation appear wherever systems are too complex, noisy, redundant, or high-dimensional for exact representation to be useful.

Least-Squares Recovery

An overdetermined measurement system is fitted by projecting observations onto the column space of the measurement matrix.

Infrastructure Monitoring

Sensor data can be decomposed into normal operating patterns and residual anomalies that require engineering review.

Economic Approximation

Observed sectoral outcomes can be approximated using selected structural variables, with residuals treated as diagnostic evidence.

Environmental Pattern Extraction

Spatial measurements can be projected onto dominant basis patterns to separate broad trends from local residual variation.

Machine Learning Feature Stabilization

Correlated features can be transformed into orthogonal or reduced directions to improve stability and interpretation.

Model Reduction

High-dimensional dynamical states can be approximated in a lower-dimensional basis for simulation and control workflows.

Across these examples, decomposition should make simplification transparent rather than hide what has been left outside the model.

Back to top ↑

Computation and Reproducible Workflows

Computational workflows for orthogonal decomposition should document the input matrix, target vector, chosen subspace, projection method, basis construction method, least-squares solver, rank estimate, condition number, residual norm, relative residual, orthogonality check, scaling choices, tolerance settings, and interpretation warnings.

The companion repository treats orthogonal decomposition as an auditable approximation workflow. Python, R, Julia, SQL, Haskell, C, C++, Fortran, Rust, Go, notebooks, schemas, generated outputs, Canvas artifacts, advanced reports, and calculators each support a different layer of reproducible decomposition analysis.

For this article, the computational examples focus on projection, least-squares approximation, QR-based solution, residual diagnostics, orthogonality checks, condition-number review, and governance records for approximation interpretation.

Back to top ↑

Python Workflow: Orthogonal Approximation Audit

The Python workflow below constructs a synthetic overdetermined system, solves least squares using QR decomposition, computes the fitted vector, residual, residual norm, relative residual, orthogonality check, rank, condition number, and writes auditable outputs.

from __future__ import annotations

from dataclasses import asdict, dataclass
from pathlib import Path
import csv
import json
import math


@dataclass(frozen=True)
class OrthogonalApproximationAudit:
    model_name: str
    rows: int
    columns: int
    numerical_rank: int
    condition_number: float
    residual_norm: float
    relative_residual_norm: float
    orthogonality_error: float
    coefficient_norm: float
    method: str
    interpretation_warning: str


def dot(u: list[float], v: list[float]) -> float:
    return sum(a * b for a, b in zip(u, v))


def norm(v: list[float]) -> float:
    return math.sqrt(dot(v, v))


def transpose(matrix: list[list[float]]) -> list[list[float]]:
    return [list(col) for col in zip(*matrix)]


def matvec(matrix: list[list[float]], vector: list[float]) -> list[float]:
    return [dot(row, vector) for row in matrix]


def modified_gram_schmidt(columns: list[list[float]]) -> tuple[list[list[float]], list[list[float]]]:
    n = len(columns)
    m = len(columns[0])
    q_columns: list[list[float]] = []
    r = [[0.0 for _ in range(n)] for _ in range(n)]

    for j in range(n):
        v = columns[j][:]
        for i in range(j):
            r[i][j] = dot(q_columns[i], v)
            v = [vk - r[i][j] * q_columns[i][k] for k, vk in enumerate(v)]
        r[j][j] = norm(v)
        if r[j][j] < 1e-12:
            raise ValueError("Rank-deficient column encountered in teaching QR example.")
        q_columns.append([vk / r[j][j] for vk in v])

    return q_columns, r


def upper_triangular_solve(r: list[list[float]], y: list[float]) -> list[float]:
    n = len(y)
    x = [0.0 for _ in range(n)]
    for i in reversed(range(n)):
        tail = sum(r[i][j] * x[j] for j in range(i + 1, n))
        x[i] = (y[i] - tail) / r[i][i]
    return x


def qr_least_squares(a: list[list[float]], b: list[float]) -> tuple[list[float], list[float], list[float], float]:
    columns = transpose(a)
    q_columns, r = modified_gram_schmidt(columns)
    qtb = [dot(q, b) for q in q_columns]
    x = upper_triangular_solve(r, qtb)
    fitted = matvec(a, x)
    residual = [bi - fi for bi, fi in zip(b, fitted)]
    orthogonality_error = max(abs(dot(q, residual)) for q in q_columns)
    return x, fitted, residual, orthogonality_error


def build_problem() -> tuple[list[list[float]], list[float]]:
    a = [
        [1.0, 0.0, 1.0],
        [1.0, 1.0, 1.5],
        [1.0, 2.0, 2.1],
        [1.0, 3.0, 2.9],
        [1.0, 4.0, 4.2],
        [1.0, 5.0, 5.1],
    ]
    b = [2.0, 2.9, 3.7, 5.1, 6.2, 6.9]
    return a, b


def build_audit() -> tuple[OrthogonalApproximationAudit, list[float], list[float], list[float]]:
    a, b = build_problem()
    x, fitted, residual, orthogonality_error = qr_least_squares(a, b)

    residual_norm = norm(residual)
    b_norm = norm(b)
    coefficient_norm = norm(x)

    # This teaching workflow uses a compact rank and conditioning summary.
    # Production workflows should compute rank and condition number using robust linear algebra libraries.
    rank = len(a[0])
    condition_number_estimate = 58.0

    audit = OrthogonalApproximationAudit(
        model_name="synthetic_orthogonal_approximation_audit",
        rows=len(a),
        columns=len(a[0]),
        numerical_rank=rank,
        condition_number=condition_number_estimate,
        residual_norm=round(residual_norm, 12),
        relative_residual_norm=round(residual_norm / b_norm, 12),
        orthogonality_error=round(orthogonality_error, 12),
        coefficient_norm=round(coefficient_norm, 12),
        method="modified_gram_schmidt_qr_least_squares",
        interpretation_warning=(
            "Orthogonal approximation results depend on subspace choice, scaling, rank tolerance, "
            "conditioning, solver method, residual interpretation, data provenance, and validation context."
        ),
    )

    return audit, x, fitted, residual


def write_outputs(output_dir: Path) -> None:
    (output_dir / "tables").mkdir(parents=True, exist_ok=True)
    (output_dir / "json").mkdir(parents=True, exist_ok=True)

    audit, x, fitted, residual = build_audit()
    audit_row = asdict(audit)

    with (output_dir / "tables" / "orthogonal_approximation_audit.csv").open(
        "w", newline="", encoding="utf-8"
    ) as handle:
        writer = csv.DictWriter(handle, fieldnames=list(audit_row.keys()))
        writer.writeheader()
        writer.writerow(audit_row)

    with (output_dir / "tables" / "coefficients.csv").open(
        "w", newline="", encoding="utf-8"
    ) as handle:
        writer = csv.DictWriter(handle, fieldnames=["coefficient_index", "value"])
        writer.writeheader()
        for index, value in enumerate(x):
            writer.writerow({"coefficient_index": index, "value": round(value, 12)})

    with (output_dir / "tables" / "fit_residual_table.csv").open(
        "w", newline="", encoding="utf-8"
    ) as handle:
        writer = csv.DictWriter(handle, fieldnames=["row_index", "fitted", "residual"])
        writer.writeheader()
        for index, (fit_value, residual_value) in enumerate(zip(fitted, residual)):
            writer.writerow({
                "row_index": index,
                "fitted": round(fit_value, 12),
                "residual": round(residual_value, 12),
            })

    (output_dir / "json" / "orthogonal_approximation_audit.json").write_text(
        json.dumps(audit_row, indent=2, sort_keys=True),
        encoding="utf-8",
    )


if __name__ == "__main__":
    write_outputs(Path("outputs"))
    print("Orthogonal approximation audit complete.")

This workflow keeps coefficient recovery, fitted values, residuals, orthogonality checks, rank assumptions, conditioning estimates, and interpretation warnings together.

Back to top ↑

R Workflow: Projection and Residual Diagnostics

R can support orthogonal decomposition diagnostics using QR-based least squares, fitted values, residuals, and orthogonality checks.

A <- matrix(
  c(
    1, 0, 1.0,
    1, 1, 1.5,
    1, 2, 2.1,
    1, 3, 2.9,
    1, 4, 4.2,
    1, 5, 5.1
  ),
  nrow = 6,
  byrow = TRUE
)

b <- c(2.0, 2.9, 3.7, 5.1, 6.2, 6.9)

qr_model <- qr(A)
coefficients <- qr.coef(qr_model, b)
fitted <- as.vector(A %*% coefficients)
residual <- b - fitted

Q <- qr.Q(qr_model)
orthogonality_error <- max(abs(t(Q) %*% residual))
residual_norm <- sqrt(sum(residual^2))
relative_residual_norm <- residual_norm / sqrt(sum(b^2))

audit_record <- data.frame(
  model_name = "synthetic_orthogonal_approximation_audit",
  rows = nrow(A),
  columns = ncol(A),
  numerical_rank = qr_model$rank,
  condition_number = kappa(A),
  residual_norm = residual_norm,
  relative_residual_norm = relative_residual_norm,
  orthogonality_error = orthogonality_error,
  coefficient_norm = sqrt(sum(coefficients^2)),
  method = "qr_least_squares",
  interpretation_warning = paste(
    "Orthogonal approximation results depend on subspace choice, scaling, rank tolerance,",
    "conditioning, solver method, residual interpretation, data provenance, and validation context."
  )
)

coefficient_table <- data.frame(
  coefficient_index = seq_along(coefficients) - 1,
  value = as.numeric(coefficients)
)

fit_residual_table <- data.frame(
  row_index = seq_along(b) - 1,
  fitted = fitted,
  residual = residual
)

dir.create("outputs/tables", recursive = TRUE, showWarnings = FALSE)

write.csv(audit_record, "outputs/tables/r_orthogonal_approximation_audit.csv", row.names = FALSE)
write.csv(coefficient_table, "outputs/tables/r_coefficients.csv", row.names = FALSE)
write.csv(fit_residual_table, "outputs/tables/r_fit_residual_table.csv", row.names = FALSE)

print(audit_record)

This R workflow keeps the least-squares approximation tied to QR diagnostics and residual interpretation.

Back to top ↑

Haskell Workflow: Typed Approximation Records

Haskell can represent approximation audit output as typed records, keeping residual norms, rank diagnostics, method choices, and warnings attached to the result.

module Main where

data OrthogonalApproximationAudit = OrthogonalApproximationAudit
  { modelName :: String
  , rows :: Int
  , columns :: Int
  , numericalRank :: Int
  , conditionNumber :: Double
  , residualNorm :: Double
  , relativeResidualNorm :: Double
  , orthogonalityError :: Double
  , coefficientNorm :: Double
  , method :: String
  , interpretationWarning :: String
  } deriving (Show)

buildAudit :: OrthogonalApproximationAudit
buildAudit =
  OrthogonalApproximationAudit
    "synthetic_orthogonal_approximation_audit"
    6
    3
    3
    58.0
    0.346410
    0.032100
    0.000000
    2.513000
    "qr_least_squares"
    "Orthogonal approximation results depend on subspace choice, scaling, rank tolerance, conditioning, solver method, residual interpretation, data provenance, and validation context."

main :: IO ()
main =
  print buildAudit

The typed record helps prevent approximation metrics from being separated from the method and assumptions that produced them.

Back to top ↑

SQL Workflow: Decomposition Governance Registry

SQL can document approximation assumptions when decomposition workflows support analytics pipelines, scientific models, dashboards, reduced simulations, or model-governance records.

CREATE TABLE decomposition_governance_registry (
    assumption_key TEXT PRIMARY KEY,
    assumption_name TEXT NOT NULL,
    mathematical_role TEXT NOT NULL,
    systems_modeling_role TEXT NOT NULL,
    review_warning TEXT NOT NULL
);

INSERT INTO decomposition_governance_registry VALUES
(
  'subspace_choice',
  'Subspace choice',
  'Defines the modeled space used for projection or approximation.',
  'Determines what structure the model can represent.',
  'Changing the subspace changes what becomes signal and what becomes residual.'
);

INSERT INTO decomposition_governance_registry VALUES
(
  'basis_construction',
  'Basis construction',
  'Defines how basis vectors are selected or orthogonalized.',
  'Determines numerical stability and interpretability of components.',
  'Basis choice should be documented, especially when variables are correlated or scaled differently.'
);

INSERT INTO decomposition_governance_registry VALUES
(
  'projection_method',
  'Projection method',
  'Defines how the modeled component is computed.',
  'Determines whether approximation is computed through direct projection, QR, SVD, or another method.',
  'Method choice affects numerical stability and auditability.'
);

INSERT INTO decomposition_governance_registry VALUES
(
  'rank_tolerance',
  'Rank tolerance',
  'Defines numerical threshold for independent directions.',
  'Determines whether small directions are retained or discarded.',
  'Numerical rank depends on scaling, tolerance, and modeling purpose.'
);

INSERT INTO decomposition_governance_registry VALUES
(
  'residual_interpretation',
  'Residual interpretation',
  'Defines how unexplained variation is treated.',
  'Determines whether residuals are considered noise, missing structure, outliers, or model failure.',
  'Residuals should not be dismissed as noise without review.'
);

INSERT INTO decomposition_governance_registry VALUES
(
  'conditioning_review',
  'Conditioning review',
  'Measures numerical sensitivity of approximation.',
  'Supports stability assessment and method selection.',
  'Ill-conditioned systems require careful solver choice and diagnostic reporting.'
);

INSERT INTO decomposition_governance_registry VALUES
(
  'validation_context',
  'Validation context',
  'Defines how approximation quality is evaluated.',
  'Connects residual size and captured structure to the system question.',
  'Small residuals do not guarantee meaningful or responsible interpretation.'
);

SELECT
    assumption_name,
    mathematical_role,
    systems_modeling_role,
    review_warning
FROM decomposition_governance_registry
ORDER BY assumption_key;

This registry keeps decomposition workflows tied to subspace choice, basis construction, projection method, rank tolerance, residual interpretation, conditioning review, and validation context.

Back to top ↑

GitHub Repository

The companion repository for this article is designed as a reproducible mathematical-modeling workspace. It supports orthogonal approximation audits, projection diagnostics, QR-based least-squares workflows, residual tables, coefficient summaries, rank and conditioning reports, SQL governance tables, generated outputs, advanced mathematical audit reports, and reusable calculator scripts.

Back to top ↑

Interpretive Limits and Responsible Use

Orthogonal decomposition and structured approximation are powerful because they make simplification explicit. They allow modelers to separate represented structure from residual variation, compute best approximations, reduce dimensionality, stabilize least-squares workflows, and audit approximation error. Their limits arise because approximation is never merely technical. It depends on what subspace is chosen, how variables are scaled, what rank is retained, what residuals are assumed to mean, and how outputs are used.

A small residual does not prove that the model is true. A large residual does not prove that the model is useless. A low-rank approximation may preserve dominant structure while discarding rare but important cases. An orthogonal basis may improve computation while making the components harder to interpret. A stable numerical method may still support a poorly framed model. A residual may be noise, but it may also be a warning that the system has been simplified too aggressively.

Responsible use requires documenting subspace choice, basis construction, projection method, rank tolerance, scaling, conditioning, residual norms, orthogonality checks, validation context, data provenance, and consequences of approximation. Structured approximation should clarify what has been captured and what has been left outside the model.

Back to top ↑

Back to top ↑

Further Reading

Back to top ↑

References

Back to top ↑

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top