Physics-Informed Machine Learning and Scientific Computing

Last Updated May 28, 2026

Physics-informed machine learning and scientific computing bring physical law, numerical approximation, experimental evidence, differentiable programming, and uncertainty-aware inference into a shared computational framework for studying complex physical systems. Classical scientific computing usually begins with equations, discretizes them, and solves the resulting numerical problem. Conventional machine learning usually begins with data, fits a predictive model, and evaluates generalization. Physics-informed machine learning sits between these traditions. It uses equations where equations are known, data where data are available, constraints where physical structure matters, and trainable approximation where closed-form solution or direct simulation becomes too costly.

The field has become important because many contemporary physics problems are neither purely equation-rich nor purely data-rich. Physical systems may have partial governing equations, unknown constitutive laws, limited sensors, noisy observations, expensive simulations, hidden states, uncertain boundary conditions, multiscale dynamics, high-dimensional parameters, or model-form errors that cannot be removed by simply increasing computational power. Physics-informed machine learning offers a disciplined way to combine conservation laws, differential equations, symmetries, boundary conditions, dimensional structure, operator maps, experimental measurements, and numerical solvers into auditable trainable models.

This article develops Physics-Informed Machine Learning and Scientific Computing as a research-grade article within the Physics knowledge series. It explains physics-informed neural networks, scientific machine learning, neural ordinary differential equations, universal differential equations, differentiable simulators, neural operators, Fourier neural operators, DeepONets, surrogate modeling, reduced-order modeling, inverse problems, data assimilation, conservation constraints, dimensional analysis, PDE residual losses, automatic differentiation, adjoint sensitivity, uncertainty quantification, identifiability, optimization pathologies, verification, validation, reproducibility, and scientific software workflows. Selected R and Python examples appear in the article body, while the companion GitHub repository extends the same logic into residual diagnostics, neural ODE examples, differentiable-solver examples, operator-learning datasets, surrogate-model validation, inverse-parameter estimation, conservation-law checks, uncertainty workflows, SQL provenance tables, C/C++/Fortran/Rust examples, and reproducible scientific machine learning resources.

Why Physics-Informed Machine Learning Matters

Physics-informed machine learning matters because modern physics increasingly confronts systems where neither classical simulation nor data-only machine learning is sufficient on its own. Classical numerical solvers are powerful when governing equations, boundary conditions, material laws, geometry, parameters, and forcing terms are known. But many physical systems are only partly observed, partly modeled, nonlinear, multiscale, noisy, or too expensive to simulate repeatedly. Data-only machine learning can identify empirical patterns, but it may violate conservation laws, ignore units, fail outside the training distribution, or produce predictions that look plausible while contradicting known physics.

Scientific machine learning addresses this gap by treating physical structure as an inductive bias. Instead of asking a model to learn everything from data, one can require it to satisfy a differential equation, conserve mass or energy, obey a boundary condition, respect symmetry, estimate unknown parameters, preserve dimensional consistency, or learn only the unresolved part of a model. The learning problem becomes constrained by physical reasoning rather than detached from it.

This matters across fluid dynamics, climate modeling, plasma physics, materials science, molecular dynamics, biomechanics, geophysics, astrophysics, quantum systems, medical physics, turbulence, structural mechanics, chemical kinetics, and inverse design. It also matters for scientific accountability. A physics-informed model can be inspected through residuals, conserved quantities, parameter estimates, error metrics, scaling assumptions, validation tests, and uncertainty summaries rather than judged only by predictive accuracy.

For the Physics knowledge series, this article belongs near Numerical Methods in Physics, Computational Physics and Scientific Simulation, Experimental Physics: Measurement, Noise, Calibration, and Inference, Nonequilibrium Statistical Mechanics, Fluid Dynamics and Continuum Mechanics, Quantum Information, Decoherence, and Measurement, and Scattering Theory, Cross Sections, and Physical Inference. It provides the bridge between numerical physics, machine learning, inverse inference, and reproducible scientific computing.

Back to top ↑

From Numerical Methods to Scientific Machine Learning

Traditional numerical methods approximate equations directly. A finite difference scheme approximates derivatives on a grid. A finite volume method enforces conservation over control volumes. A finite element method solves a weak form over a mesh. A spectral method expands fields in global or local basis functions. These methods are explicit about discretization, stability, consistency, convergence, boundary enforcement, and numerical error.

Machine learning approximates functions from data. A neural network represents a parameterized function:

Here \(\theta\) denotes trainable parameters. Training adjusts \(\theta\) to minimize a loss function. In ordinary supervised learning, the loss often measures mismatch between predictions and observed labels:

Scientific machine learning modifies this logic. The loss may include data mismatch, PDE residuals, boundary residuals, conservation violations, measurement likelihoods, regularization terms, and prior physical information. The model is not only asked to fit data. It is asked to fit physics.

This does not eliminate numerical analysis. It expands it. A physics-informed model still has approximation error, optimization error, generalization error, discretization error, floating-point error, sampling error, and model-form error. The difference is that some of the approximation may be learned rather than manually specified through a classical discretization. A rigorous workflow therefore has to ask both machine-learning questions and numerical-analysis questions: What is the training loss? What is the physical residual? How are derivatives computed? How are boundary conditions enforced? What is the domain of validity? How do errors behave under refinement, resampling, and independent validation?

Back to top ↑

What “Physics-Informed” Means

The phrase “physics-informed” can mean several related but distinct things. A model may be physics-informed because its loss function penalizes violations of a differential equation. It may be physics-informed because its architecture enforces conservation, symmetry, monotonicity, positivity, causality, or equivariance. It may be physics-informed because a neural network augments a mechanistic model rather than replacing it. It may be physics-informed because training data are generated by trusted simulators. It may be physics-informed because predictions are constrained by dimensional analysis or nondimensional groups.

These forms are not equivalent. A neural network with a PDE residual penalty is different from a finite-volume neural architecture that exactly conserves mass. A surrogate trained on simulation snapshots is different from a universal differential equation that learns an unknown forcing term inside a solver. A neural operator trained to map initial conditions to entire solution fields is different from a PINN trained to solve one PDE instance.

Precision matters because “physics-informed” should not become a decorative label. A rigorous scientific machine learning workflow should specify what physical information is included, how it is included, what is enforced exactly, what is penalized approximately, what is learned, what is assumed, what is nondimensionalized, what is validated, and how failure is tested. A model is not scientifically strong because it contains a neural network. It is scientifically strong when its assumptions, constraints, residuals, uncertainties, and limitations are visible enough to be examined.

Back to top ↑

A Taxonomy of Physics-Informed Models

Physics-informed machine learning is not a single algorithm. It is a family of modeling strategies that combine physical knowledge and trainable approximation in different ways. A practical taxonomy helps prevent confusion.

This taxonomy also clarifies what should be tested. PINNs require residual, boundary, and collocation diagnostics. Neural operators require solution-map validation across parameter regimes. Universal differential equations require model-discrepancy analysis and learned-term interpretation. Differentiable simulators require gradient checks and solver validation. Structure-preserving networks require tests of the structure they claim to preserve.

Back to top ↑

Physics-Informed Neural Networks

A physics-informed neural network, or PINN, represents a physical field with a neural network and trains it to satisfy governing equations and data constraints. Suppose a field \(u(x,t)\) satisfies a differential equation:

A neural approximation is:

The PDE residual is:

The PINN loss often includes the mean squared residual at collocation points:

The total loss may include PDE residuals, initial conditions, boundary conditions, and observed data:

PINNs are attractive because they can use scattered collocation points, incorporate differential equations through automatic differentiation, handle inverse problems, and train from limited data. They also produce diagnostic objects: residual fields, boundary errors, inferred parameters, and constraint violations. But they are not universal replacements for classical solvers. Training can be difficult, loss terms can be imbalanced, high-frequency solutions can be hard to learn, stiff systems can be challenging, and error control is less mature than in classical numerical analysis.

A responsible PINN workflow therefore treats the neural network as one component of a scientific method, not as a solver that automatically inherits the reliability of the governing equation. The governing equation may be correct, but the approximation, training procedure, sampling scheme, derivative computation, scaling convention, and validation procedure still have to be examined.

Back to top ↑

PDE Residual Losses

The PDE residual is the core mathematical object in many physics-informed neural networks. Consider the one-dimensional heat equation:

A physics-informed model forms the residual:

Training minimizes this residual at sampled collocation points. If the residual is small across the domain and boundary and initial conditions are satisfied, the network approximates a solution to the PDE.

However, residual minimization is not the same as classical convergence. A small residual at sampled points does not automatically guarantee small error everywhere. Collocation sampling, network capacity, derivative accuracy, optimizer behavior, boundary enforcement, and scaling all affect the result. Residual diagnostics should therefore be paired with validation against analytic solutions, high-fidelity numerical solvers, conservation checks, held-out physical regimes, and sensitivity analysis.

This point is central. In a finite-difference, finite-volume, or finite-element setting, refinement studies and convergence theory provide a structured language for numerical reliability. In PINNs, the residual is often continuous in principle but sampled in practice. The scientific question is not only whether the training loss is small, but whether the learned field satisfies the physical problem under independent scrutiny.

Back to top ↑

Initial, Boundary, and Observation Losses

A PDE residual alone is usually insufficient. Physical problems require initial conditions, boundary conditions, interface conditions, forcing terms, material parameters, and sometimes observational data. For an initial condition:

The initial-condition loss can be written as:

For a boundary condition:

The boundary-condition loss can be written as:

For observed data:

The data loss is:

The weighting of these loss terms matters. If the PDE residual dominates, the model may ignore data. If data dominate, the model may violate physics. If boundary losses are weak, the solution may satisfy the interior equation while failing the physical problem. Loss balancing is therefore not an implementation detail; it is part of the scientific model.

Some workflows use fixed weights, while others use adaptive weighting, gradient normalization, curriculum learning, residual-based resampling, or multi-stage training. The choice should be documented because it changes the practical meaning of the learned solution. A total loss is not a neutral objective. It encodes a judgment about which errors matter most.

Back to top ↑

Automatic Differentiation and Differentiable Programming

Automatic differentiation computes derivatives of computational programs by applying the chain rule systematically. In physics-informed machine learning, this allows derivatives of neural-network outputs with respect to inputs and parameters to be computed directly. For example, a network \(u_\theta(x,t)\) can be differentiated to obtain:

These derivatives can then be inserted into differential-equation residuals. Automatic differentiation is one reason PINNs became practical: it allows a differentiable neural approximation to be constrained by differential operators without constructing finite-difference derivatives by hand.

Differentiable programming extends this idea to larger computational systems. A numerical solver, simulator, optimization routine, control system, or scientific workflow can be made differentiable so that gradients of outputs with respect to parameters can be computed. This enables parameter estimation, inverse design, neural differential equations, differentiable simulators, and hybrid physical-neural models.

But differentiability does not automatically mean correctness. Differentiating through a solver can be memory-intensive, unstable, or sensitive to discretization. Gradients can be inaccurate if solver tolerances are loose, if discontinuities exist, if stiffness is severe, if adjoint methods are mismatched to the numerical problem, or if the computational graph does not represent the intended mathematics. Differentiable physics still requires numerical analysis.

Back to top ↑

Neural ODEs and Universal Differential Equations

A neural ordinary differential equation uses a neural network to define or modify the right-hand side of an ODE:

The model output is obtained by integrating the differential equation. Training adjusts \(\theta\) so that the solution matches observed trajectories or downstream objectives. This differs from an ordinary feedforward network because the learned representation is mediated through an ODE solver. The solver becomes part of the model.

A universal differential equation combines known mechanistic structure with learned components:

Here \(f_{\mathrm{known}}\) represents trusted physics, while \(g_\theta\) learns missing dynamics, closure terms, unresolved forcing, unknown constitutive behavior, or model discrepancy.

This hybrid structure is often more scientifically meaningful than replacing the entire system with a black-box neural network. It asks the network to learn only what is unknown. It also provides a route toward interpretability: if \(g_\theta\) learns a structured correction, one can analyze when and why the mechanistic model fails.

The same caution still applies. A learned correction term may absorb sensor bias, numerical error, missing boundary conditions, or wrong parameter values rather than a genuine missing physical mechanism. Universal differential equations therefore require identifiability analysis, parameter sensitivity, validation against independent regimes, and careful separation between model discrepancy and measurement error.

Back to top ↑

Differentiable Simulation

Differentiable simulation asks whether a simulation workflow can be embedded inside optimization and inference. Instead of treating a simulator as a black box that only returns outputs, differentiable simulation exposes gradients of outputs with respect to inputs, parameters, controls, geometries, or material laws. This makes it possible to calibrate models, optimize designs, infer hidden states, and learn unresolved physical terms by propagating information through the simulation itself.

For example, a differentiable mechanics simulator may allow gradients of displacement with respect to stiffness parameters. A differentiable fluid solver may allow gradients of drag with respect to shape. A differentiable optics simulator may allow gradients of an imaging loss with respect to a lens or phase mask. A differentiable quantum simulator may allow gradients of an observable with respect to Hamiltonian parameters. In each case, the scientific problem becomes a constrained optimization problem over a physical process.

Differentiable simulation is powerful but fragile. Gradients may explode, vanish, or become meaningless when systems are chaotic, discontinuous, stiff, contact-rich, shock-dominated, or poorly resolved. A forward simulation may be physically plausible while its gradient is unreliable. For this reason, differentiable simulation should include gradient checks, finite-difference comparisons, solver-tolerance studies, adjoint consistency checks, and validation against non-differentiated baselines.

Back to top ↑

Neural Operators and Operator Learning

Many physics problems are not just function approximation problems. They are operator approximation problems. A PDE solution map may take an initial condition, coefficient field, boundary condition, geometry, or forcing function as input and return an entire solution field as output:

A neural operator attempts to learn this mapping between function spaces. Unlike a PINN that often solves one problem instance, a neural operator can learn a family of solution maps across many parameterized inputs.

DeepONets use separate networks to encode input functions and output locations. Fourier neural operators learn mappings using Fourier-domain representations. Graph neural operators can work on irregular meshes or graph-structured physical domains. These models are attractive for surrogate modeling, rapid simulation, uncertainty propagation, and inverse design.

Operator learning changes the computational question. Instead of asking, “Can we solve this PDE once?” it asks, “Can we learn the solution operator over a class of related PDE inputs?” This is powerful when many simulations are needed, but it requires careful training data design, out-of-distribution testing, resolution checks, conservation diagnostics, and uncertainty analysis.

Neural operators are especially important when the same physical model must be evaluated repeatedly: uncertainty quantification, Bayesian inverse problems, design optimization, ensemble forecasting, control, real-time digital twins, and multiscale coupling. Their value is not that they replace physical modeling, but that they can amortize expensive simulation across a family of related problems. Their danger is that they may appear accurate inside the training distribution while failing under new geometries, parameter regimes, forcing patterns, or boundary conditions.

Back to top ↑

Surrogate Models and Reduced-Order Modeling

A surrogate model approximates an expensive simulator, experiment, or physical map with a cheaper model. In physics, surrogates are used for parameter sweeps, uncertainty quantification, optimization, control, inverse problems, design exploration, and real-time prediction.

Reduced-order modeling traditionally uses methods such as proper orthogonal decomposition, Galerkin projection, balanced truncation, dynamic mode decomposition, and reduced basis methods. Machine learning adds neural surrogates, autoencoders, Gaussian processes, neural operators, and hybrid latent dynamics.

The risk is that a surrogate may interpolate well but extrapolate poorly. It may reproduce low-order statistics while violating conservation. It may be accurate on training parameters but fail under rare conditions. It may be fast but untrustworthy. A surrogate model should therefore be evaluated not only by prediction error, but by physical diagnostics, uncertainty bounds, domain-of-validity tests, and comparison with high-fidelity simulations or experiments.

A serious surrogate workflow should document the training data source, simulator settings, mesh or resolution, parameter ranges, nondimensional quantities, boundary conditions, validation regimes, extrapolation limits, conservation checks, uncertainty method, and versioned code. A surrogate that cannot be audited is not a scientific instrument; it is only a fitted approximation.

Back to top ↑

Inverse Problems and Parameter Discovery

Inverse problems infer hidden parameters, fields, sources, boundary conditions, constitutive laws, or governing equations from observations. Physics-informed learning is especially useful for inverse problems because data alone may be insufficient and physics alone may contain unknown terms.

Suppose a PDE depends on an unknown parameter \(\lambda\):

A physics-informed model can train both the field approximation and the parameter:

by minimizing residual and data losses. The parameter is learned because only certain values of \(\lambda\) make the observed data and governing equation consistent.

However, inverse problems can be nonidentifiable. Different parameter combinations may produce nearly identical observations. Sparse sensors may be insufficient. Noise may dominate the signal. Boundary conditions may be uncertain. A learned parameter can be precise but wrong if model-form error is ignored. Inverse physics-informed learning should therefore include sensitivity analysis, uncertainty quantification, identifiability checks, and validation on independent data.

Parameter discovery also raises a deeper scientific question: is the model discovering physics or merely fitting a convenient representation? A residual can be minimized under the wrong governing equation if data are sparse enough or if the learned approximation has enough flexibility. Scientific discovery requires independent tests, physical interpretability, dimensional consistency, and comparison against alternative hypotheses.

Back to top ↑

Data Assimilation and Hybrid Modeling

Data assimilation combines model predictions with observations to estimate system states. It is central to weather prediction, oceanography, climate modeling, geophysics, plasma control, robotics, and experimental physics. Classical methods include Kalman filters, ensemble Kalman filters, variational assimilation, particle filters, and smoothing methods.

Physics-informed machine learning can interact with data assimilation in several ways. A neural network may learn model error. A differentiable simulator may allow gradient-based assimilation. A neural operator may provide fast forecast surrogates. A learned closure may improve a coarse model. A Bayesian neural surrogate may quantify uncertainty in unobserved states.

The core idea is hybridization. Data corrects the model. Physics constrains the data interpretation. Machine learning supplies flexible approximation. Numerical simulation supplies causal structure. The strongest workflows use all four deliberately rather than treating them as competing approaches.

Hybrid modeling is especially important when observations are irregular, incomplete, or biased. A model can fill gaps, but only under assumptions. Data can correct the model, but only where measurements are informative. Machine learning can approximate unmodeled structure, but only within the domain it has learned. The purpose of scientific machine learning is not to erase these limitations. It is to make them computationally tractable and scientifically inspectable.

Back to top ↑

Conservation Laws and Structure-Preserving Learning

Many physical systems are governed by conservation laws:

A model that violates conservation may produce inaccurate long-time behavior even if short-time prediction error is small. Structure-preserving learning attempts to encode physical structure into the model architecture or training process. Examples include conservative neural networks, Hamiltonian neural networks, Lagrangian neural networks, symplectic neural integrators, equivariant networks, divergence-free fields, positivity-preserving models, and finite-volume-inspired architectures.

There is an important distinction between soft and hard constraints. A soft constraint penalizes conservation violation in the loss function. A hard constraint builds conservation into the representation. Hard constraints are often more reliable, but more difficult to design. Soft constraints are easier to add, but may fail when optimization or loss weighting is poor.

Structure preservation is also problem-specific. A Hamiltonian structure may be appropriate for conservative mechanics but inappropriate for dissipative systems unless dissipation is modeled separately. A positivity constraint may be essential for density, concentration, or probability, but not for signed wave fields. A symmetry may hold only under idealized conditions. The structure encoded into a model should correspond to the physical regime being studied, not merely to an elegant mathematical architecture.

Back to top ↑

Symmetry, Dimensional Analysis, and Inductive Bias

Symmetry is one of the most powerful sources of physical structure. A system may be invariant under translation, rotation, reflection, permutation, gauge transformation, Galilean transformation, Lorentz transformation, or scaling. Machine learning models that respect relevant symmetries can generalize more efficiently because they do not waste capacity learning equivalent configurations separately.

Dimensional analysis provides another form of structure. A model should not mix physical quantities in dimensionally invalid ways. Nondimensionalization can improve learning by scaling inputs and outputs to comparable magnitudes. Dimensionless groups such as Reynolds number, Peclet number, Mach number, Knudsen number, Damköhler number, and Courant number often organize families of physical behavior.

Inductive bias is not a weakness. In scientific machine learning, inductive bias is the point. Physical law tells the model what kinds of functions are plausible, what transformations should not matter, what quantities must be conserved, and what scales should control the dynamics.

For this reason, the best physics-informed models are often not the most flexible models. They are the models whose flexibility is constrained in the right way. A model that can represent anything may be attractive in ordinary prediction, but in physics an unconstrained model may learn artifacts, shortcuts, or physically impossible functions. Scientific learning requires disciplined flexibility.

Back to top ↑

Uncertainty, Identifiability, and Trust

Scientific machine learning must distinguish prediction from knowledge. A model may predict accurately on a test set while being uncertain in physically important regimes. It may estimate a parameter without proving the parameter is identifiable. It may fit noisy data while hiding model-form error. It may satisfy an equation residual while failing a physical validation test.

Uncertainty enters through measurement noise, sparse observations, unknown parameters, discretization error, optimizer convergence, neural-network approximation, training data bias, boundary conditions, model-form assumptions, and stochastic sampling. A responsible workflow should propagate at least the dominant uncertainties.

Bayesian neural networks, ensembles, Gaussian processes, dropout approximations, likelihood-based inference, profile likelihoods, adjoint sensitivity, bootstrapping, residual diagnostics, and calibration curves can all help, but uncertainty quantification must be tied to the physical question. A credible interval on a field prediction is not the same as uncertainty in a conserved flux, inferred parameter, instability threshold, or engineering decision.

Identifiability deserves special attention. If the available measurements cannot distinguish between two physical explanations, no neural architecture can make the problem identifiable by itself. It may produce a confident answer, but confidence is not evidence. A trustworthy workflow asks whether the data, equations, and boundary conditions contain enough information to support the inferred quantity.

Back to top ↑

Optimization Pathologies and Training Difficulty

Physics-informed models can be difficult to train. Loss landscapes may be stiff. PDE residuals may have gradients at very different scales from boundary losses. High-frequency components may be learned slowly. Nonlinear systems may generate sharp fronts, shocks, boundary layers, turbulence, or chaotic sensitivity. Collocation points may be poorly distributed. Automatic differentiation may produce unstable high-order derivatives. Optimizers may reduce the loss without improving the physical quantity of interest.

Common mitigation strategies include nondimensionalization, adaptive loss weighting, curriculum learning, residual-based adaptive sampling, domain decomposition, hard boundary constraints, multi-stage optimization, hybrid classical-neural solvers, normalization, better activation functions, Fourier features, operator learning, and validation against conventional numerical solvers.

The central principle is that training failure is not merely an engineering inconvenience. It is part of the numerical method. If a physics-informed model cannot be optimized reliably, then it is not yet a reliable solver or inference tool for that problem.

Optimization diagnostics should therefore be recorded as scientific evidence. Training curves, residual maps, boundary-condition errors, seed sensitivity, optimizer comparisons, learning-rate schedules, collocation strategies, and final validation metrics should be preserved. A model that cannot be reproduced cannot be trusted as a scientific computation.

Back to top ↑

Verification, Validation, and Reproducibility

Physics-informed machine learning should be evaluated with the same seriousness as numerical simulation and experimental inference. Verification asks whether the computational implementation solves the intended mathematical problem. Validation asks whether the model represents the physical system. Reproducibility asks whether the workflow can be inspected, rerun, and audited.

Useful diagnostics include:

• residual maps over the full domain, not only training points;
• boundary and initial-condition error summaries;
• conservation-law checks;
• comparison with analytic benchmark problems;
• comparison with trusted numerical solvers;
• held-out physical regimes;
• out-of-distribution parameter tests;
• uncertainty estimates;
• random seed and optimizer sensitivity;
• unit and nondimensionalization documentation;
• training data provenance;
• versioned code, configuration, and outputs.

A physics-informed model should be judged not by whether it uses a neural network, but by whether it produces trustworthy scientific evidence under known assumptions and limitations.

Reproducibility also requires infrastructure. Scientific machine learning workflows should preserve configuration files, software versions, random seeds, solver tolerances, hardware notes, dataset hashes, preprocessing scripts, validation cases, and output artifacts. Without this infrastructure, the model may be mathematically interesting but scientifically weak.

Back to top ↑

Measurement, Units, and SI Interpretation

Physics-informed machine learning frequently uses nondimensional inputs and outputs, but the original SI or physical units must remain documented. A temperature field, velocity field, pressure field, electromagnetic field, concentration field, displacement field, or probability density carries units. Loss terms may mix quantities with different dimensions, so nondimensionalization is often essential.

For example, the heat equation:

uses a diffusion coefficient with units:

If \(x\) is measured in meters and \(t\) in seconds, the two terms have consistent units. But if a neural network takes scaled coordinates:

then derivatives transform. A residual written in scaled variables must account for these scale factors. Otherwise, the model may minimize a mathematically inconsistent residual.

Unit-aware scientific machine learning is not optional. It is part of physical correctness. Training data, collocation points, parameters, solver outputs, and evaluation metrics should all preserve or explicitly transform units. A model that fits nondimensional arrays but loses track of the physical quantities they represent cannot support reliable scientific interpretation.

Back to top ↑

Mathematical Lens

A mathematics-first view begins with a physical equation:

A neural approximation is:

The physics residual is:

The physics-informed residual loss is:

Observation loss is:

A total PINN loss is:

For a neural ODE:

For a universal differential equation:

For operator learning:

For inverse parameter estimation:

For uncertainty-aware inference, one may write a posterior:

This mathematical lens shows that physics-informed machine learning is not a single algorithm. It is a family of constrained approximation, differentiable simulation, inverse inference, operator-learning, and uncertainty-aware modeling methods built around physical structure.

Back to top ↑

Variables, Units, and Physical Interpretation

Physics-informed machine learning uses variables that connect neural approximation, physical equations, model residuals, parameter inference, and operator learning. The table below summarizes several central quantities.

Back to top ↑

Worked Example: PINN for a Decay Equation

Consider the exponential decay equation:

with initial condition:

The analytic solution is:

A physics-informed neural network approximates:

The residual is:

The residual loss is:

The initial-condition loss is:

The total loss is:

If observed data are available, a data term can be added:

This simple example demonstrates the structure of a PINN without the complexity of a PDE. The neural network is not trained only to match labeled values. It is trained to satisfy a differential equation and an initial condition. The same logic extends to heat equations, wave equations, Navier–Stokes equations, Schrödinger equations, reaction-diffusion systems, and inverse parameter estimation.

Back to top ↑

Computational Modeling

Computational modeling makes physics-informed machine learning auditable. A residual-diagnostic workflow can evaluate whether a candidate field satisfies a PDE. A PINN workflow can train a neural approximation while tracking residual and boundary losses. A neural ODE workflow can learn unknown dynamics from trajectories. A universal differential equation workflow can learn missing terms in a mechanistic model. A neural-operator workflow can approximate solution maps across parameterized PDE families. An inverse-problem workflow can estimate unknown physical parameters with uncertainty. A VVUQ workflow can document scaling, seeds, training data, solver tolerances, optimizer settings, validation cases, and model limitations.

The selected examples below focus on residual diagnostics and a small PINN because they are foundational, readable, and directly reusable. The GitHub repository extends the same logic into richer computational resources: R residual diagnostics, Python PINNs, neural ODE examples, universal differential equation examples, operator-learning dataset generators, surrogate validation, inverse parameter estimation, conservation checks, uncertainty summaries, Julia SciML-style examples, C++ residual sweeps, Fortran finite-difference baselines, SQL scientific machine learning provenance, Rust command-line utilities, C examples, documentation, and reproducible sample data.

Back to top ↑

R Workflow: Physics-Informed Residual Diagnostics

R is useful for transparent residual diagnostics, validation tables, and reproducible model-audit summaries. The following workflow evaluates whether a candidate field satisfies the one-dimensional heat equation:

using finite differences on a manufactured analytic solution:

# Physics-Informed Residual Diagnostics for the Heat Equation
#
# This workflow evaluates the PDE residual:
#
#   r(x,t) = u_t - D u_xx
#
# for the one-dimensional heat equation:
#
#   u_t = D u_xx
#
# using a manufactured analytic solution:
#
#   u(x,t) = exp(-D*pi^2*t) * sin(pi*x)
#
# The exact residual should be zero. Numerical residuals are not exactly zero
# because derivatives are approximated on a finite grid.

library(tibble)
library(dplyr)

diffusion <- 0.1
n_x <- 101
n_t <- 101

x_grid <- seq(0, 1, length.out = n_x)
t_grid <- seq(0, 1, length.out = n_t)

dx <- x_grid[2] - x_grid[1]
dt <- t_grid[2] - t_grid[1]

field_table <- expand.grid(
  x = x_grid,
  t = t_grid
) %>%
  as_tibble() %>%
  mutate(
    u = exp(-diffusion * pi^2 * t) * sin(pi * x)
  )

# Convert to matrix with rows for x and columns for t.
u_matrix <- matrix(
  field_table$u,
  nrow = n_x,
  ncol = n_t,
  byrow = FALSE
)

# Compute central finite difference approximations for interior points.
residual_rows <- list()
row_index <- 1

for (i in 2:(n_x - 1)) {
  for (j in 2:(n_t - 1)) {
    u_t <- (u_matrix[i, j + 1] - u_matrix[i, j - 1]) / (2 * dt)

    u_xx <- (
      u_matrix[i + 1, j] -
        2 * u_matrix[i, j] +
        u_matrix[i - 1, j]
    ) / dx^2

    residual <- u_t - diffusion * u_xx

    residual_rows[[row_index]] <- tibble(
      x = x_grid[i],
      t = t_grid[j],
      u = u_matrix[i, j],
      u_t = u_t,
      u_xx = u_xx,
      residual = residual,
      residual_squared = residual^2
    )

    row_index <- row_index + 1
  }
}

residual_table <- bind_rows(residual_rows)

summary_table <- residual_table %>%
  summarise(
    diffusion = diffusion,
    n_x = n_x,
    n_t = n_t,
    dx = dx,
    dt = dt,
    mean_absolute_residual = mean(abs(residual)),
    root_mean_squared_residual = sqrt(mean(residual_squared)),
    max_absolute_residual = max(abs(residual))
  )

print(summary_table)
print(head(residual_table, 10))

This workflow shows how physics-informed diagnostics can be separated from neural-network training. Before trusting a learned model, one should be able to compute residuals, boundary errors, conservation errors, and validation metrics. Residual auditability is one of the core advantages of physics-informed scientific computing.

Back to top ↑

Python Workflow: Physics-Informed Neural Network for Exponential Decay

Python is useful for neural-network training, automatic differentiation, and scientific machine learning workflows. The following example trains a small PINN for the decay equation:

with \(u(0)=1\). The code uses PyTorch-style automatic differentiation to compute the residual.

"""
Physics-Informed Neural Network for Exponential Decay

This workflow trains a small PINN for:

    du/dt = -lambda * u

with:

    u(0) = 1

The analytic solution is:

    u(t) = exp(-lambda * t)

The physics residual is:

    r(t) = du_theta/dt + lambda * u_theta(t)

The training objective combines:
    1. residual loss at collocation points
    2. initial-condition loss at t = 0
    3. optional supervised data loss at a few observation points

This is a compact teaching example for physics-informed machine learning.
"""

from __future__ import annotations

import math
import random

import numpy as np
import pandas as pd
import torch
from torch import nn


RANDOM_SEED = 42


class DecayPINN(nn.Module):
    """
    Small fully connected neural network for u_theta(t).
    """

    def __init__(self, hidden_width: int = 32) -> None:
        super().__init__()

        self.network = nn.Sequential(
            nn.Linear(1, hidden_width),
            nn.Tanh(),
            nn.Linear(hidden_width, hidden_width),
            nn.Tanh(),
            nn.Linear(hidden_width, 1),
        )

    def forward(self, t: torch.Tensor) -> torch.Tensor:
        """
        Evaluate u_theta(t).
        """
        return self.network(t)


def exact_solution(t: torch.Tensor, lambda_value: float) -> torch.Tensor:
    """
    Analytic solution for exponential decay with u(0)=1.
    """
    return torch.exp(-lambda_value * t)


def physics_residual(
    model: nn.Module,
    t_collocation: torch.Tensor,
    lambda_value: float,
) -> torch.Tensor:
    """
    Compute residual r(t) = du_theta/dt + lambda u_theta.
    """
    t_collocation = t_collocation.clone().detach().requires_grad_(True)

    u_prediction = model(t_collocation)

    du_dt = torch.autograd.grad(
        outputs=u_prediction,
        inputs=t_collocation,
        grad_outputs=torch.ones_like(u_prediction),
        create_graph=True,
    )[0]

    residual = du_dt + lambda_value * u_prediction

    return residual


def train_pinn(
    lambda_value: float = 1.5,
    n_epochs: int = 5000,
    learning_rate: float = 1e-3,
) -> tuple[nn.Module, pd.DataFrame, pd.DataFrame]:
    """
    Train the PINN and return the model plus training and validation tables.
    """
    torch.manual_seed(RANDOM_SEED)
    np.random.seed(RANDOM_SEED)
    random.seed(RANDOM_SEED)

    model = DecayPINN(hidden_width=32)
    optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

    # Collocation points for the physics residual.
    t_collocation = torch.linspace(0.0, 2.0, 100).reshape(-1, 1)

    # Initial condition point.
    t_initial = torch.zeros((1, 1))
    u_initial = torch.ones((1, 1))

    # A few optional noise-free supervised observations.
    t_data = torch.tensor([[0.25], [0.75], [1.25], [1.75]], dtype=torch.float32)
    u_data = exact_solution(t_data, lambda_value)

    history_rows = []

    for epoch in range(n_epochs + 1):
        optimizer.zero_grad()

        residual = physics_residual(
            model=model,
            t_collocation=t_collocation,
            lambda_value=lambda_value,
        )

        residual_loss = torch.mean(residual**2)

        initial_loss = torch.mean((model(t_initial) - u_initial) ** 2)

        data_loss = torch.mean((model(t_data) - u_data) ** 2)

        total_loss = residual_loss + 10.0 * initial_loss + data_loss

        total_loss.backward()
        optimizer.step()

        if epoch % 250 == 0:
            history_rows.append(
                {
                    "epoch": epoch,
                    "total_loss": float(total_loss.detach()),
                    "residual_loss": float(residual_loss.detach()),
                    "initial_loss": float(initial_loss.detach()),
                    "data_loss": float(data_loss.detach()),
                }
            )

    # Validation against analytic solution.
    t_eval = torch.linspace(0.0, 2.0, 101).reshape(-1, 1)

    with torch.no_grad():
        u_pred = model(t_eval)
        u_exact = exact_solution(t_eval, lambda_value)
        absolute_error = torch.abs(u_pred - u_exact)

    validation_table = pd.DataFrame(
        {
            "t": t_eval.numpy().reshape(-1),
            "u_pred": u_pred.numpy().reshape(-1),
            "u_exact": u_exact.numpy().reshape(-1),
            "absolute_error": absolute_error.numpy().reshape(-1),
        }
    )

    history_table = pd.DataFrame(history_rows)

    return model, history_table, validation_table


def main() -> None:
    """
    Run the PINN training workflow and print compact diagnostics.
    """
    _, history, validation = train_pinn()

    validation_summary = pd.DataFrame(
        {
            "metric": [
                "mean_absolute_error",
                "root_mean_squared_error",
                "max_absolute_error",
            ],
            "value": [
                validation["absolute_error"].mean(),
                math.sqrt(np.mean(validation["absolute_error"] ** 2)),
                validation["absolute_error"].max(),
            ],
        }
    )

    print("Training history:")
    print(history.round(10).to_string(index=False))

    print("\nValidation summary:")
    print(validation_summary.round(10).to_string(index=False))

    print("\nValidation sample:")
    print(validation.head(10).round(8).to_string(index=False))


if __name__ == "__main__":
    main()

This workflow demonstrates the core PINN logic in its simplest form. The neural network is trained to satisfy a differential equation, an initial condition, and a few observations. In more complex systems, the same pattern extends to PDEs, inverse parameter estimation, multi-physics coupling, neural operators, and differentiable simulators.

GitHub Repository

The article body includes only selected computational examples so the conceptual and mathematical argument remains readable. The full repository contains the expanded computational infrastructure: R residual diagnostics, Python PINNs, neural ODE examples, universal differential equation examples, operator-learning dataset generators, surrogate validation, inverse parameter estimation, conservation checks, uncertainty summaries, Julia SciML-style examples, C++ residual sweeps, Fortran finite-difference baselines, SQL scientific machine learning provenance, Rust command-line utilities, C examples, documentation, and reproducible sample data.

Back to top ↑

From Black-Box Prediction to Auditable Scientific Computation

Physics-informed machine learning is not valuable because it makes physics fashionable within artificial intelligence. It is valuable when it makes scientific computation more data-aware, more flexible, more efficient, more interpretable, or more useful under uncertainty. Its strongest forms do not abandon classical numerical methods; they extend them with differentiable programming, trainable surrogates, operator learning, inverse inference, and physically constrained approximation.

Within the Physics knowledge series, this article belongs near Numerical Methods in Physics, Computational Physics and Scientific Simulation, Experimental Physics: Measurement, Noise, Calibration, and Inference, Nonequilibrium Statistical Mechanics, Fluid Dynamics and Continuum Mechanics, Scattering Theory, Cross Sections, and Physical Inference, and Quantum Information, Decoherence, and Measurement. It provides one of the most important modern bridges between physical theory, numerical approximation, data, and scientific inference.

The next conceptual steps are natural. Neural Operators and Surrogate Modeling in Physics develops operator learning and fast simulation. Differentiable Simulation and Inverse Physics develops gradient-based physical inference. Uncertainty Quantification for Scientific Machine Learning develops credibility and calibration. AI for Physics Discovery and Model Building develops equation discovery, symbolic regression, and hypothesis generation.

The deeper lesson is methodological. Scientific machine learning should not be treated as a way to escape physical reasoning. It should be treated as a way to make physical reasoning computationally richer. The best physics-informed systems remain accountable to equations, measurements, uncertainty, units, error analysis, and reproducibility. They expand the toolkit of physics without weakening the standards that make physics a science.

Back to top ↑

Related Articles

• What Is Physics?
• Measurement, Mathematics, and the Structure of Physical Inquiry
• Numerical Methods in Physics
• Computational Physics and Scientific Simulation
• Experimental Physics: Measurement, Noise, Calibration, and Inference
• Nonequilibrium Statistical Mechanics
• Fluid Dynamics and Continuum Mechanics
• Scattering Theory, Cross Sections, and Physical Inference
• Quantum Information, Decoherence, and Measurement
• Neural Operators and Surrogate Modeling in Physics
• Differentiable Simulation and Inverse Physics
• Uncertainty Quantification for Scientific Machine Learning
• AI for Physics Discovery and Model Building

Further Reading

• Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S. and Yang, L. (2021) ‘Physics-informed machine learning’, Nature Reviews Physics, 3, pp. 422–440. Available at: https://www.nature.com/articles/s42254-021-00314-5 (Accessed: 15 May 2026).
• Raissi, M., Perdikaris, P. and Karniadakis, G.E. (2019) ‘Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations’, Journal of Computational Physics, 378, pp. 686–707. Available at: https://www.sciencedirect.com/science/article/pii/S0021999118307125 (Accessed: 15 May 2026).
• Cuomo, S., Di Cola, V.S., Giampaolo, F., Rozza, G., Raissi, M. and Piccialli, F. (2022) ‘Scientific Machine Learning Through Physics-Informed Neural Networks: Where we are and What’s Next’, Journal of Scientific Computing, 92, 88. Available at: https://link.springer.com/article/10.1007/s10915-022-01939-z (Accessed: 15 May 2026).
• Chen, R.T.Q., Rubanova, Y., Bettencourt, J. and Duvenaud, D. (2018) ‘Neural Ordinary Differential Equations’, Advances in Neural Information Processing Systems. Available at: https://arxiv.org/abs/1806.07366 (Accessed: 15 May 2026).
• Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K., Supekar, R., Skinner, D., Ramadhan, A. and Edelman, A. (2020) ‘Universal Differential Equations for Scientific Machine Learning’. Available at: https://arxiv.org/abs/2001.04385 (Accessed: 15 May 2026).
• Lu, L., Jin, P., Pang, G., Zhang, Z. and Karniadakis, G.E. (2021) ‘Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators’, Nature Machine Intelligence, 3, pp. 218–229. Available at: https://www.nature.com/articles/s42256-021-00302-5 (Accessed: 15 May 2026).
• Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. and Anandkumar, A. (2021) ‘Fourier Neural Operator for Parametric Partial Differential Equations’. Available at: https://arxiv.org/abs/2010.08895 (Accessed: 15 May 2026).
• Lu, L., Meng, X., Mao, Z. and Karniadakis, G.E. (2021) ‘DeepXDE: A Deep Learning Library for Solving Differential Equations’, SIAM Review, 63(1), pp. 208–228. Available at: https://epubs.siam.org/doi/10.1137/19M1274067 (Accessed: 15 May 2026).
• SciML (2026) SciML: Open Source Software for Scientific Machine Learning. Available at: https://sciml.ai/ (Accessed: 15 May 2026).
• SciML Documentation (2026) Scientific Machine Learning Documentation. Available at: https://docs.sciml.ai/ (Accessed: 15 May 2026).
• MIT Scientific Machine Learning Group (2026) Parallel Computing and Scientific Machine Learning. Available at: https://book.sciml.ai/course/ (Accessed: 15 May 2026).

Back to top ↑

References

• Chen, R.T.Q., Rubanova, Y., Bettencourt, J. and Duvenaud, D. (2018) ‘Neural Ordinary Differential Equations’, Advances in Neural Information Processing Systems. Available at: https://arxiv.org/abs/1806.07366 (Accessed: 15 May 2026).
• Cuomo, S., Di Cola, V.S., Giampaolo, F., Rozza, G., Raissi, M. and Piccialli, F. (2022) ‘Scientific Machine Learning Through Physics-Informed Neural Networks: Where we are and What’s Next’, Journal of Scientific Computing, 92, 88. Available at: https://link.springer.com/article/10.1007/s10915-022-01939-z (Accessed: 15 May 2026).
• Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S. and Yang, L. (2021) ‘Physics-informed machine learning’, Nature Reviews Physics, 3, pp. 422–440. Available at: https://www.nature.com/articles/s42254-021-00314-5 (Accessed: 15 May 2026).
• Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A. and Anandkumar, A. (2021) ‘Fourier Neural Operator for Parametric Partial Differential Equations’. Available at: https://arxiv.org/abs/2010.08895 (Accessed: 15 May 2026).
• Lu, L., Jin, P., Pang, G., Zhang, Z. and Karniadakis, G.E. (2021) ‘Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators’, Nature Machine Intelligence, 3, pp. 218–229. Available at: https://www.nature.com/articles/s42256-021-00302-5 (Accessed: 15 May 2026).
• Lu, L., Meng, X., Mao, Z. and Karniadakis, G.E. (2021) ‘DeepXDE: A Deep Learning Library for Solving Differential Equations’, SIAM Review, 63(1), pp. 208–228. Available at: https://epubs.siam.org/doi/10.1137/19M1274067 (Accessed: 15 May 2026).
• Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K., Supekar, R., Skinner, D., Ramadhan, A. and Edelman, A. (2020) ‘Universal Differential Equations for Scientific Machine Learning’. Available at: https://arxiv.org/abs/2001.04385 (Accessed: 15 May 2026).
• Raissi, M., Perdikaris, P. and Karniadakis, G.E. (2019) ‘Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations’, Journal of Computational Physics, 378, pp. 686–707. Available at: https://www.sciencedirect.com/science/article/pii/S0021999118307125 (Accessed: 15 May 2026).
• SciML (2026) SciML: Open Source Software for Scientific Machine Learning. Available at: https://sciml.ai/ (Accessed: 15 May 2026).
• SciML Documentation (2026) Scientific Machine Learning Documentation. Available at: https://docs.sciml.ai/ (Accessed: 15 May 2026).

Back to top ↑