Markov Chains and Transition Matrices: How Linear Algebra Models State Transitions

Last Updated June 29, 2026

Markov chains and transition matrices connect probability, linear algebra, and systems modeling through state transitions. A system may move among categories, conditions, locations, regimes, risk levels, behaviors, or operating states. A transition matrix organizes the probabilities of those movements, allowing the modeler to study short-run movement, repeated transitions, steady states, convergence, absorbing behavior, and long-run structure.

This article introduces Markov chains as matrix-based models of state change. It connects transition probabilities, stochastic matrices, state vectors, matrix multiplication, powers of transition matrices, steady-state distributions, eigenvalues, eigenvectors, absorbing states, reducibility, periodicity, ergodicity, convergence, uncertainty, validation, and responsible interpretation.

The central modeling question is not only “What are the transition probabilities?” It is “What assumptions make this transition matrix a defensible representation of how the system changes over time?”

Markov chains are useful when the future state of a system depends on its current state through transition probabilities. The model does not require the full past history once the current state is known. That simplifying assumption makes Markov chains powerful, but also demanding: if history, memory, hidden variables, policy changes, or structural breaks matter, the transition matrix may mislead.

For systems modeling, transition matrices provide a bridge between linear algebra and probability. A state vector represents how probability, population, demand, risk, attention, condition, or system mass is distributed across states. Multiplying by a transition matrix updates that distribution. Repeating the multiplication models system evolution.

Why Markov Chains Matter

Markov chains matter because many systems can be modeled as movement among states. A road segment may move from good condition to fair condition to poor condition. A household may move among income categories. A user may move among attention states. A habitat may transition among ecological conditions. A machine may move among operating, degraded, and failed states.

Linear algebra makes these transitions computable. A transition matrix collects the probabilities of moving from one state to another. A state vector records the current distribution across states. Matrix multiplication updates the distribution. Matrix powers describe repeated transitions.

Markov chains are useful because they turn state change into a structured matrix workflow.

Back to top ↑

States and State Vectors

A Markov chain begins with a set of states. These states should be mutually clear, collectively meaningful, and appropriate to the modeling purpose. In applied systems work, states might represent condition grades, risk classes, policy regimes, asset statuses, disease stages, market categories, customer behaviors, or ecological conditions.

A state vector stores the distribution across states. For a probability distribution, entries should be nonnegative and sum to one.

A Markov chain is only as meaningful as its state definitions. State design is not clerical; it is part of the model.

Back to top ↑

Transition Probabilities

A transition probability describes the chance of moving from one state to another over a specified time step. For example, a road segment in fair condition might have a probability of remaining fair, improving to good after maintenance, or deteriorating to poor.

Transition probabilities are conditional probabilities:

The Markov assumption states that the next-state distribution depends on the current state, not the full prior history:

Transition probabilities are modeling commitments, not just matrix entries.

Back to top ↑

Transition Matrices

A transition matrix organizes all transition probabilities. In this article, the main convention uses a row-stochastic matrix: each row represents the current state, each column represents the next state, and each row sums to one.

For a row-stochastic transition matrix:

A transition matrix is a compact representation of how a system moves among states.

Back to top ↑

Row and Column Conventions

Markov-chain notation has two common conventions. In a row-vector convention, the state distribution is written as a row vector and multiplied on the right by a row-stochastic matrix:

In a column-vector convention, the state distribution is written as a column vector and multiplied on the left by a column-stochastic matrix, or by the transpose of a row-stochastic matrix:

The convention is not merely cosmetic. Confusing row-stochastic and column-stochastic orientation can transpose the meaning of the model. Every workflow should document its convention.

Back to top ↑

One-Step and Multi-Step Transitions

A one-step transition updates the state distribution once. Under the row-vector convention:

After two steps:

After \(t\) steps:

Multi-step transitions are where Markov chains become systems models rather than static probability tables.

Back to top ↑

Matrix Powers

Matrix powers allow the transition matrix to describe repeated change. The entry of \(P^t\) at row \(i\), column \(j\), gives the probability of being in state \(j\) after \(t\) steps given an initial state \(i\), under the row-stochastic convention.

Matrix powers connect Markov chains to eigenvalues. Eigenvalues shape how quickly powers of \(P\) approach a long-run pattern, whether cycles persist, and whether some states absorb probability mass.

Matrix powers make state-transition systems analyzable over multiple horizons.

Back to top ↑

Steady-State Distributions

A steady-state distribution is a distribution that remains unchanged by the transition matrix. Under the row-vector convention, a steady-state distribution \(\boldsymbol{\pi}\) satisfies:

This is an eigenvector equation. The steady-state distribution is a left eigenvector of \(P\) associated with eigenvalue \(1\), normalized so its entries sum to one.

A steady-state distribution is a mathematical fixed point of the transition process, not an automatic claim about social, ecological, economic, or institutional value.

Back to top ↑

Eigenvalues and Markov Chains

Eigenvalues shape Markov-chain behavior. For a stochastic matrix, \(1\) is a central eigenvalue. Other eigenvalues influence convergence speed, persistence, oscillation, and long-run structure.

When a Markov chain converges to a unique steady state, eigenvalues with magnitudes less than one usually describe modes that decay over repeated transitions. Eigenvalues close to one indicate slow decay. Negative or complex eigenvalues can indicate alternating or oscillatory behavior.

Eigenvalues help connect Markov chains to the broader linear algebra of modes, stability, and long-run behavior.

Back to top ↑

Absorbing States

An absorbing state is a state that, once entered, cannot be left. In a row-stochastic transition matrix, state \(i\) is absorbing when:

Absorbing states appear in failure models, default models, completion processes, death states, retirement states, disease progressions, terminal statuses, and irreversible classifications. They can be useful, but they can also encode strong assumptions about irreversibility.

Absorbing states are powerful for lifecycle models, but they should be reviewed carefully when recovery, repair, reversal, or reclassification is possible.

Back to top ↑

Reducibility and Communication

States communicate if each can be reached from the other through some sequence of transitions. A Markov chain is irreducible if all states communicate with each other. It is reducible if the state space breaks into separate communicating classes or one-way structures.

Reducibility is a structural property. It tells whether the transition system is one connected probabilistic process or a collection of communicating regions, traps, and one-way pathways.

Back to top ↑

Periodicity and Convergence

Periodicity occurs when a state can be revisited only at multiples of a fixed cycle length. Periodic chains can fail to converge in the simple way expected from a steady-state calculation. A distribution may cycle rather than settle.

Aperiodicity means the system does not have rigid return cycles. Together with irreducibility and suitable stochastic structure, aperiodicity supports convergence toward a unique stationary distribution.

Convergence is not guaranteed merely because a transition matrix exists. The chain’s structure determines its long-run behavior.

Back to top ↑

Ergodicity and Long-Run Behavior

In many finite-state settings, an ergodic Markov chain has a unique stationary distribution and converges toward it from any initial distribution. This makes long-run analysis especially useful: the initial condition eventually matters less than the transition structure.

However, “long-run” is not always operationally relevant. A chain may converge only after many steps. External conditions may change before the long-run limit matters. The transition matrix may be estimated from short-lived data. Slow mixing can make initial conditions important for a long time.

Ergodicity supports strong long-run claims, but those claims still depend on whether the transition matrix remains valid over the relevant horizon.

Back to top ↑

Estimation and Validation

Transition matrices are often estimated from data. For observed transitions from state \(i\) to state \(j\), a simple estimate is:

This estimate can be fragile when data are sparse, states are rare, measurement error is high, or transition behavior changes over time. Validation should compare predicted distributions with observed outcomes and test sensitivity to state definitions, time interval, smoothing, and missing data.

Markov-chain workflows should treat transition matrices as estimated, reviewable objects, not as neutral facts.

Back to top ↑

Markov Chains in Systems Modeling

Markov chains appear across systems modeling because they describe transitions among states. They are used in reliability modeling, infrastructure condition forecasting, queueing systems, credit migration, epidemiology, ecological transitions, land-use change, customer behavior, attention systems, maintenance planning, operations research, and policy analysis.

The strength of Markov chains is their clarity. The weakness is also their clarity: they simplify history into the current state and a transition rule. If that simplification hides memory, structural change, heterogeneity, feedback, or intervention, the model may become clean but misleading.

A Markov chain can clarify transition structure, but it should not conceal the real drivers of transition behavior.

Back to top ↑

Mathematical Deepening

This section adds a more formal layer. Markov chains connect stochastic matrices, probability vectors, matrix powers, eigenvalues, stationary distributions, communicating classes, irreducibility, recurrence, transience, absorbing chains, periodicity, ergodicity, convergence, spectral gaps, Perron-Frobenius theory, and long-run system behavior.

Back to top ↑

Back to top ↑

Computation and Reproducible Workflows

Computational workflows for Markov chains should document state definitions, transition-matrix orientation, row or column sums, nonnegativity checks, initial distribution, one-step forecast, multi-step forecast, stationary distribution, eigenvalue diagnostics, absorbing states, reducibility flags, convergence checks, and validation warnings.

The companion repository treats transition matrices as auditable systems models. Python, R, Julia, SQL, Haskell, C, C++, Fortran, Rust, Go, notebooks, schemas, generated outputs, Canvas artifacts, advanced reports, and calculators each support a different layer of reproducible Markov-chain analysis.

For this article, the computational examples focus on row-stochastic validation, state-vector updates, multi-step transition powers, steady-state approximation, convergence diagnostics, and model-governance records.

Back to top ↑

Python Workflow: Transition Matrix Audit

The Python workflow below uses a row-stochastic transition matrix. It checks row sums, validates nonnegativity, computes one-step and multi-step distributions, approximates a steady state, and records interpretation warnings.

from __future__ import annotations

from dataclasses import asdict, dataclass
from pathlib import Path
import csv
import json
import math


Matrix = list[list[float]]
Vector = list[float]


@dataclass(frozen=True)
class MarkovAudit:
    system_name: str
    states: str
    orientation: str
    transition_matrix: str
    initial_distribution: str
    row_sum_error: float
    nonnegative: bool
    one_step_distribution: str
    ten_step_distribution: str
    steady_state_estimate: str
    conservation_error: float
    interpretation_warning: str


def matrix_to_string(A: Matrix) -> str:
    return ";".join(",".join(f"{value:.6f}" for value in row) for row in A)


def vector_to_string(v: Vector) -> str:
    return ",".join(f"{value:.6f}" for value in v)


def row_sum_error(P: Matrix) -> float:
    return max(abs(sum(row) - 1.0) for row in P)


def is_nonnegative(P: Matrix) -> bool:
    return all(value >= 0.0 for row in P for value in row)


def row_vector_matrix_multiply(pi: Vector, P: Matrix) -> Vector:
    return [
        sum(pi[i] * P[i][j] for i in range(len(pi)))
        for j in range(len(P[0]))
    ]


def evolve(pi: Vector, P: Matrix, steps: int) -> Vector:
    current = pi[:]
    for _ in range(steps):
        current = row_vector_matrix_multiply(current, P)
    return current


def normalize(v: Vector) -> Vector:
    total = sum(v)
    if abs(total) <= 1e-12:
        raise ValueError("cannot normalize vector with near-zero sum")
    return [x / total for x in v]


def build_audit() -> MarkovAudit:
    states = ["good", "fair", "poor"]

    # Row-stochastic convention:
    # rows = current state, columns = next state.
    P = [
        [0.82, 0.16, 0.02],
        [0.10, 0.76, 0.14],
        [0.03, 0.22, 0.75],
    ]

    pi0 = [0.60, 0.30, 0.10]
    pi1 = evolve(pi0, P, 1)
    pi10 = evolve(pi0, P, 10)

    # Simple power iteration for a stationary distribution estimate.
    steady = [1.0 / len(states)] * len(states)
    for _ in range(500):
        steady = evolve(steady, P, 1)
    steady = normalize(steady)

    conservation_error = abs(sum(pi10) - 1.0)

    return MarkovAudit(
        system_name="infrastructure_condition_transition_audit",
        states="|".join(states),
        orientation="row_stochastic_row_vector_update_pi_next_equals_pi_P",
        transition_matrix=matrix_to_string(P),
        initial_distribution=vector_to_string(pi0),
        row_sum_error=round(row_sum_error(P), 12),
        nonnegative=is_nonnegative(P),
        one_step_distribution=vector_to_string(pi1),
        ten_step_distribution=vector_to_string(pi10),
        steady_state_estimate=vector_to_string(steady),
        conservation_error=round(conservation_error, 12),
        interpretation_warning=(
            "Transition matrices depend on state definitions, time step, data quality, stationarity, "
            "and the Markov assumption. A steady state is a model-implied fixed distribution, "
            "not automatically a desirable system outcome."
        ),
    )


def write_outputs(output_dir: Path) -> None:
    (output_dir / "tables").mkdir(parents=True, exist_ok=True)
    (output_dir / "json").mkdir(parents=True, exist_ok=True)

    audit = build_audit()
    row = asdict(audit)

    with (output_dir / "tables" / "markov_transition_audit.csv").open(
        "w", newline="", encoding="utf-8"
    ) as handle:
        writer = csv.DictWriter(handle, fieldnames=list(row.keys()))
        writer.writeheader()
        writer.writerow(row)

    (output_dir / "json" / "markov_transition_audit.json").write_text(
        json.dumps(row, indent=2, sort_keys=True),
        encoding="utf-8",
    )


if __name__ == "__main__":
    write_outputs(Path("outputs"))
    print("Markov transition matrix audit complete.")

This workflow keeps the transition convention, stochastic checks, state evolution, steady-state estimate, and interpretation warning together.

Back to top ↑

R Workflow: Markov Chain Diagnostics

R can support Markov-chain diagnostics by checking stochastic structure, updating distributions, computing matrix powers, and estimating steady-state behavior.

states <- c("good", "fair", "poor")

P <- matrix(
  c(
    0.82, 0.16, 0.02,
    0.10, 0.76, 0.14,
    0.03, 0.22, 0.75
  ),
  nrow = 3,
  byrow = TRUE
)

pi0 <- c(0.60, 0.30, 0.10)

row_sum_error <- max(abs(rowSums(P) - 1))
nonnegative <- all(P >= 0)

step_distribution <- function(pi, P, steps) {
  current <- pi
  for (k in seq_len(steps)) {
    current <- current %*% P
  }
  as.numeric(current)
}

pi1 <- step_distribution(pi0, P, 1)
pi10 <- step_distribution(pi0, P, 10)

steady <- rep(1 / length(states), length(states))
for (k in seq_len(500)) {
  steady <- step_distribution(steady, P, 1)
}
steady <- steady / sum(steady)

eigen_result <- eigen(t(P))
eigenvalues <- eigen_result$values

audit_record <- data.frame(
  system_name = "infrastructure_condition_transition_audit",
  states = paste(states, collapse = "|"),
  orientation = "row_stochastic_row_vector_update_pi_next_equals_pi_P",
  row_sum_error = row_sum_error,
  nonnegative = nonnegative,
  initial_distribution = paste(round(pi0, 6), collapse = ","),
  one_step_distribution = paste(round(pi1, 6), collapse = ","),
  ten_step_distribution = paste(round(pi10, 6), collapse = ","),
  steady_state_estimate = paste(round(steady, 6), collapse = ","),
  leading_eigenvalues = paste(round(Re(eigenvalues), 6), collapse = ","),
  interpretation_warning = paste(
    "Transition matrices depend on state definitions, time step, stationarity, and the Markov assumption.",
    "A steady state is model-implied, not automatically desirable."
  )
)

dir.create("outputs/tables", recursive = TRUE, showWarnings = FALSE)

write.csv(
  audit_record,
  "outputs/tables/r_markov_transition_audit.csv",
  row.names = FALSE
)

print(audit_record)

This R workflow keeps stochastic validation, transition simulation, eigenvalue review, and steady-state estimation in one reproducible output.

Back to top ↑

Haskell Workflow: Typed Transition Records

Haskell can represent Markov-chain diagnostics as typed records, keeping state definitions, orientation, stochastic checks, and interpretation warnings explicit.

module Main where

data MarkovAudit = MarkovAudit
  { systemName :: String
  , states :: String
  , orientation :: String
  , transitionMatrix :: String
  , initialDistribution :: String
  , rowSumError :: Double
  , nonnegative :: Bool
  , oneStepDistribution :: String
  , tenStepDistribution :: String
  , steadyStateEstimate :: String
  , interpretationWarning :: String
  } deriving (Show)

buildAudit :: MarkovAudit
buildAudit =
  MarkovAudit
    "infrastructure_condition_transition_audit"
    "good|fair|poor"
    "row_stochastic_row_vector_update_pi_next_equals_pi_P"
    "0.820000,0.160000,0.020000;0.100000,0.760000,0.140000;0.030000,0.220000,0.750000"
    "0.600000,0.300000,0.100000"
    0.0
    True
    "0.525000,0.346000,0.129000"
    "0.286282,0.478868,0.234850"
    "0.233333,0.488889,0.277778"
    "Transition matrices depend on state definitions, time step, stationarity, data quality, and the Markov assumption."

main :: IO ()
main =
  print buildAudit

The typed workflow makes the transition convention visible, which helps prevent silent row-column orientation errors.

Back to top ↑

SQL Workflow: Transition Matrix Assumption Registry

SQL can document Markov-chain assumptions when transition matrices support dashboards, infrastructure planning, risk analysis, forecasting workflows, or institutional decision systems.

CREATE TABLE markov_assumption_registry (
    assumption_key TEXT PRIMARY KEY,
    assumption_name TEXT NOT NULL,
    mathematical_role TEXT NOT NULL,
    systems_modeling_role TEXT NOT NULL,
    review_warning TEXT NOT NULL
);

INSERT INTO markov_assumption_registry VALUES
(
  'state_definitions',
  'State definitions',
  'Define the coordinates of the transition system.',
  'Determine what the model can observe and update.',
  'Ambiguous or incomplete states distort the transition matrix.'
);

INSERT INTO markov_assumption_registry VALUES
(
  'transition_orientation',
  'Transition orientation',
  'Specifies row-stochastic or column-stochastic convention.',
  'Determines how state vectors are multiplied by the matrix.',
  'Row-column confusion can transpose the model meaning.'
);

INSERT INTO markov_assumption_registry VALUES
(
  'time_step',
  'Time step',
  'Defines the interval for each transition.',
  'Determines whether probabilities are daily, monthly, annual, or otherwise.',
  'Transition probabilities cannot be reused across time scales without transformation.'
);

INSERT INTO markov_assumption_registry VALUES
(
  'markov_assumption',
  'Markov assumption',
  'Assumes next state depends on current state rather than full history.',
  'Simplifies state evolution into a transition matrix.',
  'Path dependence, memory, and hidden variables can violate the assumption.'
);

INSERT INTO markov_assumption_registry VALUES
(
  'stationarity',
  'Stationarity',
  'Assumes transition probabilities remain constant over the modeled horizon.',
  'Supports repeated use of the same transition matrix.',
  'Policy changes, shocks, climate, behavior, and technology can alter transitions.'
);

INSERT INTO markov_assumption_registry VALUES
(
  'steady_state',
  'Steady state',
  'Distribution unchanged by the transition matrix.',
  'Supports long-run composition analysis.',
  'A stable distribution is not automatically desirable, equitable, or safe.'
);

SELECT
    assumption_name,
    mathematical_role,
    systems_modeling_role,
    review_warning
FROM markov_assumption_registry
ORDER BY assumption_key;

This registry keeps Markov-chain analysis tied to state definitions, orientation, time step, the Markov assumption, stationarity, steady-state interpretation, and responsible systems modeling.

Back to top ↑

GitHub Repository

The companion repository for this article is designed as a reproducible mathematical-modeling workspace. It supports Markov transition audits, stochastic-matrix validation, state-vector updates, matrix-power diagnostics, steady-state estimation, eigenvalue review, SQL governance tables, generated outputs, advanced mathematical audit reports, and reusable calculator scripts.

Back to top ↑

Interpretive Limits and Responsible Use

Markov chains are powerful because they make transition systems mathematically clear. They are limited because the clarity depends on strong assumptions. The state definitions must be meaningful. The time step must be explicit. The transition probabilities must be valid. The system must be reasonably represented by current-state dependence. The transition matrix must remain stable enough over the modeled horizon for repeated multiplication to make sense.

The Markov assumption can be too restrictive when history matters. Infrastructure deterioration may depend on age and maintenance history. Health progression may depend on cumulative exposure. Credit migration may depend on macroeconomic regime. User behavior may depend on prior recommendations. Ecological transitions may depend on path-dependent thresholds. In such settings, the state space may need to be expanded, or the model may need a non-Markov, semi-Markov, hidden Markov, time-varying, or agent-based alternative.

Responsible use requires documenting state definitions, transition orientation, time step, data source, estimation method, row or column sums, nonnegativity checks, missing states, stationarity assumptions, validation results, uncertainty, absorbing states, reducibility, convergence behavior, and the social, ecological, economic, or institutional meaning of long-run distributions.

Back to top ↑

Related Articles

• What Is Linear Algebra for Systems Modeling?
• Scalars, Vectors, and System States
• Vector Spaces and System Representation
• Span, Linear Independence, and Basis
• Dimension and the Structure of Solution Spaces
• Matrices and the Organization of Multivariable Systems
• Matrix Arithmetic and the Logic of Combination
• Systems of Linear Equations
• Gaussian Elimination and Row Reduction
• Pivot Structure and Solvability
• Rank, Nullity, and Structural Dependence
• Determinants and Invertibility
• Inverse Matrices and Structural Recovery
• Overdetermined Systems and Least Squares Thinking
• Linear Transformations and Model Behavior
• Matrix Multiplication and Interaction Effects
• Change of Basis and Alternative Representations
• Projections, Reflections, and Geometric Interpretation
• Orthogonality and Structured Simplification
• Inner Products, Norms, and Distance in State Space
• Eigenvalues, Eigenvectors, and System Modes
• Diagonalization and Decoupled System Behavior
• Stability Analysis with Eigenvalues
• Linear Algebra for Systems Modeling
• Mathematical Modeling
• Systems Modeling
• Scientific Computing for Systems Modeling

Back to top ↑

Further Reading

• Boyd, S. and Vandenberghe, L. (2018) Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares. Cambridge: Cambridge University Press. Available at: https://vmls-book.stanford.edu/.
• Boyd, S. and Vandenberghe, L. (2018) Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares. PDF edition. Stanford University and UCLA. Available at: https://web.stanford.edu/~boyd/vmls/vmls.pdf.
• Durrett, R. (2019) Probability: Theory and Examples. 5th edn. Cambridge: Cambridge University Press. Available at: https://services.math.duke.edu/~rtd/PTE/PTE5_011119.pdf.
• Georgia Institute of Technology (n.d.) Interactive Linear Algebra. Georgia Institute of Technology. Available at: https://textbooks.math.gatech.edu/ila/.
• Grinstead, C.M. and Snell, J.L. (1997) Introduction to Probability. Providence, RI: American Mathematical Society. Available at: https://www.ams.org/open-math-notes/omn-view-listing?listingId=110645.
• Higham, N.J. (2002) Accuracy and Stability of Numerical Algorithms. 2nd edn. Philadelphia, PA: Society for Industrial and Applied Mathematics. Available at: https://epubs.siam.org/doi/book/10.1137/1.9780898718027.
• Horn, R.A. and Johnson, C.R. (2013) Matrix Analysis. 2nd edn. Cambridge: Cambridge University Press. Available at: https://www.cambridge.org/highereducation/books/matrix-analysis/FDA3627DC2B9F5C3DF2FD8C3CC136B48.
• Kemeny, J.G. and Snell, J.L. (1976) Finite Markov Chains. New York: Springer. Available at: https://link.springer.com/book/10.1007/978-1-4684-9455-6.
• Levin, D.A., Peres, Y. and Wilmer, E.L. (2017) Markov Chains and Mixing Times. 2nd edn. Providence, RI: American Mathematical Society. Available at: https://pages.uoregon.edu/dlevin/MARKOV/.
• Massachusetts Institute of Technology OpenCourseWare (2013) Introduction to Probability. Cambridge, MA: MIT OpenCourseWare. Available at: https://ocw.mit.edu/courses/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/.
• Meyer, C.D. (2023) Matrix Analysis and Applied Linear Algebra. 2nd edn. Philadelphia, PA: Society for Industrial and Applied Mathematics. Available at: https://epubs.siam.org/doi/book/10.1137/1.9781611977448.
• MIT Mathematics (2023) Introduction to Linear Algebra, Sixth Edition. Cambridge, MA: Massachusetts Institute of Technology. Available at: https://math.mit.edu/~gs/linearalgebra/ila6/indexila6.html.
• Norris, J.R. (1997) Markov Chains. Cambridge: Cambridge University Press. Available at: https://www.cambridge.org/highereducation/books/markov-chains/4ED5AB94CE50FE6A4A03C0C742D1E900.
• NumPy Developers (n.d.) Linear Algebra: numpy.linalg. NumPy Documentation. Available at: https://numpy.org/doc/stable/reference/routines.linalg.html.
• Ross, S.M. (2014) Introduction to Probability Models. 11th edn. Amsterdam: Academic Press. Available at: https://www.elsevier.com/books/introduction-to-probability-models/ross/978-0-12-407948-9.
• SciPy Developers (n.d.) Linear Algebra: scipy.linalg. SciPy Documentation. Available at: https://docs.scipy.org/doc/scipy/reference/linalg.html.
• Stewart, W.J. (1994) Introduction to the Numerical Solution of Markov Chains. Princeton, NJ: Princeton University Press. Available at: https://press.princeton.edu/books/hardcover/9780691036990/introduction-to-the-numerical-solution-of-markov-chains.
• Strang, G. (2016) Introduction to Linear Algebra. Wellesley, MA: Wellesley-Cambridge Press. Available at: https://math.mit.edu/~gs/linearalgebra/.

Back to top ↑

References

• Boyd, S. and Vandenberghe, L. (2018) Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares. Cambridge: Cambridge University Press. Available at: https://vmls-book.stanford.edu/.
• Durrett, R. (2019) Probability: Theory and Examples. 5th edn. Cambridge: Cambridge University Press. Available at: https://services.math.duke.edu/~rtd/PTE/PTE5_011119.pdf.
• Grinstead, C.M. and Snell, J.L. (1997) Introduction to Probability. Providence, RI: American Mathematical Society. Available at: https://www.ams.org/open-math-notes/omn-view-listing?listingId=110645.
• Higham, N.J. (2002) Accuracy and Stability of Numerical Algorithms. 2nd edn. Philadelphia, PA: Society for Industrial and Applied Mathematics. Available at: https://epubs.siam.org/doi/book/10.1137/1.9780898718027.
• Horn, R.A. and Johnson, C.R. (2013) Matrix Analysis. 2nd edn. Cambridge: Cambridge University Press. Available at: https://www.cambridge.org/highereducation/books/matrix-analysis/FDA3627DC2B9F5C3DF2FD8C3CC136B48.
• Kemeny, J.G. and Snell, J.L. (1976) Finite Markov Chains. New York: Springer. Available at: https://link.springer.com/book/10.1007/978-1-4684-9455-6.
• Levin, D.A., Peres, Y. and Wilmer, E.L. (2017) Markov Chains and Mixing Times. 2nd edn. Providence, RI: American Mathematical Society. Available at: https://pages.uoregon.edu/dlevin/MARKOV/.
• Massachusetts Institute of Technology OpenCourseWare (2013) Introduction to Probability. Cambridge, MA: MIT OpenCourseWare. Available at: https://ocw.mit.edu/courses/6-041sc-probabilistic-systems-analysis-and-applied-probability-fall-2013/.
• Meyer, C.D. (2023) Matrix Analysis and Applied Linear Algebra. 2nd edn. Philadelphia, PA: Society for Industrial and Applied Mathematics. Available at: https://epubs.siam.org/doi/book/10.1137/1.9781611977448.
• Norris, J.R. (1997) Markov Chains. Cambridge: Cambridge University Press. Available at: https://www.cambridge.org/highereducation/books/markov-chains/4ED5AB94CE50FE6A4A03C0C742D1E900.
• Ross, S.M. (2014) Introduction to Probability Models. 11th edn. Amsterdam: Academic Press. Available at: https://www.elsevier.com/books/introduction-to-probability-models/ross/978-0-12-407948-9.
• Stewart, W.J. (1994) Introduction to the Numerical Solution of Markov Chains. Princeton, NJ: Princeton University Press. Available at: https://press.princeton.edu/books/hardcover/9780691036990/introduction-to-the-numerical-solution-of-markov-chains.
• Strang, G. (2016) Introduction to Linear Algebra. Wellesley, MA: Wellesley-Cambridge Press. Available at: https://math.mit.edu/~gs/linearalgebra/.

Back to top ↑