Model Comparison and Selection: How to Choose the Right Mathematical Model

Last Updated June 12, 2026

Model comparison and selection help analysts decide which mathematical model is most credible, useful, and appropriate for a given purpose. A model may fit data well, but comparison asks whether another model explains better, predicts better, uses fewer assumptions, handles uncertainty more responsibly, or supports decisions more transparently.

Mathematical modeling rarely produces only one possible model. Analysts may compare linear and nonlinear forms, deterministic and stochastic models, simple baselines and complex simulations, mechanistic and statistical models, or different assumptions about boundaries, parameters, interactions, and uncertainty. Model selection is the disciplined process of choosing among these alternatives.

The goal is not to find a universally best model. The goal is to choose a model that is fit for a purpose, supported by evidence, honest about uncertainty, appropriate in complexity, interpretable enough for its use, and accountable to the consequences of decisions made from it.

Responsible model comparison resists two common mistakes. The first is choosing the model that fits calibration data best without checking generalization. The second is choosing the most complex model because it appears more sophisticated. A useful model must be judged against purpose, evidence, error behavior, uncertainty, robustness, and decision relevance.

Why Model Comparison Matters

Model comparison matters because no model should be evaluated in isolation when plausible alternatives exist. A model that seems impressive on its own may be no better than a simple baseline. A model that fits historical data may generalize poorly. A model that is accurate may be too opaque for the decision context. A model that is interpretable may be too crude for prediction.

Comparison makes these tradeoffs visible. It asks what each model gains and loses: explanatory clarity, predictive accuracy, computational efficiency, robustness, interpretability, uncertainty awareness, domain plausibility, and decision usefulness.

Model comparison improves accountability because it forces analysts to explain why one model was preferred and what alternatives were considered.

Back to top ↑

What Model Comparison Is

Model comparison is the systematic evaluation of two or more candidate models against evidence, criteria, assumptions, uncertainty, and intended use. Model selection is the decision to prefer one model, a set of models, or an ensemble for a particular purpose.

Comparison may involve statistical scores, predictive performance, validation diagnostics, information criteria, interpretability, computational cost, robustness, stakeholder needs, or ethical risk. Different purposes require different selection criteria.

Model selection should produce a documented rationale, not only a winning score. The selected model should be justified in relation to the task it is expected to perform.

Back to top ↑

Model Selection, Validation, and Assessment

Validation asks whether a model is credible enough for a purpose. Model selection asks which model, among alternatives, should be used for that purpose. These practices are connected but not identical.

A model can be validated but not selected if another model is simpler, more robust, easier to communicate, or more appropriate for the decision context. A model can also be selected provisionally when no model is fully satisfactory but one is useful enough for limited use.

Selection should not erase uncertainty. The rejected models may still contain useful information about structural uncertainty, alternative assumptions, or failure modes.

Back to top ↑

Candidate Models and Baselines

Good comparison begins with a clear set of candidate models. Candidates should be chosen because they represent plausible assumptions, useful levels of complexity, alternative mechanisms, or different modeling purposes.

Baselines are especially important. A baseline is a simple reference model that provides a minimum standard. If a complex model does not outperform a reasonable baseline, its complexity may not be justified.

A comparison set should be broad enough to test meaningful alternatives but focused enough to remain interpretable and reviewable.

Back to top ↑

Comparison Criteria: Fit, Error, Complexity, and Purpose

Model comparison requires criteria. The criteria should be chosen before selection whenever possible. Otherwise, analysts may choose the model they prefer and then select metrics that justify it afterward.

Useful comparison criteria include in-sample fit, validation error, parsimony, interpretability, robustness, uncertainty, domain plausibility, computational cost, and decision relevance. These criteria may conflict.

Selection should state which criteria mattered most and why. A model chosen for prediction may differ from a model chosen for explanation or public communication.

Back to top ↑

Goodness-of-Fit and Its Limits

Goodness-of-fit measures how closely a model matches observed data. These measures are useful but limited. A model can fit known data well while failing under new conditions or representing the system poorly.

Common fit metrics include residual sum of squares, mean absolute error, root mean squared error, likelihood, and explained variance. Each emphasizes different features of model-data mismatch.

Goodness-of-fit should be treated as one part of model assessment, not as the entire basis for selection. A model that fits too well may be fitting noise.

Back to top ↑

Prediction, Validation Error, and Generalization

Generalization asks whether a model performs credibly beyond the data used to fit it. For predictive models, validation error is often more important than calibration error.

When a model performs well on calibration data and poorly on validation data, it may be overfitting. When a model performs poorly on both, it may be underfitting or structurally wrong. When a model performs reasonably on both, it may be more credible, but further review is still required.

Generalization is especially important when models are used for forecasting, decision support, risk assessment, and policy analysis. The model must perform under relevant future or external conditions, not only under known data.

Back to top ↑

Information Criteria and Parsimony

Information criteria help compare models by balancing fit and complexity. They penalize models that use more parameters to improve fit. This supports parsimony: the principle that additional complexity should earn its place.

Two common criteria are Akaike Information Criterion and Bayesian Information Criterion. They are often used for statistical models where likelihood can be computed.

Information criteria do not eliminate judgment. They help compare models under certain assumptions, but selection must still consider purpose, interpretability, uncertainty, and decision consequences.

Back to top ↑

Cross-Validation and Holdout Testing

Cross-validation and holdout testing estimate how well models generalize. The basic idea is to fit models on one portion of data and evaluate them on data not used for fitting.

Holdout testing uses a reserved validation set. Cross-validation repeats this process across multiple splits. Time-series and spatial models require special care because random splits can destroy temporal or spatial structure.

Validation-based selection should avoid leakage. If model selection, tuning, and performance assessment all use the same validation data, performance estimates can become overly optimistic.

Back to top ↑

Uncertainty, Sensitivity, and Robustness in Selection

Model selection should include uncertainty and robustness. A model with slightly better average error may be less trustworthy if its conclusions collapse under plausible parameter changes, data uncertainty, or structural alternatives.

Robust model selection asks whether the preferred model remains preferred across reasonable assumptions and whether the decision supported by the model is stable.

A robust model may be preferable to a slightly more accurate but fragile model, especially in decision contexts where failure has significant consequences.

Back to top ↑

Interpretability, Mechanism, and Decision Use

Model selection is not only a technical scoring problem. Interpretability matters when users need to understand why the model behaves as it does. Mechanism matters when the model is used to explain a system or evaluate interventions.

A highly accurate model may be inappropriate if its reasoning is opaque and the decision context requires explanation. A simpler model may be preferable if it is transparent, stable, and adequate for the purpose.

Interpretability is not a decorative preference. It is part of fitness for purpose. A model that cannot be explained may be unsuitable for some uses even if it scores well technically.

Back to top ↑

Mathematical Lens: Selection as Structured Tradeoff

Model selection can be represented as choosing among candidate models \(M_1, M_2, \ldots, M_K\) according to a purpose-specific assessment function.

A comparison criterion may combine error, complexity, and decision relevance:

Model comparison can also preserve alternatives rather than selecting only one model:

This mathematical lens shows that model selection is a structured tradeoff. The choice depends on evidence, purpose, complexity, uncertainty, and the consequences of being wrong.

Back to top ↑

Example: Selecting Among Resource Models

Consider a resource-management problem. Analysts want to model future resource stock under extraction policies. Several candidate models are available: a naive baseline, a linear trend model, a logistic growth model, and a stochastic shock model.

The selected model depends on purpose. If the task is public explanation, the logistic model may be best because it shows mechanism. If the task is near-term forecasting, the trend model might perform well enough. If the task is risk management, the stochastic model may be necessary despite added complexity.

Selection should document why the preferred model was chosen and what the alternatives revealed.

Back to top ↑

Model Ensembles and When Not to Select Only One Model

Sometimes the responsible choice is not a single model. When structural uncertainty is high, multiple plausible models may need to remain in view. Ensembles, scenario sets, and multimodel comparison can preserve uncertainty that a single selected model would hide.

Model selection should be honest about structural uncertainty. In complex systems, the question may not be “Which model is best?” but “Which conclusions are stable across plausible models?”

Back to top ↑

Ethical Stakes of Model Selection

Model selection has ethical stakes because chosen models shape what evidence is visible, what uncertainty is emphasized, which decisions appear justified, and which alternatives disappear from view.

A model may be selected because it supports a preferred conclusion, because it is easier to communicate, because it looks sophisticated, or because it performs well on a convenient metric. Responsible selection guards against these pressures.

Ethical model selection makes the basis for preference visible. It does not pretend that discarded alternatives never mattered.

Back to top ↑

Python Workflow: Model Comparison Register and Selection Diagnostics

The Python workflow below compares candidate models using calibration error, validation error, complexity, interpretability, and robustness. It exports a comparison table, selection register, model ranking, and selection audit card.

# model_comparison_and_selection_workflow.py
# Dependency-light model comparison and selection workflow.

from __future__ import annotations

from dataclasses import asdict, dataclass
from pathlib import Path
import csv
import json
import math


ARTICLE_ROOT = Path(__file__).resolve().parents[1]
OUTPUTS = ARTICLE_ROOT / "outputs"
TABLES = OUTPUTS / "tables"
JSON_DIR = OUTPUTS / "json"


@dataclass(frozen=True)
class ModelCandidate:
    model_id: str
    model_family: str
    calibration_rmse: float
    validation_rmse: float
    parameter_count: int
    interpretability_score: float
    robustness_score: float
    decision_relevance_score: float


@dataclass(frozen=True)
class SelectionRecord:
    key: str
    selection_layer: str
    modeling_role: str
    review_question: str
    status: str


def candidate_models() -> list[ModelCandidate]:
    return [
        ModelCandidate("baseline_naive", "baseline", 2.90, 3.05, 0, 0.95, 0.72, 0.55),
        ModelCandidate("linear_trend", "statistical", 1.80, 2.10, 2, 0.88, 0.70, 0.68),
        ModelCandidate("logistic_growth", "mechanistic", 1.25, 1.42, 3, 0.76, 0.82, 0.86),
        ModelCandidate("stochastic_shock", "stochastic", 1.05, 1.60, 6, 0.58, 0.88, 0.90),
        ModelCandidate("high_flex_curve", "flexible", 0.45, 2.75, 9, 0.35, 0.40, 0.52),
    ]


def selection_register() -> list[SelectionRecord]:
    return [
        SelectionRecord(
            key="candidate_set",
            selection_layer="alternatives",
            modeling_role="Defines the models being compared.",
            review_question="Are plausible baselines and alternatives included?",
            status="review",
        ),
        SelectionRecord(
            key="validation_error",
            selection_layer="generalization",
            modeling_role="Compares performance on data not used for fitting.",
            review_question="Does the selected model generalize?",
            status="active",
        ),
        SelectionRecord(
            key="complexity_penalty",
            selection_layer="parsimony",
            modeling_role="Penalizes unnecessary complexity.",
            review_question="Is added complexity justified by evidence?",
            status="review",
        ),
        SelectionRecord(
            key="interpretability",
            selection_layer="communication",
            modeling_role="Assesses whether model behavior can be explained.",
            review_question="Can users understand why this model was selected?",
            status="review",
        ),
        SelectionRecord(
            key="robustness",
            selection_layer="uncertainty",
            modeling_role="Reviews stability under assumptions and stress.",
            review_question="Does the preferred model remain credible under uncertainty?",
            status="review",
        ),
        SelectionRecord(
            key="decision_relevance",
            selection_layer="decision_support",
            modeling_role="Links model selection to the intended use.",
            review_question="Does model performance matter for the decision?",
            status="review",
        ),
    ]


def complexity_penalty(parameter_count: int) -> float:
    return 0.08 * parameter_count


def comparison_score(model: ModelCandidate) -> float:
    return round(
        model.validation_rmse
        + complexity_penalty(model.parameter_count)
        - 0.35 * model.interpretability_score
        - 0.40 * model.robustness_score
        - 0.35 * model.decision_relevance_score,
        8,
    )


def overfit_gap(model: ModelCandidate) -> float:
    return round(model.validation_rmse - model.calibration_rmse, 8)


def model_rows(models: list[ModelCandidate]) -> list[dict[str, object]]:
    rows = []
    for model in models:
        gap = overfit_gap(model)
        rows.append({
            **asdict(model),
            "overfit_gap": gap,
            "complexity_penalty": round(complexity_penalty(model.parameter_count), 8),
            "comparison_score": comparison_score(model),
            "overfit_flag": gap > 1.0,
        })
    return sorted(rows, key=lambda row: float(row["comparison_score"]))


def selection_risk_score(record: SelectionRecord) -> float:
    score = {"active": 1.0, "review": 5.0, "revise": 8.0, "archive": 2.0}.get(
        record.status.lower(),
        4.0,
    )
    text = f"{record.selection_layer} {record.modeling_role} {record.review_question}".lower()
    for term in ["alternative", "validation", "complexity", "uncertainty", "decision", "robust", "interpret"]:
        if term in text:
            score += 1.0
    return round(score, 3)


def write_csv(path: Path, rows: list[dict[str, object]]) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    if not rows:
        raise ValueError(f"No rows supplied for {path}")
    with path.open("w", newline="", encoding="utf-8") as handle:
        writer = csv.DictWriter(handle, fieldnames=list(rows[0].keys()))
        writer.writeheader()
        writer.writerows(rows)


def write_json(path: Path, payload: object) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with path.open("w", encoding="utf-8") as handle:
        json.dump(payload, handle, indent=2, sort_keys=True)


def main() -> None:
    models = candidate_models()
    records = selection_register()

    ranked = model_rows(models)
    selected = ranked[0]

    register_rows = [
        {**asdict(record), "selection_risk_score": selection_risk_score(record)}
        for record in records
    ]

    write_csv(TABLES / "model_comparison_table.csv", ranked)
    write_csv(TABLES / "model_selection_register.csv", register_rows)

    write_json(JSON_DIR / "model_selection_audit_card.json", {
        "article": "Model Comparison and Selection",
        "selected_model": selected,
        "ranking_method": "Validation error plus complexity penalty minus interpretability, robustness, and decision-relevance credits.",
        "overfit_warning_models": [row for row in ranked if bool(row["overfit_flag"])],
        "selection_register": register_rows,
        "use_limit": "Selection is purpose-specific and should not be generalized beyond the comparison criteria.",
        "diagnostic_checks": [
            "candidate models include a baseline",
            "validation error is separated from calibration error",
            "complexity penalty is visible",
            "overfit gap is reported",
            "interpretability and decision relevance are included",
            "alternative models are preserved rather than erased",
        ],
    })

    print("Model comparison and selection workflow complete.")
    print(f"Selected model: {selected['model_id']}")
    print(f"Wrote outputs to {OUTPUTS}")


if __name__ == "__main__":
    main()

This workflow preserves the comparison logic. It records candidate models, validation error, complexity, interpretability, robustness, decision relevance, overfit gaps, and the final selection rationale.

Back to top ↑

R Workflow: Model Selection Review and Comparison Diagnostics

The R workflow below reviews the comparison table, classifies overfitting risk, and creates a base R plot comparing calibration and validation error by candidate model.

# model_comparison_and_selection_review.R
# Base R workflow for model comparison and selection diagnostics.

args <- commandArgs(trailingOnly = FALSE)
file_arg <- grep("^--file=", args, value = TRUE)

if (length(file_arg) > 0) {
  script_path <- normalizePath(sub("^--file=", "", file_arg[1]), mustWork = TRUE)
  article_root <- normalizePath(file.path(dirname(script_path), ".."), mustWork = TRUE)
} else {
  article_root <- getwd()
}

tables_dir <- file.path(article_root, "outputs", "tables")
figures_dir <- file.path(article_root, "outputs", "figures")
dir.create(tables_dir, recursive = TRUE, showWarnings = FALSE)
dir.create(figures_dir, recursive = TRUE, showWarnings = FALSE)

comparison_path <- file.path(tables_dir, "model_comparison_table.csv")
register_path <- file.path(tables_dir, "model_selection_register.csv")

if (!file.exists(comparison_path) || !file.exists(register_path)) {
  stop("Missing model comparison outputs. Run the Python workflow first.")
}

comparison <- read.csv(comparison_path, stringsAsFactors = FALSE)
register <- read.csv(register_path, stringsAsFactors = FALSE)

comparison$calibration_rmse <- as.numeric(comparison$calibration_rmse)
comparison$validation_rmse <- as.numeric(comparison$validation_rmse)
comparison$overfit_gap <- as.numeric(comparison$overfit_gap)
comparison$comparison_score <- as.numeric(comparison$comparison_score)

comparison$overfit_risk <- ifelse(
  comparison$overfit_gap > 1.0,
  "high overfit risk",
  ifelse(comparison$overfit_gap > 0.5, "moderate overfit risk", "lower overfit risk")
)

register$priority <- ifelse(
  register$selection_risk_score >= 8,
  "high",
  ifelse(register$selection_risk_score >= 6, "medium", "low")
)

selected_model <- comparison[which.min(comparison$comparison_score), ]

write.csv(
  comparison,
  file.path(tables_dir, "r_model_comparison_review.csv"),
  row.names = FALSE
)

write.csv(
  register,
  file.path(tables_dir, "r_model_selection_review_queue.csv"),
  row.names = FALSE
)

write.csv(
  selected_model,
  file.path(tables_dir, "r_selected_model_summary.csv"),
  row.names = FALSE
)

png(file.path(figures_dir, "r_model_comparison_error_plot.png"), width = 1100, height = 750)

mat <- rbind(comparison$calibration_rmse, comparison$validation_rmse)
colnames(mat) <- comparison$model_id

barplot(
  mat,
  beside = TRUE,
  las = 2,
  ylab = "RMSE",
  main = "Calibration vs Validation Error"
)
legend("topright", legend = c("Calibration RMSE", "Validation RMSE"), fill = gray.colors(2))

dev.off()

print(selected_model)
print(comparison)
print(register)

The R layer helps reveal whether a candidate model looks strong only because it overfits calibration data. It also preserves the selected model summary and review queue.

Back to top ↑

Haskell Workflow: Typed Model Selection Records

Haskell is useful here because model selection concepts should remain distinct. A candidate set is not a validation metric. A complexity penalty is not a decision. A selected model is not a universal truth.

{-# OPTIONS_GHC -Wall #-}

module Main where

data SelectionLayer
  = Alternatives
  | Generalization
  | Parsimony
  | Communication
  | Uncertainty
  | DecisionSupport
  | Governance
  deriving (Eq, Show)

data ReviewStatus
  = Active
  | RequiresReview
  | RequiresValidation
  | RequiresUncertaintyCheck
  | Revise
  deriving (Eq, Show)

data SelectionRecord = SelectionRecord
  { key :: String
  , layer :: SelectionLayer
  , modelingRole :: String
  , reviewFocus :: String
  , status :: ReviewStatus
  } deriving (Eq, Show)

selectionRegister :: [SelectionRecord]
selectionRegister =
  [ SelectionRecord
      "candidate_set"
      Alternatives
      "Defines the models being compared."
      "Plausible baselines and alternatives."
      RequiresReview
  , SelectionRecord
      "validation_error"
      Generalization
      "Compares performance beyond fitting data."
      "Generalization."
      Active
  , SelectionRecord
      "complexity_penalty"
      Parsimony
      "Penalizes unnecessary complexity."
      "Complexity justification."
      RequiresReview
  , SelectionRecord
      "interpretability"
      Communication
      "Assesses whether model behavior can be explained."
      "User understanding."
      RequiresReview
  , SelectionRecord
      "robustness"
      Uncertainty
      "Reviews stability under assumptions and stress."
      "Uncertainty-aware selection."
      RequiresUncertaintyCheck
  , SelectionRecord
      "decision_relevance"
      DecisionSupport
      "Links model selection to intended use."
      "Fitness for purpose."
      RequiresValidation
  ]

needsReview :: SelectionRecord -> Bool
needsReview item =
  case status item of
    Active -> False
    _ -> True

main :: IO ()
main = do
  putStrLn "Typed model selection records:"
  mapM_ print selectionRegister

  putStrLn "\nSelection records requiring review:"
  mapM_ print (filter needsReview selectionRegister)

This typed layer supports model selection governance by keeping alternatives, generalization, parsimony, interpretability, uncertainty, and decision relevance conceptually separate.

Back to top ↑

GitHub Repository

The companion repository for this article is designed as a reproducible mathematical-modeling workspace. It contains article-specific code, data, documentation, notebooks, schemas, and generated outputs for model comparison registers, candidate model scoring, validation error comparison, complexity penalties, overfit-gap diagnostics, interpretability and robustness review, typed Haskell selection records, and responsible decision-support workflows.

Back to top ↑

A Practical Method for Model Comparison and Selection

Model comparison should follow a documented process. The goal is not simply to find a winning score, but to justify why a model is preferred for a purpose.

This method keeps selection accountable. It ensures that the chosen model is not merely the best-looking model, but the model whose evidence and tradeoffs best fit the intended use.

Back to top ↑

Common Pitfalls

Model comparison can become misleading when criteria are vague, alternatives are weak, or the selection process is shaped after the desired result is already known.

• No baseline: selecting a complex model without checking whether a simple model performs nearly as well.
• Calibration-only selection: choosing the model that fits known data best while ignoring validation error.
• Overfitting hidden by score: allowing flexible models to win because they fit noise.
• Metric shopping: selecting the metric that favors a preferred model.
• Ignoring complexity: failing to ask whether extra parameters are justified.
• Ignoring interpretability: choosing a model users cannot understand in a context where explanation matters.
• Ignoring uncertainty: presenting a single selected model as if alternatives no longer matter.
• Weak candidate set: comparing only models that make the selected model look good.
• Unclear purpose: selecting one model without saying whether it is for explanation, prediction, control, or decision support.
• No selection record: leaving future users unable to understand why a model was chosen.

These pitfalls can be reduced through baselines, validation evidence, parsimony criteria, robustness review, transparent metrics, candidate registers, and selection audit cards.

Back to top ↑

Conclusion: Selection Is a Judgment About Purpose

Model comparison and selection help analysts decide which model is most credible, useful, and appropriate for a specific purpose. Selection is not a mechanical search for the model with the lowest error score. It is a structured judgment about evidence, complexity, uncertainty, interpretability, robustness, and decision relevance.

A model that fits calibration data best may overfit. A model that predicts well may be too opaque for public reasoning. A model that is simple may be preferable when it is transparent and adequate. A model that is complex may be justified when it captures essential dynamics and performs robustly.

Responsible model selection documents alternatives, criteria, diagnostics, uncertainty, and use limits. It preserves the fact that different models answer different questions and that rejected models may still reveal important uncertainty.

Used well, model comparison and selection help analysts avoid false precision, justify modeling choices, communicate tradeoffs, and support accountable decisions. The selected model is not the final truth. It is the best-supported choice for a stated purpose, under stated evidence and limits.

Back to top ↑

Related Articles

• What Is Mathematical Modeling?
• Model Purpose: Explanation, Prediction, Control, and Decision Support
• Optimization Models and Objective Functions
• Calibration, Estimation, and Parameter Fitting
• Validation and Model Assessment
• Overfitting, Underfitting, and Model Generalization
• Diagnostics, Residuals, and Model Error
• Sensitivity Analysis and Robustness
• Uncertainty in Mathematical Models
• Model Interpretation and Decision-Making

Back to top ↑

Further Reading

• Burnham, K.P. and Anderson, D.R. (2002) Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. 2nd edn. New York: Springer.
• Hastie, T., Tibshirani, R. and Friedman, J. (2009) The Elements of Statistical Learning. 2nd edn. New York: Springer.
• Claeskens, G. and Hjort, N.L. (2008) Model Selection and Model Averaging. Cambridge: Cambridge University Press.
• Gelman, A. et al. (2013) Bayesian Data Analysis. 3rd edn. Boca Raton, FL: CRC Press.
• Konishi, S. and Kitagawa, G. (2008) Information Criteria and Statistical Modeling. New York: Springer.
• Arlot, S. and Celisse, A. (2010) ‘A survey of cross-validation procedures for model selection’, Statistics Surveys, 4, pp. 40–79.
• Stone, M. (1974) ‘Cross-validatory choice and assessment of statistical predictions’, Journal of the Royal Statistical Society: Series B, 36(2), pp. 111–147.
• Box, G.E.P., Hunter, J.S. and Hunter, W.G. (2005) Statistics for Experimenters: Design, Innovation, and Discovery. 2nd edn. Hoboken, NJ: Wiley.
• Saltelli, A. et al. (2008) Global Sensitivity Analysis: The Primer. Chichester: Wiley.
• Oberkampf, W.L. and Roy, C.J. (2010) Verification and Validation in Scientific Computing. Cambridge: Cambridge University Press.

Back to top ↑

References

• Arlot, S. and Celisse, A. (2010) ‘A survey of cross-validation procedures for model selection’, Statistics Surveys, 4, pp. 40–79.
• Box, G.E.P., Hunter, J.S. and Hunter, W.G. (2005) Statistics for Experimenters: Design, Innovation, and Discovery. 2nd edn. Hoboken, NJ: Wiley.
• Burnham, K.P. and Anderson, D.R. (2002) Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. 2nd edn. New York: Springer.
• Claeskens, G. and Hjort, N.L. (2008) Model Selection and Model Averaging. Cambridge: Cambridge University Press.
• Gelman, A. et al. (2013) Bayesian Data Analysis. 3rd edn. Boca Raton, FL: CRC Press.
• Hastie, T., Tibshirani, R. and Friedman, J. (2009) The Elements of Statistical Learning. 2nd edn. New York: Springer.
• Konishi, S. and Kitagawa, G. (2008) Information Criteria and Statistical Modeling. New York: Springer.
• Oberkampf, W.L. and Roy, C.J. (2010) Verification and Validation in Scientific Computing. Cambridge: Cambridge University Press.
• Saltelli, A. et al. (2008) Global Sensitivity Analysis: The Primer. Chichester: Wiley.
• Stone, M. (1974) ‘Cross-validatory choice and assessment of statistical predictions’, Journal of the Royal Statistical Society: Series B, 36(2), pp. 111–147.

Back to top ↑