Model Repositories, Data, and Reproducible Research

Last Updated June 12, 2026

Model repositories, data, and reproducible research turn mathematical modeling into a transparent, inspectable, and reusable practice. A model is not fully reproducible when equations, code, data, parameters, environments, outputs, and documentation are scattered or incomplete.

Mathematical modeling increasingly depends on computational workflows. A model may include equations, simulation code, numerical methods, data transformations, calibration routines, validation checks, uncertainty analyses, figures, generated tables, and decision-support summaries. Without a repository structure, those materials can become difficult to rerun, review, or preserve.

A model repository is more than a place to store code. It is an organized evidence system. It shows how assumptions, data, methods, software environments, tests, outputs, and documentation fit together. Reproducible research depends on that structure.

Responsible repository design asks practical questions. Can another analyst find the model inputs? Can they rerun the workflow? Can they identify which code generated which output? Can they inspect assumptions? Can they understand the data provenance? Can they distinguish raw data from processed data? Can they cite the repository? Can they know what should not be reused?

Why Model Repositories Matter

Model repositories matter because modeling claims need evidence. A published result, figure, parameter estimate, simulation output, or decision-support recommendation should be traceable to code, data, assumptions, and computational settings. Without that traceability, the result may be difficult to verify or reuse.

Repositories help make modeling work durable. They preserve the materials needed to rerun analysis, inspect assumptions, revise models, compare versions, train collaborators, support peer review, and communicate limitations.

A repository does not automatically make research reproducible. But without repository discipline, reproducibility becomes much harder to achieve.

Back to top ↑

What a Model Repository Is

A model repository is a structured collection of materials needed to understand, run, inspect, validate, and reuse a model. It usually includes code, data, documentation, metadata, workflows, generated outputs, tests, and governance notes.

In computational modeling, the repository becomes part of the model’s public record. It shows what was implemented, which assumptions were active, what data were used, how outputs were produced, and what limitations remain.

The repository should make the modeling workflow legible. It should help someone see not only what the model produced, but how the result came into being.

Back to top ↑

What Reproducible Research Means

Reproducible research means that the reported results can be regenerated from documented data, code, configuration, environment, and workflow instructions. In modeling, this includes both computational reproducibility and interpretive transparency.

Reproducibility is not the same as correctness. A wrong model can be reproducible. But reproducibility makes the model available for inspection, testing, correction, extension, and critique.

For model-based work, reproducibility should include assumptions, not only code. If a workflow can be rerun but its assumptions remain hidden, the research is computationally available but not fully accountable.

Back to top ↑

Repository Structure and Project Organization

A clear repository structure helps users know where to find code, data, documentation, outputs, notebooks, schemas, and review materials. The exact structure depends on the project, but consistent organization reduces confusion and supports collaboration.

Repository structure should support the intended audience. A research collaborator, reviewer, student, policy analyst, or future maintainer should be able to understand the project without reconstructing it from scattered files.

Back to top ↑

Data Provenance, Raw Data, and Processed Data

Data provenance records where data came from, how it was collected, how it was transformed, and how it entered the model. Without provenance, model outputs become detached from their evidentiary foundation.

A reproducible repository should distinguish raw data from processed data. Raw data should usually be preserved unchanged when possible. Processed data should be generated through documented scripts rather than manually edited.

Data handling is part of model design. A cleaning rule, aggregation level, imputation method, or unit conversion can change model outputs. Reproducible research makes those choices visible.

Back to top ↑

Metadata, Schemas, and Documentation

Metadata describes the repository and its contents. Schemas define what data and configuration files should contain. Documentation explains how the model works, how to run it, and how outputs should be interpreted.

Good metadata lowers the cost of review. It tells users what the repository is, who created it, what version they are using, what data are included, what licenses apply, and what the model is intended to support.

Documentation should not be decorative. It should help another person run, inspect, validate, cite, and responsibly reuse the repository.

Back to top ↑

Version Control, Change History, and Releases

Version control records how a model repository changes over time. This matters because model outputs depend on specific versions of code, data, assumptions, configuration, and dependencies.

A repository should make it possible to identify the version that produced a given result. Commit history, release tags, changelogs, and archived releases help preserve that connection.

Version control helps prevent model drift. It makes model evolution visible and allows analysts to distinguish stable, reviewed workflows from exploratory work.

Back to top ↑

Environments, Dependencies, and Computational Context

A reproducible repository should describe its computational environment. Software versions, package dependencies, operating systems, compilers, numerical libraries, and random-number generators can affect results.

Environment capture can be lightweight or formal. A small educational model may need a simple requirements file. A high-stakes scientific workflow may need lockfiles, containers, and automated tests.

Computational context should be documented enough that future analysts understand what was run and what may change if the environment changes.

Back to top ↑

Workflows, Scripts, Tests, and Automation

Reproducible repositories need executable workflows. A model should not require someone to guess which files to run or in what order. Scripts, command-line interfaces, Makefiles, task runners, and workflow managers make execution explicit.

Tests and smoke checks help confirm that the repository still works. They do not prove the model is valid, but they help catch broken code, missing files, invalid schemas, or failed output generation.

Automation should make modeling work easier to review, not more opaque. A good command should produce documented outputs and diagnostics that can be inspected afterward.

Back to top ↑

Outputs, Logs, Archives, and Evidence Trails

Generated outputs should be organized and traceable. Tables, figures, logs, JSON summaries, model cards, validation reports, and run manifests are part of the evidence trail.

Not every output needs to be committed to a repository. Large generated files may be stored externally or regenerated on demand. But the repository should make clear which outputs are canonical, which are examples, and how outputs can be reproduced.

Evidence trails help maintain trust. They allow future users to see not only the final result, but the computational path that produced it.

Back to top ↑

Licensing, Citation, and Reuse

Reproducible research should clarify how code, data, figures, and documentation may be reused. Code licenses and data licenses may differ. Sensitive or restricted data may not be publishable even when code can be shared.

Citation metadata helps others give credit and identify the exact version used. For model repositories, versioned releases and archival identifiers are especially helpful.

Reuse is not only a legal issue. It is an interpretive issue. A repository should help others understand what can be reused responsibly and what remains context-specific.

Back to top ↑

Mathematical Lens: Reproducibility as a Mapping

A reproducible model repository can be understood as a structured mapping from documented inputs to documented outputs.

The repository also produces diagnostic evidence:

Reproducibility asks whether another user can regenerate the output and evidence:

This lens clarifies why repositories matter. A model result is not only a number or graph. It is the product of a documented transformation that should be available for review.

Back to top ↑

Example: Reproducible Resource Model Repository

Consider a resource dynamics model. The research question concerns whether extraction policies keep resource stock above a safety threshold under uncertainty. A reproducible repository would preserve the model, data, assumptions, scripts, outputs, and diagnostics needed to inspect that claim.

This repository does more than store files. It preserves the reasoning trail from question to evidence.

Back to top ↑

Repository Governance and Decision Support

When models inform decisions, repositories need governance. Governance asks whether the repository is complete enough to support the intended claim and whether users can understand the model’s limits.

A decision-support repository should distinguish reviewed outputs from exploratory outputs, public data from restricted data, validated routines from experimental scripts, and stable releases from active development.

Repository governance helps keep models from becoming decontextualized artifacts. It preserves the conditions under which a result should be trusted, questioned, or revised.

Back to top ↑

Ethical Stakes of Reproducible Research

Reproducible research is an ethical practice because model outputs can influence public policy, engineering decisions, environmental management, health planning, financial allocation, institutional strategy, and public understanding.

When repositories are incomplete, opaque, or misleading, users may treat model outputs as more reliable than they are. When repositories are well designed, they invite scrutiny and improve accountability.

Reproducibility is not only about convenience. It is about allowing claims to be inspected, corrected, challenged, and responsibly reused.

Back to top ↑

Python Workflow: Repository Audit and Reproducibility Manifest

The Python workflow below creates a repository audit register, checks expected project files, writes an output index, and generates a reproducibility manifest. It is dependency-light and designed to support repository governance.

# model_repository_reproducibility_audit.py
# Dependency-light repository audit and reproducibility manifest example.

from __future__ import annotations

from dataclasses import asdict, dataclass
from datetime import datetime, timezone
from pathlib import Path
import csv
import hashlib
import json
import platform
import sys


ARTICLE_ROOT = Path(__file__).resolve().parents[1]
OUTPUTS = ARTICLE_ROOT / "outputs"
TABLES = OUTPUTS / "tables"
JSON_DIR = OUTPUTS / "json"
LOGS = OUTPUTS / "logs"


@dataclass(frozen=True)
class RepositoryRecord:
    key: str
    repository_layer: str
    artifact: str
    modeling_role: str
    review_question: str
    status: str


@dataclass(frozen=True)
class ExpectedArtifact:
    artifact: str
    path: str
    required: bool
    purpose: str


def repository_register() -> list[RepositoryRecord]:
    return [
        RepositoryRecord(
            key="readme",
            repository_layer="documentation",
            artifact="README.md",
            modeling_role="Explains project purpose, structure, setup, and run commands.",
            review_question="Can a new analyst understand and run the repository?",
            status="review",
        ),
        RepositoryRecord(
            key="metadata",
            repository_layer="metadata",
            artifact="article-metadata.yml",
            modeling_role="Records title, slug, focus keyword, tags, and excerpt.",
            review_question="Is repository metadata complete and consistent?",
            status="active",
        ),
        RepositoryRecord(
            key="data_provenance",
            repository_layer="data",
            artifact="data provenance notes and schemas",
            modeling_role="Documents data sources, transformations, and constraints.",
            review_question="Can inputs be traced to their sources?",
            status="review",
        ),
        RepositoryRecord(
            key="run_manifest",
            repository_layer="reproducibility",
            artifact="run_manifest.json",
            modeling_role="Records execution context and output hashes.",
            review_question="Can outputs be regenerated and checked?",
            status="active",
        ),
        RepositoryRecord(
            key="model_card",
            repository_layer="governance",
            artifact="model_card.json",
            modeling_role="Summarizes purpose, assumptions, validation, and use limits.",
            review_question="Are intended use and limits visible?",
            status="review",
        ),
    ]


def expected_artifacts() -> list[ExpectedArtifact]:
    return [
        ExpectedArtifact("README", "README.md", True, "Project overview and run instructions."),
        ExpectedArtifact("metadata", "article-metadata.yml", True, "Article and repository metadata."),
        ExpectedArtifact("Makefile", "Makefile", True, "Repeatable workflow targets."),
        ExpectedArtifact("Python package", "python", True, "Executable model and audit code."),
        ExpectedArtifact("R workflow", "r", False, "Independent review workflow."),
        ExpectedArtifact("SQL schema", "sql/schema.sql", False, "Structured governance tables."),
        ExpectedArtifact("data folder", "data", True, "Data, metadata, and scenario files."),
        ExpectedArtifact("docs folder", "docs", True, "Documentation and governance notes."),
        ExpectedArtifact("outputs folder", "outputs", True, "Generated tables, figures, JSON, and logs."),
        ExpectedArtifact("schemas folder", "schemas", False, "Machine-readable validation schemas."),
        ExpectedArtifact("canvas manifest", "canvas/canvas_manifest.json", False, "Catalyst Canvas governance metadata."),
    ]


def hash_file(path: Path) -> str:
    if not path.exists() or not path.is_file():
        return "not_applicable"
    digest = hashlib.sha256()
    with path.open("rb") as handle:
        for block in iter(lambda: handle.read(65536), b""):
            digest.update(block)
    return digest.hexdigest()


def artifact_inventory(root: Path, artifacts: list[ExpectedArtifact]) -> list[dict[str, object]]:
    rows = []
    for artifact in artifacts:
        path = root / artifact.path
        rows.append({
            "artifact": artifact.artifact,
            "path": artifact.path,
            "required": artifact.required,
            "exists": path.exists(),
            "is_file": path.is_file(),
            "is_dir": path.is_dir(),
            "sha256": hash_file(path),
            "purpose": artifact.purpose,
            "review_status": "present" if path.exists() else ("missing_required" if artifact.required else "missing_optional"),
        })
    return rows


def repository_risk_score(record: RepositoryRecord) -> float:
    score = {"active": 1.0, "review": 5.0, "revise": 8.0, "archive": 2.0}.get(
        record.status.lower(),
        4.0,
    )
    text = f"{record.repository_layer} {record.artifact} {record.review_question}".lower()
    for term in ["data", "manifest", "metadata", "governance", "schema", "reproduce", "license"]:
        if term in text:
            score += 1.0
    return round(score, 3)


def write_csv(path: Path, rows: list[dict[str, object]]) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    if not rows:
        raise ValueError(f"No rows supplied for {path}")
    with path.open("w", newline="", encoding="utf-8") as handle:
        writer = csv.DictWriter(handle, fieldnames=list(rows[0].keys()))
        writer.writeheader()
        writer.writerows(rows)


def write_json(path: Path, payload: object) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    with path.open("w", encoding="utf-8") as handle:
        json.dump(payload, handle, indent=2, sort_keys=True)


def main() -> None:
    records = repository_register()
    artifacts = expected_artifacts()
    inventory = artifact_inventory(ARTICLE_ROOT, artifacts)

    register_rows = [
        {**asdict(record), "repository_risk_score": repository_risk_score(record)}
        for record in records
    ]

    inventory_path = TABLES / "repository_artifact_inventory.csv"
    register_path = TABLES / "repository_audit_register.csv"
    manifest_path = JSON_DIR / "reproducibility_manifest.json"
    model_card_path = JSON_DIR / "model_repository_card.json"

    write_csv(inventory_path, inventory)
    write_csv(register_path, register_rows)

    manifest = {
        "article": "Model Repositories, Data, and Reproducible Research",
        "run_timestamp_utc": datetime.now(timezone.utc).isoformat(),
        "python_version": sys.version,
        "platform": platform.platform(),
        "repository_root": str(ARTICLE_ROOT),
        "artifact_inventory": inventory,
        "audit_register": register_rows,
        "required_artifacts_missing": [
            row for row in inventory
            if bool(row["required"]) and not bool(row["exists"])
        ],
    }

    write_json(manifest_path, manifest)

    write_json(model_card_path, {
        "article": "Model Repositories, Data, and Reproducible Research",
        "model_repository_purpose": "Demonstrate reproducible model repository design.",
        "intended_use": "Educational and analytical workflow governance.",
        "not_for": "Direct operational decisions without domain-specific validation.",
        "audit_checks": [
            "README and metadata are present",
            "data and documentation folders are present",
            "workflow targets are defined",
            "artifact inventory is generated",
            "reproducibility manifest is written",
        ],
    })

    LOGS.mkdir(parents=True, exist_ok=True)
    (LOGS / "repository_audit.log").write_text(
        "Repository reproducibility audit completed successfully.\n",
        encoding="utf-8",
    )

    print("Repository reproducibility audit complete.")
    print(f"Wrote outputs to {OUTPUTS}")


if __name__ == "__main__":
    main()

This workflow treats repository structure itself as an auditable modeling object. It checks expected artifacts, records metadata, and writes a reproducibility manifest.

Back to top ↑

R Workflow: Repository Review and Data Diagnostics

The R workflow below reviews the generated repository audit outputs, classifies missing artifacts, and creates a simple base R summary of repository completeness.

# model_repository_reproducibility_review.R
# Base R workflow for repository audit review.

args <- commandArgs(trailingOnly = FALSE)
file_arg <- grep("^--file=", args, value = TRUE)

if (length(file_arg) > 0) {
  script_path <- normalizePath(sub("^--file=", "", file_arg[1]), mustWork = TRUE)
  article_root <- normalizePath(file.path(dirname(script_path), ".."), mustWork = TRUE)
} else {
  article_root <- getwd()
}

tables_dir <- file.path(article_root, "outputs", "tables")
figures_dir <- file.path(article_root, "outputs", "figures")
dir.create(tables_dir, recursive = TRUE, showWarnings = FALSE)
dir.create(figures_dir, recursive = TRUE, showWarnings = FALSE)

inventory_path <- file.path(tables_dir, "repository_artifact_inventory.csv")
register_path <- file.path(tables_dir, "repository_audit_register.csv")

if (!file.exists(inventory_path) || !file.exists(register_path)) {
  stop("Missing repository audit outputs. Run the Python workflow first.")
}

inventory <- read.csv(inventory_path, stringsAsFactors = FALSE)
register <- read.csv(register_path, stringsAsFactors = FALSE)

inventory$exists_logical <- inventory$exists == "True" | inventory$exists == TRUE
inventory$required_logical <- inventory$required == "True" | inventory$required == TRUE

inventory$review_class <- ifelse(
  inventory$required_logical & !inventory$exists_logical,
  "missing required artifact",
  ifelse(inventory$exists_logical, "present", "missing optional artifact")
)

register$priority <- ifelse(
  register$repository_risk_score >= 8,
  "high",
  ifelse(register$repository_risk_score >= 6, "medium", "low")
)

completeness_summary <- aggregate(
  exists_logical ~ required_logical,
  data = inventory,
  FUN = function(x) round(mean(x), 4)
)

names(completeness_summary) <- c("required_artifact", "presence_rate")

write.csv(
  inventory,
  file.path(tables_dir, "r_repository_artifact_review.csv"),
  row.names = FALSE
)

write.csv(
  register,
  file.path(tables_dir, "r_repository_review_queue.csv"),
  row.names = FALSE
)

write.csv(
  completeness_summary,
  file.path(tables_dir, "r_repository_completeness_summary.csv"),
  row.names = FALSE
)

png(file.path(figures_dir, "r_repository_artifact_completeness.png"), width = 1000, height = 700)

counts <- table(inventory$review_class)

barplot(
  counts,
  las = 2,
  ylab = "Artifact count",
  main = "Repository Artifact Review"
)

grid()

dev.off()

print(completeness_summary)
print(register)

The R layer helps review repository completeness and flags missing required artifacts. It is intentionally simple so that repository diagnostics remain easy to inspect.

Back to top ↑

Haskell Workflow: Typed Repository Records

Haskell is useful here because repository components should remain distinct. Data provenance is not a license. A run manifest is not a README. A model card is not a test suite.

{-# OPTIONS_GHC -Wall #-}

module Main where

data RepositoryLayer
  = Documentation
  | DataLayer
  | CodeLayer
  | Metadata
  | Reproducibility
  | Validation
  | Governance
  | Licensing
  deriving (Eq, Show)

data ReviewStatus
  = Active
  | RequiresReview
  | RequiresValidation
  | RequiresArchiveCheck
  | Revise
  deriving (Eq, Show)

data RepositoryRecord = RepositoryRecord
  { key :: String
  , layer :: RepositoryLayer
  , artifact :: String
  , modelingRole :: String
  , reviewFocus :: String
  , status :: ReviewStatus
  } deriving (Eq, Show)

repositoryRegister :: [RepositoryRecord]
repositoryRegister =
  [ RepositoryRecord
      "readme"
      Documentation
      "README.md"
      "Explains project purpose, structure, setup, and run commands."
      "Usability and onboarding."
      RequiresReview
  , RepositoryRecord
      "data_provenance"
      DataLayer
      "data provenance notes"
      "Documents sources, transformations, units, and constraints."
      "Evidence traceability."
      RequiresReview
  , RepositoryRecord
      "run_manifest"
      Reproducibility
      "reproducibility_manifest.json"
      "Records execution context and output hashes."
      "Rerun capability."
      Active
  , RepositoryRecord
      "model_card"
      Governance
      "model_repository_card.json"
      "Summarizes purpose, assumptions, validation, and use limits."
      "Decision-support governance."
      RequiresValidation
  ]

needsReview :: RepositoryRecord -> Bool
needsReview item =
  case status item of
    Active -> False
    _ -> True

main :: IO ()
main = do
  putStrLn "Typed model repository records:"
  mapM_ print repositoryRegister

  putStrLn "\nRepository records requiring review:"
  mapM_ print (filter needsReview repositoryRegister)

This typed layer supports repository governance by keeping documentation, data, metadata, reproducibility, validation, licensing, and governance conceptually separate.

Back to top ↑

GitHub Repository

The companion repository for this article is designed as a reproducible mathematical-modeling workspace. It contains article-specific code, data, documentation, notebooks, schemas, and generated outputs for repository audits, artifact inventories, reproducibility manifests, model repository cards, output hashes, typed Haskell repository records, data provenance review, validation planning, and responsible decision-support workflows.

Back to top ↑

A Practical Method for Reproducible Model Repository Design

Reproducible repository design should begin with the question: what would another analyst need in order to inspect, rerun, validate, and responsibly reuse this model?

This method treats a repository as modeling infrastructure. It preserves not only the code, but also the context needed to understand the model’s evidence.

Back to top ↑

Common Pitfalls

Model repositories can look organized while still being difficult to reproduce. The most common failures involve missing context, undocumented assumptions, or unclear execution paths.

• Code-only repository: sharing scripts without data, parameters, outputs, or documentation.
• Data without provenance: storing files without source, date, version, units, or transformation notes.
• Manual workflow: requiring users to guess execution order.
• Notebook-only reproducibility: relying on hidden execution state or manual cell order.
• Untracked outputs: publishing figures without linking them to generating scripts.
• No environment capture: ignoring package versions, compilers, and runtime context.
• Missing license: leaving reuse rights ambiguous.
• No citation metadata: making scholarly reuse difficult.
• Sensitive-data leakage: publishing data that should be restricted or synthetic.
• No use-limit statement: allowing models to be reused beyond their intended scope.

These pitfalls can be reduced through repository templates, structured documentation, workflow automation, validation checks, data governance, version control, licensing, citation metadata, and model cards.

Back to top ↑

Conclusion: Reproducibility Is Modeling Accountability

Model repositories, data, and reproducible research make mathematical modeling more transparent, inspectable, and reusable. They preserve the code, data, configuration, environments, outputs, tests, metadata, and documentation needed to understand how results were produced.

Reproducibility does not guarantee truth. It does something more foundational: it allows claims to be checked. It makes assumptions visible, workflows runnable, outputs traceable, and limitations easier to evaluate.

Responsible model repositories support better science, engineering, policy, sustainability, education, and complex systems practice because they turn modeling from a private calculation into a reviewable evidence system.

Used well, repositories help analysts rerun results, inspect assumptions, validate claims, share methods, preserve evidence, and improve modeling work over time. They make mathematical modeling not only more computationally powerful, but more accountable.

Back to top ↑

Related Articles

• What Is Mathematical Modeling?
• Simulation and Computational Modeling
• Numerical Methods for Mathematical Models
• Monte Carlo Simulation and Uncertainty Propagation
• Scientific Computing for Modeling Workflows
• Calibration, Estimation, and Parameter Fitting
• Validation and Model Assessment
• Model Comparison and Selection
• Uncertainty in Mathematical Models
• Model Governance and Accountability

Back to top ↑

Further Reading

• Stodden, V., Leisch, F. and Peng, R.D. (eds) (2014) Implementing Reproducible Research. Boca Raton, FL: CRC Press.
• National Academies of Sciences, Engineering, and Medicine (2019) Reproducibility and Replicability in Science. Washington, DC: National Academies Press.
• Peng, R.D. (2011) ‘Reproducible research in computational science’, Science, 334(6060), pp. 1226–1227.
• Sandve, G.K. et al. (2013) ‘Ten simple rules for reproducible computational research’, PLoS Computational Biology, 9(10), e1003285.
• Wilson, G. et al. (2017) ‘Good enough practices in scientific computing’, PLoS Computational Biology, 13(6), e1005510.
• Wilson, G. et al. (2014) ‘Best practices for scientific computing’, PLoS Biology, 12(1), e1001745.
• Noble, W.S. (2009) ‘A quick guide to organizing computational biology projects’, PLoS Computational Biology, 5(7), e1000424.
• Brinckman, A. et al. (2019) ‘Computing environments for reproducibility: Capturing the “whole tale”’, Future Generation Computer Systems, 94, pp. 854–867.
• Katz, D.S., Niemeyer, K.E. and Smith, A.M. (2016) ‘Software vs. data in the context of citation’, PeerJ Computer Science, 2, e86.
• Wilkinson, M.D. et al. (2016) ‘The FAIR Guiding Principles for scientific data management and stewardship’, Scientific Data, 3, 160018.

Back to top ↑

References

• Brinckman, A. et al. (2019) ‘Computing environments for reproducibility: Capturing the “whole tale”’, Future Generation Computer Systems, 94, pp. 854–867.
• Katz, D.S., Niemeyer, K.E. and Smith, A.M. (2016) ‘Software vs. data in the context of citation’, PeerJ Computer Science, 2, e86.
• National Academies of Sciences, Engineering, and Medicine (2019) Reproducibility and Replicability in Science. Washington, DC: National Academies Press.
• Noble, W.S. (2009) ‘A quick guide to organizing computational biology projects’, PLoS Computational Biology, 5(7), e1000424.
• Peng, R.D. (2011) ‘Reproducible research in computational science’, Science, 334(6060), pp. 1226–1227.
• Sandve, G.K. et al. (2013) ‘Ten simple rules for reproducible computational research’, PLoS Computational Biology, 9(10), e1003285.
• Stodden, V., Leisch, F. and Peng, R.D. (eds) (2014) Implementing Reproducible Research. Boca Raton, FL: CRC Press.
• Wilkinson, M.D. et al. (2016) ‘The FAIR Guiding Principles for scientific data management and stewardship’, Scientific Data, 3, 160018.
• Wilson, G. et al. (2014) ‘Best practices for scientific computing’, PLoS Biology, 12(1), e1001745.
• Wilson, G. et al. (2017) ‘Good enough practices in scientific computing’, PLoS Computational Biology, 13(6), e1005510.

Back to top ↑