Project Consistency Framework

Keeping code, config, and manuscripts aligned

This framework ensures that numerical values, method descriptions, and parameter settings remain synchronized across your entire project.

The Problem

In research projects, inconsistencies creep in:

  • Paper says “1000 replications” but code uses 500
  • Config defines alpha = 0.05 but manuscript says 0.10
  • Methods section describes “Latin hypercube sampling” but code uses Sobol sequences

These inconsistencies undermine reproducibility and can delay publication.

The Solution

A registry-based system that:

  1. Tracks claims - Central registry of what the paper states
  2. Validates alignment - Scripts verify code matches claims
  3. Catches drift - CI checks prevent merging inconsistent changes

Framework Components

Component Purpose
Config Values Centralized parameter management
Method Claims Registry of methodological statements
Data Provenance Figure/table source tracing
Validation Running consistency checks

Quick Start

1. Set Up Centralized Config

Create config/globals.yml:

project:
  name: "My Analysis"
  seed: 2024

simulation:
  n_reps_low: 500
  n_reps_high: 10000

analysis:
  alpha: 0.05
  power_target: 0.80

2. Create Consistency Registry

Create config/consistency_registry.yml:

claims:
  - id: sim_reps
    category: simulation
    manuscript_text: "10,000 Monte Carlo replications"
    config_key: simulation.n_reps_high
    expected_value: 10000
    locations:
      - manuscript/paper.qmd:Methods

3. Reference Config in Manuscript

source("R/globals_loader.R")
cfg <- load_globals()

In your manuscript:

All simulations used `r cfg$simulation$n_reps_high` replications.

4. Validate Before Submission

make validate-consistency

Directory Structure

project/
├── config/
│   ├── globals.yml              # Centralized parameters
│   └── consistency_registry.yml # Claim tracking
├── R/
│   └── globals_loader.R         # Config loading utilities
├── scripts/
│   └── validate_consistency.R   # Validation script
├── manuscript/
│   └── paper.qmd
└── DEFAULTS.md                  # Parameter documentation

Integration Points

With Targets

# _targets.R
library(targets)
source("R/globals_loader.R")

cfg <- load_globals()

list(
  tar_target(sim_results,
    run_simulation(n_reps = cfg$simulation$n_reps_high))
)

With CI/CD

# .github/workflows/validate.yml
- name: Check Consistency
  run: Rscript scripts/validate_consistency.R

With Pre-Submission

# Makefile
submit: validate-consistency
    @echo "Ready for submission"
    quarto render manuscript/paper.qmd

Best Practices

TipSingle Source of Truth

Never hardcode values. Always reference globals.yml via the loader.

TipDocument Defaults

Maintain DEFAULTS.md explaining why each parameter has its value.

TipValidate Often

Run consistency checks after any config or manuscript changes.

Getting Started

  1. Copy the template files to your project
  2. Customize globals.yml for your parameters
  3. Add claims to the registry as you write
  4. Set up validation in your Makefile