Bayesian workflow: generative models, diagnostics and revision

Audience and output

For readers with probability, regression, Bayesian inference and R or Python. Produce a model specification, runnable analysis, predictive checks and revision log. Use simulated or licensed public data for this platform-designed exercise.

Gelman, Vehtari and colleagues frame model building, computation and checking as an iterative workflow. McElreath supplies lecture and code examples. This workbook practises that process rather than replacing a probabilistic programming course.

Describe generation and dependence

Imagine task completion times from several laboratories, with unequal samples and repeated observations per participant. Define observation and research units. Repeated rows are not independent participants.

Specify how positive, possibly skewed times, participant differences and laboratory variation arise. Justify a candidate likelihood, such as a lognormal model, against the mechanism. Causal interpretations still require a separate design argument.

Predict before interpreting

State priors in meaningful units. Simulate prior predictive observations and check whether they exclude plausible variation or frequently generate impossible times. Record revisions and include influential prior choices in sensitivity analysis.

Construct a small laboratory with an extreme observed mean. Compare complete, separate and partial pooling estimates and uncertainty. Partial pooling depends on structural assumptions, including appropriate exchangeability; it is not universally correct.

Separate computational from model failure

Examine chain mixing, R-hat, effective sample sizes and divergences as relevant to the sampler. Resolve and document abnormalities before interpreting precise posterior differences.

Then test whether posterior predictions reproduce tails, between-group variation and within-participant dependence. An overall mean plot is insufficient. Investigate measurement, likelihood and omitted structure rather than deleting inconvenient observations.

Validate against the intended use

For new-laboratory generalization, leave out laboratories rather than randomly holding out their records. For future prediction, prevent future information entering preprocessing. Predictive comparison alone does not validate an explanation.

Vary defensible priors, likelihoods and missingness assumptions. Label result-informed revisions exploratory; confirmation requires appropriately held-out or new evidence.

Review criteria

• Specify units, parameters, priors, dependence and the scientific target.
• Supply prior predictive, sampling diagnostic, posterior predictive and sensitivity plots with an explanation of each failure check.
• Log each observed failure, revision, rationale and effect on conclusions.
• Preserve unexplained discrepancies. Narrow posterior intervals do not establish credible assumptions.

Open the modelling resources.

Advanced practice and review evidence

Simulate prior predictions and inspect impossible or implausible values, then assess whether posterior predictions reproduce distributional shape and group differences. Separate computation from adequacy: good R-hat or effective sample size does not establish a theoretical interpretation. When comparing complete, separate and partial pooling, explain exchangeability and where small groups obtain information.

Deliver prior rationale, predictive plots, computation diagnostics and a revision log linking changes to conclusions. Retain the exploratory nature of data-informed revisions. Transfer: data from a new institution show systematic bias; inspect measurement and the generating process before retaining the hierarchy.

Open practice, AI review and work export.