Causal design: estimands, DAGs and sample justification

Audience and output

For readers comfortable with conditional probability and regression. Produce a target-trial table, a causal diagram with assumptions, an analysis plan and design simulations. This platform exercise uses a fictional example.

Use Hernán and Robins's What If for causal methods, McElreath's controls lecture to examine adjustment choices, and Lakens's chapter for sample justification.

Define the effect before the estimator

Does a research training programme improve research ability? Participants' better outcomes may also reflect pre-existing ability and motivation.

Specify eligibility, assignment strategies, comparator, a common time zero, follow-up, outcome measurement, analysis and estimand. Distinguish assignment effects from effects of sustained participation. A confidence questionnaire alone does not establish independent research ability. For observational data, map each available field to the ideal trial and identify missing elements.

Make assumptions visible

Let T denote participation, Y the outcome and M pre-training motivation. A candidate DAG contains M→T, M→Y and T→Y. Adjustment for adequately measured M may block this backdoor path, conditional on the graph's assumptions. A correlation matrix cannot establish those assumptions.

Post-training effort may be a mediator, so adjusting for it changes the target. Selecting only recommended students can also induce bias if recommendation depends on both training and unmeasured ability. Document every covariate's timing and rationale.

Explain consistency, exchangeability and positivity in this setting. Check ambiguous treatment definitions, absent comparators and missing confounders. If identification is unsupported, narrow the conclusion.

Test the design and justify its information

Simulate known effects; compare no adjustment, graph-informed adjustment and inappropriate mediator adjustment. Add unmeasured confounding, measurement error or extreme propensity scores. Simulation establishes behaviour under those settings, not that real-world assumptions hold.

Before examining main outcomes, specify the smallest effect of interest and desired power or precision. Account for clustering, attrition and resources. Plot sensitivity to assumptions. A budget limitation should lead to an honest detectable-effect range, not a convenient target chosen for significance.

Review criteria

• Align estimand, measurement, time zero and parameter interpretation.
• Justify each covariate and the risks of its inclusion or omission.
• Submit simulations and separate primary from sensitivity analyses.
• Name the most consequential unverified assumption; significance does not establish identification.

Open the causal resources and McElreath lectures.

Advanced practice and review evidence

Define the estimand, population, treatment, comparator, time zero and outcome for a teaching intervention. Diagram prior ability, study time and outcomes, distinguishing confounding, mediation and colliders. Do not automatically adjust for intervention-affected study time when estimating a total effect; better regression fit does not validate adjustment.

Use explicitly simulated data to compare no adjustment, defensible adjustment and mistaken adjustment, explaining structural assumptions. Deliver identification assumptions, analysis target, information justification and sensitivity checks. Transfer: randomization is withdrawn; bound the claim and state additional assumptions required.

Open practice, AI review and work export.