Evidence synthesis: protocols, bias and disagreement

Choose the review type

Distinguish a representative field review, a scoping review and a systematic review addressing a bounded question. Meta-analysis is a statistical synthesis requiring compatible evidence. Deliver a search protocol, extraction table and argument organized around disagreement.

Cochrane provides intervention-review methods; PRISMA 2020 supports transparent reporting. RoB 2 evaluates a particular randomized-trial result, rather than acting as a universal quality score. The exercise below is platform-designed and fictional.

Specify a protocol before screening

Consider whether structured peer feedback improves university students' academic writing. Define population, intervention, comparator, outcomes and eligible designs. Decide how to handle satisfaction-only measures, multiple reports of one study and abstract-only records.

1. Build concept blocks and database-specific search syntax. Save exact queries, fields, dates and exports.
2. Test retrieval against known relevant studies; revise missing concepts without tuning searches toward desired conclusions.
3. Log deduplication, title/abstract decisions, full-text exclusions and disagreements. Follow the selected method's independent screening requirements for a formal review; disclose a solo exercise's limitations.
4. Version the protocol, explaining any post-hoc changes and their implications.

Extract comparable evidence

Give each study a stable ID linking its reports. Record design, sample, unit, measurement, follow-up, effect and uncertainty, missingness, bias judgement and full-text location. Several outcomes from the same participants are not independent studies.

Random assignment and voluntary writing-centre attendance may both accompany higher scores, but selection complicates the latter. Immediate essays and independent writing three months later also measure different outcomes. Assess comparability before combining them.

Synthesize and review

For each important result, explain potential material bias with supporting evidence. Journal names, citation counts and counts of significant findings do not establish evidential quality. Statistical synthesis needs a separate argument about estimands, measures, dependence and heterogeneity; this workbook does not teach a complete meta-analysis.

• Submit rerunnable queries, a dated protocol and screening records whose counts reconcile with the PRISMA flow diagram.
• Trace a sampled conclusion to a specific result, source location and bias assessment.
• Compare studies within each theme: agreement, disagreement and limitations. Avoid one abstract summary per paragraph.
• Explain non-comparability and disclose procedures you did not perform.

Read Cochrane on scope, RoB 2 and PRISMA 2020, then return to the resource library.

Advanced practice and review evidence

Pilot extraction on three full texts: population, intervention/phenomenon, comparator, measure, timing and design. Predefine treatment of unavailable texts, overlapping samples and different constructs. Examine measurement and bias behind conflicting findings rather than counting significant results. PRISMA checks reporting completeness; it does not replace risk-of-bias assessment.

Deliver search strings/dates, screening reasons, passage locations and bias rationale. If independent duplicate screening is unavailable, report it; AI is not a second independent researcher. Transfer: add another report from the same cohort and decide how to link reports without double-counting participants.

Open practice, AI review and work export.