5. Quality Appraisal Form Planning

Risk of Bias Tools 45 min 24+ Standard Templates

📋 Quality Assessment Tools Dashboard

Design custom tools or select from 24+ standard templates (JBI, Cochrane ROBINS-I, ROB 2, Newcastle-Ottawa, etc.) to assess methodological quality and risk of bias of included studies.

Tool Name	Domains	Created By	Actions
ROBINS-I (Risk Of Bias In Non-randomized Studies - of Interventions) A tool for assessing risk of bias in non-randomized studies comparing health effects of two or more interventions.	7	Admin
Cochrane RoB 2 (RCTs) – 2023 Revised tool for randomized controlled trials covering randomization, deviations, missing data, outcome measurement, and selective reporting.	5	System

1 Click "Add New Tool" → choose starting point: Select a standard template or build from scratch.

2 Provide Tool Name (e.g., "JBI Critical Appraisal for RCTs") and optional Description.

3 Define Domain Rating Scale Options: e.g., Low Risk, Some Concerns, High Risk, No Info. Attach icon suggestions for visual clarity.

4 Define Overall Judgment Scale: e.g., Grade A (High Certainty), Grade B (Moderate Certainty), Grade C (Low Certainty).

5 Add Assessment Domains with custom text, category, appraisal level, guidance, and optional sub-domains.

Example: Editing ROBINS-I Tool

ROBINS-I (Risk Of Bias In Non-randomized Studies - of Interventions)

Domain 1: D1: Bias due to confounding – Consider whether the most important confounding domains were controlled for in design or analysis.

Domain 2: D2: Bias in selection of participants into the study

Domain 3: D3: Bias in classification of interventions

Domain 4: D4: Bias due to deviations from intended interventions

Domain 5: D5: Bias due to missing data

Domain 6: D6: Bias in measurement of outcomes

Domain 7: D7: Bias in selection of the reported result

Low Some concerns High

Overall Judgment Scale: Grade A: High Certainty | Grade B: Moderate Certainty | Grade C: Low Certainty

Export options: Each QA tool can be exported as DOCX, PDF, or CSV for external documentation or protocol appendices.

Quality Appraisal Form Builder interface

Figure 1: Creating a custom QA tool – domain structure and rating scales.

6. Extraction Form Planning

Data Extraction 1‑to‑Many Repeater Logic 60 min

📊 Data Extraction Forms Dashboard

Design structured forms to ensure consistent data collection across all included studies. For qualitative meta‑aggregation, use the Repeater Field to capture multiple findings per study.

Form Name	Description	Fields	Last Updated	Actions
Meta-Analysis of Effectiveness - Count Data (e.g., RCTs)	For extracting data from studies comparing two groups to calculate effect size.	11	4/6/2026
Qualitative Synthesis - Meta‑Aggregation with Repeater	Extract themes, categories, and illustrative quotes using 1‑to‑many repeater rows.	8 (+ repeater group)	4/10/2026

1 Click "Create New Form" → enter Form Name and Description.

2 Optionally start from a template (blank or pre‑defined structure).

3 Add fields: Field Label, Field Name (machine‑friendly), Field Type (Text, Textarea, Number, Dropdown, Date, Checkbox).

4 Toggle "Repeater?" to YES for fields that require 1‑to‑many entries (e.g., multiple qualitative findings per study).

5 Save form – it becomes available for reviewers during the Conducting phase.

⭐ Major Feature: 1‑to‑Many Repeater Logic for Qualitative Evidence
In qualitative systematic reviews (JBI meta‑aggregation), a single study may contribute multiple themes, categories, or findings. The Repeater Field allows reviewers to click "Add Another Row" to extract unlimited findings from one source. When you design an extraction form, set the field type as Repeater Group – inside it, include fields like "Theme Name", "Illustrative Quote", "Participants' Voice", and "Level of Credibility". The system will group these during synthesis.

Edit Extraction Form Example

Form: Meta-Analysis of Effectiveness - Count Data (e.g., RCTs)

Order 1 Study Author & Year (Text) – Repeater? No

Order 2 Study Design (Text) – Repeater? No

Order 3 Intervention Group Description (Textarea) – Repeater? No

Order 4 Intervention Group Sample Size N1 (Number)

Order 5 Intervention Group Events n1 (Number)

Order 6 Control Group Sample Size N2 (Number)

Order 7 Control Group Events n2 (Number)

Order 8 Control/Comparator Group Description (Text)

Order 9 Outcome Measure Description (Text)

Order 10 Follow-up Period (Text)

Order 11 Notes (Textarea)

Creating a Repeater Group for Qualitative Synthesis

REPEATER GROUP ENABLED

Field Label: Qualitative Findings (Repeater Group)
Inside Repeater – fields that can be repeated multiple times per study:

Theme/Category Name (Text) – e.g., "Barriers to adherence"
Illustrative Quote (Textarea) – verbatim quote from participants
Credibility Rating (Dropdown) – Unequivocal / Equivocal / Unsupported
Country / Setting (Text)

During extraction, reviewers click "Add Another Row" to insert a new set of fields for each unique finding.

Repeater field configuration in form builder

Figure 2: Designing a data extraction form with 1‑to‑many repeater rows – critical for qualitative evidence synthesis.

Form Preview & Validation

Preview: Meta-Analysis of Effectiveness - Count Data

📌 Two-Phase Extraction (Study Characteristics + Outcome Data)
RevProHub now supports separation of static study characteristics (author, year, design) from repeatable outcome data (effect sizes, events, qualitative findings). This ensures cleaner datasets for meta‑analysis or thematic synthesis.

🧪 Hands-On Activity: Tool Planning (QA & Extraction) – 35 min

Create a QA tool: Navigate to Planning → Quality Appraisal Form Planning. Click "Add New Tool". Select a template (e.g., "Cochrane RoB 2 for RCTs"). Name it "My RoB2 Training Tool". Add domain rating scale (Low / Some concerns / High) and overall judgment (High certainty / Moderate / Low). Save.
Add a custom domain: Edit the tool → Add a new domain: "D6: Blinding of outcome assessment" with guidance text. Save changes.
Create an Extraction Form with Repeater: Go to Extraction Form Planning → Create New Form. Name: "Qualitative Synthesis - Themes". Add fields: "Study ID" (Text), "Country" (Text), then add a Repeater Group. Inside the group add: "Theme Name" (Text), "Illustrative Quote" (Textarea), "Credibility" (Dropdown with options: High/Moderate/Low). Set Repeater = YES for the group.
Preview the form: Click "View Form" to see how reviewers will see the "Add Another Row" button.
Export the QA tool: From QA Tools dashboard, click export (PDF) – observe how domains and rating scales are formatted.

Debrief Questions:

Why is the Repeater logic essential for qualitative evidence synthesis compared to traditional flat extraction forms?
How does pre‑defining domain rating scales improve consistency across multiple reviewers?
What is the advantage of separating Study Characteristics from Outcome Data (two‑phase extraction)?
How do these planning tools (QA + extraction) link to the later Conducting phase modules?

Integration with Protocol & Conducting: QA and extraction forms created in the Planning phase are automatically available during the Conducting phase (Quality Appraisal and Data Extraction stages). Changes to forms after extraction begins are restricted to maintain data integrity – so plan carefully.

Two-Phase Extraction Architecture:
Phase 1 – Study Characteristics (static, extracted once per study): Author, Year, Country, Design, Funding source.
Phase 2 – Outcome Data / Qualitative Findings (repeatable): Effect sizes, confidence intervals, themes, quotes. This separation enables efficient subgroup analysis and meta‑aggregation.

Next session transition: After completing tool planning (QA + extraction), the project moves to the Conducting Phase where these instruments are deployed for risk of bias assessment and data extraction.

Next session: Importing Studies, Deduplication, and Preparing for Screening (Conducting Phase).

Extraction and Quality Appraisal Planning in RevProHub