🧠 Concept

1. The Distinction That Matters

Novice reviewers frequently confuse three distinct things:

A systematic review cannot be conducted on a topic. It requires an answerable question — one that specifies enough detail that a search strategy, eligibility criteria, and synthesis plan can be derived.

2. What Makes a Question "Answerable"

A well-formulated review question typically satisfies the following criteria:

• Specific — the population, intervention, and outcome are clearly defined.
• Focused — narrow enough that a manageable number of studies will be found.
• Feasible — there is likely to be sufficient evidence (a scoping search confirms this).
• Relevant — it addresses a real gap that matters to patients, practitioners, or policymakers.
• Timely — the answer is still valuable at the time the review will be published.
• Ethical — it does not require evidence that could not have been ethically produced.

3. Why Question Formulation Is the Most Important Stage

Every downstream decision depends on the question:

• The question determines the search strategy — the concepts in the question become the search concepts.
• It determines the eligibility criteria — a specific question yields specific inclusion/exclusion rules.
• It determines the data extraction fields — you extract data relevant to the question.
• It determines the synthesis method — an effectiveness question may involve meta-analysis; a qualitative question may involve meta-aggregation.
• It determines the reporting structure — PRISMA requires the objectives to be clearly stated and matched by the methods.

A weak question cannot be fixed by strong methods — every subsequent stage inherits its flaws.

4. Common Pitfalls in Question Formulation

5. A Structured Process for Question Formulation

A practical seven-step process:

1. Start with a real-world problem. Where is there genuine uncertainty? What decision would the review inform? Who cares about the answer?
2. Conduct a preliminary literature scan. Rapidly review the existing literature — especially existing systematic reviews, guidelines, and recent primary studies — to identify the specific gap.
3. Check PROSPERO and Cochrane Library. Has a similar review been registered or published? If yes, is an update warranted? If no, proceed.
4. Draft the question using a framework (PICO, PECO, PICo, SPIDER — see Chapter 3.2). Frameworks force specificity.
5. Test the draft against a scoping search. Run an initial search on one or two databases to check whether the question returns a manageable number of studies. Too many → narrow; too few → broaden.
6. Engage stakeholders. Clinicians, patients, policymakers, and program managers can validate that the question addresses their needs. Their input often refines outcomes or population definitions.
7. Refine iteratively. The first draft is almost never final. Iterate until the question is specific, focused, feasible, relevant, timely, and ethical.

6. Stakeholder Engagement

Meaningful stakeholder engagement improves the review by:

• Ensuring the question addresses priorities that matter to the people affected.
• Refining outcome selection — especially patient-centered outcomes that may not be captured in primary studies.
• Facilitating implementation and dissemination — stakeholders who shaped the question are more likely to use the answer.
• Increasing the review's credibility with funders, journals, and policy audiences.

Stakeholders to consider: patients or service users, clinicians, program managers, policymakers, methodologists, and representatives of advocacy organizations.

7. From Question to PICO — A First Look

The most commonly used framework for intervention questions is PICO:

• P — Population: who is the question about?
• I — Intervention: what is being done?
• C — Comparator: what is it being compared against?
• O — Outcome: what change is being measured?

Chapter 3.2 covers PICO and other frameworks in detail.

🧠 Concept

1. Why Frameworks?

Frameworks provide structure. Without them, questions drift — the population changes, outcomes multiply, the intervention becomes ambiguous. Frameworks force the reviewer to specify each element, which in turn makes the question answerable. Frameworks also produce the concepts that will become search terms, eligibility criteria, and data extraction fields.

2. PICO — The Most Common Framework

PICO is designed for intervention questions:

Optional extensions:

• T — Time: over what period?
• S — Study design: which designs are eligible?
• Setting: where does the review apply (LMIC, hospital, community)?

3. PECO — For Exposure / Etiology Questions

When the question concerns an exposure (not an intervention you control), PECO replaces I with E:

• P — Population
• E — Exposure
• C — Comparator (unexposed or different exposure level)
• O — Outcome

Example: "In adults (P), does chronic exposure to air pollution (E), compared with low exposure (C), increase the risk of asthma hospitalization (O)?"

PECO is used for risk factor reviews, environmental exposure reviews, and observational etiology questions.

4. PICo — For Qualitative Questions

Qualitative questions rarely have a "comparator" or "outcome" in the traditional sense. PICo adapts the framework:

• P — Population (or phenomenon context)
• I — Interest (phenomenon of interest)
• Co — Context

Example: "What are the experiences (I) of pregnant women (P) with antenatal care in rural Ethiopia (Co)?"

5. SPIDER — For Qualitative and Mixed-Methods Questions

SPIDER is designed for qualitative and mixed-methods reviews, particularly when research design is a key filter:

• S — Sample
• PI — Phenomenon of Interest
• D — Design
• E — Evaluation
• R — Research type

Example: "In nurses working in intensive care (S), what is the impact of shift work (PI) on well-being (E), as reported in phenomenological and ethnographic studies (D, R)?"

6. SPICE — For Social Science and Policy Questions

SPICE is commonly used in social science, health policy, and library science reviews:

• S — Setting
• P — Perspective
• I — Intervention
• C — Comparison
• E — Evaluation

7. PIRT — For Diagnostic and Prognostic Questions

PIRT is used for test accuracy questions:

• P — Population
• I — Index test
• R — Reference standard
• T — Target condition

Example: "In adults with suspected pulmonary embolism (P), what is the accuracy of CT pulmonary angiography (I) compared with V/Q scanning (R) for the detection of pulmonary embolism (T)?"

8. Framework Selection Guide

9. Limitations and Adaptations

Frameworks are tools, not dogmas. Adaptations are common:

• Some reviews add a fifth element (e.g., PICOS with Study design, or PICOT with Time).
• Complex interventions may not fit a single I — the reviewer may define an intervention "family" with a shared mechanism.
• Prevalence reviews may drop the C — the question is simply "what is the prevalence of X in Y?"
• Umbrella reviews may add an "S" for "Synthesis of reviews."

10. From Framework to Search Strategy

Each PICO element becomes a search concept:

("Hypertension"[Mesh] OR hypertens*[tiab] OR "high blood pressure"[tiab])
AND
("Community Health Workers"[Mesh] OR "community health worker*"[tiab] OR "lay health worker*"[tiab] OR "village health worker*"[tiab])

🧠 Concept

1. What Are Eligibility Criteria?

Eligibility criteria are the rules that determine whether a study is included in or excluded from the review. They operationalize the review question: the PICO framework defines what the review is about; eligibility criteria define which studies count as evidence.

Well-defined eligibility criteria:

• Reduce the risk of selection bias in the review.
• Enable consistent decisions by independent reviewers.
• Make the review reproducible.
• Align the study pool with the review question.

2. Structure of Eligibility Criteria

Eligibility criteria typically follow the PICO(S) framework:

3. Inclusion vs Exclusion

Each criterion can be expressed as either an inclusion rule (study is eligible if it meets X) or an exclusion rule (study is not eligible if it fails Y). Both forms are valid; the choice depends on clarity:

• Inclusion: "Randomized controlled trials of adults ≥18 with type 2 diabetes..."
• Exclusion: "Studies in children (<18) will be excluded..."

Reviewers should avoid double-negative or ambiguous criteria. "Studies that are not unpublished" is unclear — write "Only published studies."

4. Writing Unambiguous Criteria

Every criterion should be:

• Specific — no ambiguous terms like "older adults," "moderate dose," or "long-term follow-up."
• Operationalized — define exact thresholds ("≥65 years," "≥8 weeks," "≥6-month follow-up").
• Justified — the criterion should be linked to the review question, not arbitrary.
• Consistently applicable — two reviewers, using the criteria, should arrive at the same decision.

5. Common Edge Cases — Decide in Advance

Real-world reviews encounter predictable edge cases. Deciding how to handle them in the protocol prevents inconsistent decisions later:

6. Criteria That Introduce Bias

Some criteria seem reasonable but introduce systematic bias:

• Language restrictions — excluding non-English studies often excludes high-quality research and inflates effect estimates (language bias).
• Publication status restrictions — excluding unpublished studies amplifies publication bias.
• Date restrictions without justification — may exclude relevant older studies, especially for stable interventions.
• Sample size minimums — excluding small studies is generally not appropriate; small studies contribute to pooled estimates and to assessment of publication bias.
• Only "high-quality" studies — impossible to judge quality before appraisal; the criterion is circular.
• Restricting to studies from one country — may reduce applicability without a methodological reason.

Where such restrictions are used (e.g., in rapid reviews), they must be transparently documented and justified, and their impact assessed via sensitivity analysis.

7. Aligning Criteria with the Protocol

The eligibility criteria must appear:

• In the protocol, registered before the search begins.
• In the PRISMA-P checklist (item 8).
• In the final PRISMA 2020 reporting (item 5).
• In the screening interface of RevProHub (or equivalent platform).

Any subsequent change to eligibility criteria after screening begins is a protocol amendment and must be documented and reported.

8. Pilot Testing Eligibility Criteria

Before starting formal screening:

1. Sample 20–30 records from the pool.
2. Have two reviewers independently apply the criteria.
3. Compare decisions and discuss disagreements.
4. Refine ambiguous criteria as needed.
5. Repeat until agreement is high (typically ≥80–90% before formal screening).

This pilot process is often called calibration and is essential to prevent divergent decisions during the main screening phase.

🧠 Concept

1. What Is a Scoping Search?

A scoping search is a preliminary, exploratory search conducted before the systematic review is formally launched. Its purpose is not to identify all evidence — that is the job of the systematic search — but to:

• Gauge the approximate size of the evidence base.
• Identify existing systematic reviews on the topic.
• Test the viability of the draft question and eligibility criteria.
• Discover the terminology used by researchers in this field.
• Spot gaps, overlaps, or unexpected populations/interventions.
• Avoid wasting resources on a question that has already been answered.

Scoping searches are also called preliminary searches, exploratory searches, or sensitivity checks.

2. Scoping Search vs Systematic Search

Scoping searches are typically not reported in the manuscript — but their influence on question refinement should be documented in the protocol's background section.

3. How to Conduct a Scoping Search

A practical workflow:

1. Start with your draft PICO/PECO/PICo. Translate the population and intervention/exposure into 2–4 key terms.
2. Run a simple search in one database. Use PubMed, Scopus, or Cochrane Library — whichever is most accessible. Avoid complex syntax; just OR synonyms within each concept and AND across concepts.
3. Skim the results. Look at the first 2–3 pages of titles. Are they relevant? Are there obvious themes?
4. Check for existing reviews. Search Cochrane Library, PROSPERO, and Epistemonikos specifically for systematic reviews on the same topic.
5. Estimate the volume. How many records does the search return in total?
6. Assess language and geography. Are the studies mostly from HICs or LMICs? Which languages dominate?
7. Note terminology. What MeSH terms and synonyms are commonly used? These will inform the formal search strategy.
8. Refine the question if needed. If the search returns too many records, narrow the population or intervention. If it returns too few, broaden.

4. Interpreting Scoping Results

5. Checking for Existing Reviews

Before finalizing the question, always check whether a similar review already exists. The main sources:

If an existing review is found:

• Is it recent (within the last 3–5 years)?
• Does it match your population, intervention, and outcomes?
• Is its methodology rigorous (e.g., AMSTAR 2 rating)?
• Are you planning a meaningful update or extension?

If the existing review is high quality, recent, and directly applicable, your planned review may not be needed — or it may need to be repositioned as a distinct question.

6. Documenting the Scoping Search

While scoping searches are not formally reported, documentation helps the team and is useful for the protocol's "rationale" section:

• Databases searched and date.
• Approximate number of records returned.
• Number of existing systematic reviews found (with citations).
• Key observations about populations, interventions, or outcomes.
• Any adjustments made to the question as a result.

This internal record becomes part of the review's audit trail.

7. From Scoping to Protocol

Once the scoping search confirms feasibility:

1. Finalize the PICO/PECO/PICo.
2. Lock the eligibility criteria.
3. Draft the search concepts for the formal search.
4. Register the protocol (PROSPERO or equivalent).
5. Begin the systematic search.

("mHealth" OR "mobile health" OR "smartphone") AND diabetes AND Africa