12.1 Systematic review methodology
6 min read•august 21, 2024
Systematic reviews are a powerful tool in communication research, providing a comprehensive synthesis of existing literature on specific questions. They employ rigorous methods to identify, evaluate, and summarize relevant studies, reducing bias and increasing reliability compared to traditional literature reviews.
The systematic review process involves defining a clear research question, developing a search strategy, screening studies, extracting data, and assessing quality. Researchers can use various synthesis methods, including narrative approaches and , to integrate findings and draw meaningful conclusions.
Overview of systematic reviews
- Systematic reviews form a cornerstone of evidence-based research in Advanced Communication Research Methods
- Provide a comprehensive, unbiased synthesis of existing literature on a specific research question
- Employ rigorous, transparent methods to identify, evaluate, and summarize relevant studies
Purpose and importance
- Synthesize large bodies of research to inform evidence-based decision making
- Identify gaps in current knowledge and guide future research directions
- Reduce bias and increase reliability compared to traditional literature reviews
- Enhance reproducibility and transparency in research synthesis processes
Key characteristics
- Clearly defined research question and objectives
- Comprehensive search strategy to identify all relevant studies
- Explicit inclusion and for
- Systematic and quality assessment procedures
- Transparent reporting of methods and findings
Types of systematic reviews
Meta-analysis vs narrative synthesis
Top images from around the web for Meta-analysis vs narrative synthesis
Frontiers | Ketosis and migraine: a systematic review of the literature and meta-analysis View original
Is this image relevant?
Frontiers | A systematic review and meta-analysis of the effectiveness and safety of COVID-19 ... View original
Is this image relevant?
Frontiers | Associations of COVID-19 vaccination during pregnancy with adverse neonatal and ... View original
Is this image relevant?
Frontiers | Ketosis and migraine: a systematic review of the literature and meta-analysis View original
Is this image relevant?
Frontiers | A systematic review and meta-analysis of the effectiveness and safety of COVID-19 ... View original
Is this image relevant?
1 of 3
Top images from around the web for Meta-analysis vs narrative synthesis
Frontiers | Ketosis and migraine: a systematic review of the literature and meta-analysis View original
Is this image relevant?
Frontiers | A systematic review and meta-analysis of the effectiveness and safety of COVID-19 ... View original
Is this image relevant?
Frontiers | Associations of COVID-19 vaccination during pregnancy with adverse neonatal and ... View original
Is this image relevant?
Frontiers | Ketosis and migraine: a systematic review of the literature and meta-analysis View original
Is this image relevant?
Frontiers | A systematic review and meta-analysis of the effectiveness and safety of COVID-19 ... View original
Is this image relevant?
1 of 3
- Meta-analysis combines quantitative data from multiple studies statistically
- Calculates pooled effect sizes and confidence intervals
- Allows for assessment of heterogeneity across studies
- summarizes findings qualitatively
- Used when statistical combination is not feasible or appropriate
- Employs thematic analysis or other qualitative techniques
Qualitative vs quantitative reviews
- Qualitative reviews focus on non-numerical data and interpretive analysis
- Synthesize findings from qualitative studies (interviews, focus groups)
- Use methods like meta-ethnography or
- Quantitative reviews analyze numerical data and statistical outcomes
- Include meta-analyses and other statistical synthesis techniques
- Often used for evaluating intervention effectiveness
Steps in systematic review process
Defining research question
- Formulate a clear, focused research question using PICO framework
- Population: Who or what is being studied?
- Intervention: What is the main intervention or exposure?
- Comparison: What is the intervention being compared to?
- Outcome: What are the relevant outcomes or effects?
- Develop a protocol outlining review methods and procedures
Developing search strategy
- Identify relevant databases and information sources
- Create a comprehensive list of search terms and keywords
- Consult with subject matter experts and librarians
- Document search strategy for transparency and reproducibility
Screening and selection criteria
- Establish clear inclusion and exclusion criteria based on research question
- Develop a screening form or checklist for consistent application
- Conduct pilot screening to ensure inter-rater reliability
- Implement a two-stage screening process (title/abstract, then full-text)
Data extraction methods
- Design standardized data extraction forms
- Extract relevant information from included studies
- Study characteristics, participant details, interventions, outcomes
- Use multiple reviewers to ensure accuracy and reduce bias
- Resolve discrepancies through consensus or third-party adjudication
Quality assessment tools
- Select appropriate quality assessment tools based on study design
- Evaluate in individual studies
- Assess overall quality of evidence for each outcome
- Consider using validated tools (Cochrane Risk of Bias, GRADE)
Conducting literature search
Database selection
- Choose databases relevant to research question and field of study
- Include both general (PubMed, Web of Science) and subject-specific databases
- Consider grey literature sources (conference proceedings, dissertations)
- Search trial registries and ongoing studies databases
Search term development
- Identify key concepts from research question
- Use controlled vocabulary (MeSH terms) and free-text keywords
- Incorporate synonyms, alternative spellings, and related terms
- Consider truncation and wildcard symbols to capture variations
Boolean operators
- Combine search terms using Boolean operators (AND, OR, NOT)
- Use parentheses to group related concepts and create complex searches
- Employ proximity operators to specify word order and distance
- Adapt search strategy for different database interfaces
Study selection process
Inclusion vs exclusion criteria
- Define specific criteria based on PICO elements and study characteristics
- Consider factors like publication date, language, and study design
- Ensure criteria are specific enough to identify relevant studies
- Document reasons for exclusion during screening process
Title and abstract screening
- Conduct initial screening based on title and abstract information
- Use two independent reviewers to minimize selection bias
- Err on the side of inclusion when uncertain about relevance
- Calculate inter-rater agreement using kappa statistic
Full-text review
- Retrieve full-text articles for studies passing initial screening
- Apply inclusion/exclusion criteria more rigorously
- Extract detailed information for included studies
- Document reasons for exclusion at this stage
Data extraction techniques
Standardized extraction forms
- Develop forms tailored to research question and study designs
- Include fields for study characteristics, methods, and results
- Use structured formats (dropdown menus, checkboxes) when possible
- Ensure consistency in data extraction across multiple reviewers
Pilot testing procedures
- Test extraction form on a sample of included studies
- Refine form based on pilot results and reviewer feedback
- Provide clear instructions and definitions for each data field
- Train reviewers to ensure consistent interpretation and application
Quality assessment
Risk of bias evaluation
- Assess potential sources of bias in individual studies
- Consider selection bias, performance bias, detection bias, attrition bias
- Use validated tools appropriate for study design (Cochrane RoB 2, ROBINS-I)
- Present risk of bias assessments in summary tables or figures
Critical appraisal tools
- Select tools based on study design and research question
- Use checklists or scales to evaluate methodological quality
- Consider tools like CASP (Critical Appraisal Skills Programme) checklists
- Assess impact of study quality on overall strength of evidence
Data synthesis methods
Narrative synthesis approach
- Organize findings into logical categories or themes
- Use tables and figures to present study characteristics and results
- Explore patterns, similarities, and differences across studies
- Consider using framework synthesis or thematic analysis techniques
Meta-analysis techniques
- Pool quantitative data using appropriate statistical methods
- Calculate effect sizes and confidence intervals
- Assess heterogeneity using measures like I² statistic
- Conduct subgroup analyses and meta-regression when appropriate
Reporting guidelines
PRISMA statement
- Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- 27-item checklist covering all aspects of systematic review reporting
- Includes flow diagram illustrating study selection process
- Enhances transparency and reproducibility of systematic reviews
MOOSE guidelines
- Meta-analysis Of Observational Studies in Epidemiology
- Provides guidance for reporting meta-analyses of observational studies
- Addresses unique challenges in synthesizing non-randomized studies
- Includes recommendations for background, search strategy, and methods
Challenges and limitations
Publication bias
- Tendency for positive results to be published more frequently
- Can lead to overestimation of intervention effects
- Assess using funnel plots and statistical tests (Egger's test)
- Consider searching grey literature to mitigate publication bias
Heterogeneity issues
- Variability in study designs, populations, and outcomes
- Can complicate data synthesis and interpretation of results
- Explore sources of heterogeneity through subgroup analyses
- Use random-effects models in meta-analyses when heterogeneity is high
Software tools for systematic reviews
- Reference management software (, Zotero) for organizing citations
- Screening tools (Covidence, Rayyan) for streamlining study selection
- Data extraction and management platforms (EPPI-Reviewer, DistillerSR)
- Meta-analysis software (, Comprehensive Meta-Analysis) for statistical synthesis
Updating systematic reviews
- Establish a protocol for periodic updates of systematic reviews
- Monitor new publications in the field through alerts and saved searches
- Consider living systematic reviews for rapidly evolving research areas
- Update search strategies and as needed
Ethical considerations
- Ensure transparency in reporting methods and results
- Disclose any potential conflicts of interest
- Respect copyright and data sharing agreements when accessing studies
- Consider ethical implications of findings and recommendations
Applications in communication research
- Synthesize evidence on media effects and audience behavior
- Evaluate effectiveness of communication interventions and campaigns
- Analyze trends in communication technologies and platforms
- Inform policy decisions and best practices in communication strategies
Key Terms to Review (19)
Clinical guidelines: Clinical guidelines are systematically developed statements that assist healthcare professionals and patients in making informed decisions about appropriate healthcare for specific clinical circumstances. They are based on evidence from systematic reviews, expert consensus, and clinical expertise, ensuring that the recommendations are both relevant and applicable to real-world scenarios.
Cochrane Handbook: The Cochrane Handbook is a comprehensive guide that provides systematic reviewers with a framework for conducting high-quality systematic reviews and meta-analyses in healthcare. It details methodologies for planning, conducting, and reporting systematic reviews, making it an essential resource for researchers seeking to evaluate evidence in health-related fields.
Data extraction: Data extraction is the process of retrieving relevant information from various sources, often for the purpose of analysis and synthesis in research. This term is crucial in systematic reviews and meta-analyses as it involves collecting data from multiple studies to ensure comprehensive understanding and accurate results. The quality and accuracy of the extracted data can significantly impact the findings of research, making it a vital step in these methodologies.
Endnote: An endnote is a reference or note placed at the end of a document or section that provides additional information or citations related to the content. In systematic reviews, endnotes play a crucial role in ensuring transparency and traceability of sources, allowing readers to verify and further explore the references cited throughout the text.
Evidence hierarchy: The evidence hierarchy is a system used to rank the strength and reliability of different types of research evidence, with higher levels indicating more robust and trustworthy findings. It helps researchers, practitioners, and policymakers assess the quality of evidence when making decisions or conducting systematic reviews. Understanding this hierarchy is essential for evaluating the credibility of research and guiding effective practice.
Exclusion Criteria: Exclusion criteria are specific factors or conditions that disqualify individuals from participating in a research study. These criteria help researchers narrow down their sample to ensure that the participants align with the study's goals and objectives. By defining who cannot be included, exclusion criteria help minimize bias and ensure that the findings are more reliable and applicable to the intended population.
Grade framework: The grade framework is a systematic approach used to evaluate the quality of evidence and the strength of recommendations in research studies, particularly in health and social sciences. It helps in making transparent and consistent decisions about the reliability of research findings by categorizing evidence into different levels, guiding practitioners and policymakers in interpreting the research effectively.
Inclusion criteria: Inclusion criteria are the specific characteristics or requirements that participants must meet to be eligible for inclusion in a study. These criteria ensure that the sample population is appropriate for the research question and help to maintain the validity and reliability of the findings by defining who can participate.
Literature search: A literature search is a systematic process of identifying, locating, and evaluating existing research and publications relevant to a specific topic or research question. This process is crucial for synthesizing information and understanding the current state of knowledge in a field, forming the basis for systematic reviews and meta-analyses.
Meta-analysis: Meta-analysis is a statistical technique that combines the results of multiple studies to identify overall trends, patterns, and relationships across the research. This method enhances the power of statistical analysis by pooling data, allowing for more robust conclusions than individual studies alone. It connects deeply with hypothesis testing, systematic reviews, effect size calculations, heterogeneity assessments, publication bias considerations, and the quality assessment of studies to create a comprehensive understanding of a particular research question.
Narrative synthesis: Narrative synthesis is a method of integrating findings from multiple studies, particularly in systematic reviews, by summarizing and interpreting the results in a cohesive narrative format. This approach helps to convey complex information and highlights patterns or themes across different research works, making it easier to understand the overall evidence in a particular area of study.
PRISMA: PRISMA stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses. It is a set of guidelines designed to improve the transparency and quality of reporting in systematic reviews and meta-analyses, ensuring that researchers provide all necessary information to evaluate the validity and reliability of their findings. By following PRISMA, researchers can help ensure that systematic reviews are comprehensive and reproducible, which is essential for making informed decisions based on evidence.
RevMan: RevMan, short for Review Manager, is a software tool developed by Cochrane for preparing and maintaining systematic reviews and meta-analyses. It provides a user-friendly interface for managing references, analyzing data, and generating reports, making it an essential resource in the field of evidence-based healthcare research. This tool streamlines the systematic review methodology process and ensures that reporting standards for meta-analyses are met effectively.
Risk of bias: Risk of bias refers to the potential for systematic errors or deviations from the truth in research findings, which can impact the validity and reliability of the conclusions drawn from studies. This concept is crucial when assessing the quality of evidence in systematic reviews and meta-analyses, as it helps identify factors that may distort the results due to flawed study design, data collection, or reporting practices.
Scoping Review: A scoping review is a type of literature review that aims to map the existing literature on a particular topic, identifying key concepts, gaps, and evidence available. Unlike systematic reviews, which focus on answering specific research questions through rigorous methods, scoping reviews provide a broader overview to inform future research or policy directions, making them especially useful in emerging fields or complex areas of study.
Statistical pooling: Statistical pooling is a method used to combine data from multiple studies or experiments to achieve a more comprehensive understanding of a particular phenomenon. This approach helps to increase the overall sample size, improve the precision of estimates, and enhance the generalizability of the findings across different contexts or populations.
Study selection: Study selection refers to the process of identifying, screening, and choosing relevant studies to be included in a systematic review or meta-analysis. This critical step ensures that only high-quality and pertinent research contributes to the evidence synthesis, allowing for valid conclusions and recommendations. It involves establishing clear inclusion and exclusion criteria based on research questions and predefined methodological standards.
Subgroup analysis: Subgroup analysis is a method used in research to assess how different subgroups within a study population respond to an intervention or treatment. This type of analysis helps identify variations in outcomes based on specific characteristics, such as age, gender, or other demographic factors, enabling researchers to understand the effects of an intervention more deeply and tailor findings to specific groups.
Thematic synthesis: Thematic synthesis is a qualitative research method that involves integrating findings from multiple studies to identify common themes and patterns across the literature. This approach helps to create a comprehensive understanding of a particular topic by analyzing and synthesizing qualitative data from various sources, thereby enabling researchers to derive insights that may not be apparent from individual studies alone.