Systematic Reviews & Evidence Synthesis Methods

This key stage in a research synthesis includes...

• Deduplication
• Screening, critical appraisal and study selection

• Coding and data extraction

Any time you search in multiple databases, there will be duplicate results. There are many different approaches and tools for removing duplicates (de-depulication), and they all have their pros and cons. No matter which approach you choose, keep track of result totals before and after deduplication in your PRISMA diagram!

• Zotero (recommended) - Zotero is a citation manager that is free and open source. It's deduplication tool is not perfect, but it allows you to make decisions and will not remove anything it considers a duplicate unless you tell it to.
  • For more information, check out this video on Identifying and Merging Duplicates in Zotero.
• EndNote - EndNote is a Web of Science product and has both a free basic and paid version. Deduplication is about the same as with Zotero, but the free version of EndNote is a limited product, over all, compared to the paid version.
  • For more information, check out this video on Eliminating Citations.
• Mendeley - This is another free citation manager that is a product of Elsevier. Mendeley auto-deduplicates, which is not recommended for systematic reviews.
• Rayyan - Rayyan is a free and open source screening tool. It has deduplication abilities, but they are not as user friendly as the others. Because Rayyan is browser-based, it tends to be somewhat less reliable when it comes to deduplication. (Recommended as secondary de-duplication)
  • For more information, check out this Rayyan Tutorial video.
• Excel - This is the option requiring the most work by you and many researchers like it because they control the full process. (Recommended as secondary, "by-hand" deduplication)

Once results have been deduplicated, you will begin the screening process. This is where you'll apply your eligibility criteria to each result. While strict systematic review methodology requires all eligibility criteria to be defined a priori (in advance), some research synthesis approaches allow for ad hoc (as you go) criteria based on the researchers' increasing familiarity with the available research. Just be sure to document if your approach shifts!

Screening often happens in two steps.

1. Title and abstract level screen to exclude studies that are clearly outside the study parameters.
2. Full text screening, ideally done by multiple, independent screeners. Some do the coding and data extraction during this final screening stage.

Some reviews, particularly in medicine, require a critical appraisal or a risk of bias assessment for each study that makes it through the screening process. For any review, it's important to consider the impact that poorly designed studies could have on your findings and to rule out inaccurate or biased work.

Key resources:

• Critical Appraisal Tools from Joanna Briggs Institute
• Rayyan: a free, web-based SR screening software
• Risk of Bias 2: a screening tool from Cochrane

Researchers typically develop a coding schema to outline the data points that will be collected and synthesized from each study included in the review. This often includes things like participant demographics, sample sizes, interventions, methods, outcomes and more. Most often, data is collected in some type of form, table or spreadsheet.

Key resources:

• Collecting data from Cochrane Training
• Data coding & data extraction from Collaboration for Environmental Evidence
• A framework for developing a coding scheme for meta-analysis from Western Journal of Nursing Research