Systematic Reviews & Evidence Synthesis Methods

A protocol outlines your objectives, planned methodology, and eligibility criteria. Developing and publishing a prospective protocol is considered a best practice in conducting a systematic review. Publishing an a prior protocol (in advance of the work) is a requirement for many publishers and a recommendation of others. Additionally, publishing a complete protocol provides the authors with a set of guidelines to follow while working on the initial review and facilitates reliable reproducibility post-publication.

Recommended Items for a Protocol

• Introduction
  • Research Question 
  • Background and Rationale
  • Objectives
• Methods
  • Criteria for selecting studies for this review (eligibility criteria): study characteristics such as study design, setting, time frame, participants, interventions, outcome measures, years of publication, language.
  • Search strategies and databases in which to conduct the searches
  • Information about the data collection, screening and analysis processes
  • List of data that will be sought from the studies
  • Process that will be employed to assess risk of bias and strength of evidence
  • Data synthesis process

Note: elements may vary by discipline

"The preparation of a protocol is an essential component of the systematic review process; it ensures that a systematic review is carefully planned and that what is planned is explicitly documented before the review starts, thus promoting consistent conduct by the review team, accountability, research integrity, and transparency of the eventual completed review."

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement

Eligibility criteria, also known as inclusion/exclusion criteria for a systematic review should be established before beginning your review. Eligibility criteria may include the types of studies most relevant for answering your research question. For clinical fields, you may also want to define the population you are looking at and include specifics in your eligibility criteria related to the population data.

Be sure that you have a clear, defensible reason for each criteria you include. While it may be tempting to limit by date or by peer review, this can introduce an element of bias into your methodology. Instead of including only peer reviewed articles, consider empirical studies as an eligibility criteria.

If you want to exclude older papers that may not be as relevant to your research question, think about why a particular date may be relevant. For example, was a technique, technology, or theory essential to your research question developed in a certain year? Then you have defensible grounds for excluding papers written before that year.

Librarian tip: Review existing systematic reviews and protocols on topics similar to yours to see what types of eligibility criteria other authors are using. This can guide you in developing your own.

As part of your protocol, you will want to think through the types of data and outcomes that you will be looking for in the studies. Sometimes a systematic review is part of a meta-analysis, but even if you don't plan to conduct that level of data analysis, you should clearly state in your protocol what data you are looking for in the studies. This may be qualitative, quantitative, or mixed data.

For an example of this, view the Data Collection and Analysis section of this protocol registered in the Cochrane Library.

Aye, S.Z., Ni, H., Sein, H.H., Mon, S.T., Zheng, Q., & Wong, Y.K.Y. (2019) D‐cycloserine for autism spectrum disorder. Cochrane Database of Systematic Reviews. Issue 10. Art. No.: CD013457. DOI: 10.1002/14651858.CD013457.

As part of your Protocol, you will want to consider if you have an appropriately sized team of researchers for conducting a Systematic Review. Disciplines vary on team size expectations, however there are some general best practices that you may want to consider.

• 3+ team members involved in study selection process.
  • It will also be helpful at this stage to outline how you will resolve disagreements about which studies meet eligibility criteria.
• 2+ team members involved in data analysis.
• Librarian involvement, including peer review of search strategies using PRESS guidelines, see Librarian Support page for more information.

Establishing team member roles in advance is extremely beneficial for smoothly conducting and documenting the Systematic Review process.