Abstract
Objective We investigated the use of systematic review automation tools by systematic reviewers, health technology assessors and clinical guideline developers.
Study design and settings An online, 16-question survey was distributed across several evidence synthesis, health technology assessment and guideline development organisations internationally. We asked the respondents what tools they use and abandon, how often and when they use the tools, their perceived time savings and accuracy, and desired new tools. Descriptive statistics were used to report the results.
Results 253 respondents completed the survey; 89% have used systematic review automation tools – most frequently whilst screening (79%). Respondents’ ‘top 3’ tools include: Covidence (45%), RevMan (35%), Rayyan and GRADEPro (both 22%); most commonly abandoned were Rayyan (19%), Covidence (15%), DistillerSR (14%) and RevMan (13%). Majority thought tools saved time (80%) and increased accuracy (54%). Respondents taught themselves to how to use the tools (72%), and were most often prevented by lack of knowledge from their adoption (51%). Most new tool development was suggested for the searching and data extraction stages.
Conclusion Automation tools are likely to take on an increasingly important role in high quality and timely reviews. Further work is required in training and dissemination of automation tools and ensuring they meet the desirable features of those conducting systematic reviews.
Competing Interest Statement
MC, JC and CF report having developed several of the systematic review automation tools that were included in the survey. PG, JC, and AMS report having received an infrastructure grant for systematic review automation tool development from Bond University. PG reports having received an NHMRC Centre for Research Excellence in Digital Health grant (APP1134919) which partly funds systematic review automation tool development. ZM is employed by JBI, an evidence-based healthcare research and development organisation situated within the University of Adelaide and had a key role in the development of JBI SUMARI. ZM is a GIN Board Trustee, Vice-Chair and a member of the GIN Tech group. ZM is supported by an NHMRC Investigator Grant APP1195676. No specific funding was obtained to conduct this project, and none of the reported funders were involved in the conception, design, conduct, or reporting of this work.
Clinical Trial
Not applicable
Funding Statement
No specific funding was obtained to conduct this project, and none of the reported funders were involved in the conception, design, conduct, or reporting of this work.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Bond University Human Research Ethics Committee provided approval for this project (AS200903)
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The data collected as part of this study are available on reasonable request from the corresponding author, AMS.