#HEORInsights

Systematic literature reviews (SLR) are a crucial tool to identify high-quality evidence for research, policy, and practice. The success of SLRs heavily relies on the performance of the literature search strategy, which involves utilizing appropriate information retrieval (IR) techniques to locate relevant studies. (1,2) Appropriate IR techniques are essential to capture all relevant studies, and to avoid retrieving irrelevant studies as far as possible. (3) In this direction, systematic reviewers often use combination of specific search strings, Boolean operators, limits and search filters to optimize the search strategy, and also to ensure that the search is comprehensive, systematic, and reproducible. (4,5)

Search filters are tools that help to retrieve certain types of records based on their characteristics, such as methodology, study design, or publication type. Filters can be applied to various aspects of the search, such as the population, intervention, study design, and so on, and are usually guided by the PICOS of the SLR. For example, while conducting an SLR on the clinical efficacy and safety of a specific intervention, a filter may be applied to only include randomized controlled trials (RCTs) involving the specific intervention and a predefined set of comparators. (5) On the other hand, ‘limits’ in a search strategy restricts the number of search results by setting a predefined maximum number of results, and can be based on aspects such as publication type, language, or publication year. For example, for updating a previously published SLR, a limit may be used to restrict the search results for the past 5 years. Other examples for limits include country restrictions and language restrictions. (6)

Having a comprehensive search strategy without any restrictions or limits is highly desirable for pooling together all relevant research to help unbiassed evidence synthesis. However, at times, researchers will need to use limits in search strategy to optimize the literature search process and make the SLR more time-efficient:

• By narrowing the scope of search to specific fields, such as title, author, descriptors, language, publication year, and country of publication, limits help to conduct more focused and targeted searches. (7)
• Since the same limits can be applied across multiple searches, consistency and reproducibility of search results can be enhanced. (8)
• Using limits can be particularly useful in cases where a large amount of literature is available on a given topic, and a rapid evidence synthesis is needed to support quick decision-making.

That said, using limits are associated with certain disadvantages:

• Since limits can reduce the sensitivity of the search, some relevant studies may be missed out, thereby decreasing the completeness of the SLR. (3)
• Bias may be introduced towards certain type of studies, especially if too many limiting commands are used. (3)
• Over-reliance on search limits can decrease the transparency of the systematic review process. (7)
• Limits may potentially cause an oversimplification of the research question, and this may reflect in incomplete or inaccurate SLR conclusions. (6)

At times it becomes essential to use limits in search strategy for various reasons. In such cases, the following guidelines may be helpful to utilise the advantages of the limits while reducing their disadvantages:

• The research question and the SLR objectives musts be considered carefully for determining which limits are appropriate for the search strategy, and which limits are to be avoided. (9)
• Limits must be judiciously selected in order to avoid bias and ensure a comprehensive and systematic search without missing relevant information. (5)
• Limits must be combined with appropriate search strings to maximize the effectiveness of the search strategy, thus ensuring relevant results. (5)
• The appropriateness of each limit used must be carefully considered against the backdrop of the potential benefits of limiting the search and the potential loss of valuable studies. (10)
• The limits applied to the search strategy must be recorded diligently, including the rationale for selecting the limits, to evaluate their impact on the search results, and also to ensure that the search strategy is transparent and reproducible. (7)

The use of limits in literature searching requires a balanced approach. If used with proper justification, they can help refine search results and improve the review’s precision, and also ensure timely completion of SLR. (10) Systematic reviewers must weigh the benefits and potential drawbacks of limiting their search, make informed decisions based on the research question and the scope of the review. By using limits appropriately and judiciously, it is possible to improve the accuracy and efficiency of literature searches and thus produce a high-quality SLR.

Become A Certified HEOR Professional – Enrol yourself here!

References

1. Hemingway P, Brereton N. What is a systematic review? Hayward Medical Communications. What is…? series. 2009 Available from http://www.bandolier.org.uk/painres/download/whatis/Syst-review.pdf 
2. del Amo IF, Erkoyuncu JA, Roy R, Palmarini R, Onoufriou D. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications. Computers in Industry. 2018 Dec 1;103:47-71.
3. McGowan J, Sampson M. Systematic reviews need systematic searchers. Journal of the Medical Library Association. 2005 Jan;93(1):74
4. Kugley S, Wade A, Thomas J, Mahood Q, Jørgensen AM, Hammerstrøm K, Sathe N. Searching for studies: A guide to information retrieval for Campbell. Campbell Systematic Reviews, 13: 1-73.
5. Papaioannou D, Sutton A, Booth A. Systematic approaches to a successful literature review. Systematic approaches to a successful literature review. London, Sage, 2016:1-336..
6. Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Information & Libraries Journal. 2010 Jun;27(2):114-22.
7. Higgins JP, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane handbook for systematic reviews of interventions. John Wiley & Sons; 2019 Sep 23. Available from https://training.cochrane.org/handbook 
8. Koffel JB, Rethlefsen ML. Reproducibility of search strategies is poor in systematic reviews published in high-impact pediatrics, cardiology and surgery journals: a cross-sectional study. PLoS One. 2016 Sep 26;11(9):e0163309.
9. Greenhalgh T. How to read a paper: the basics of evidence-based medicine and healthcare. Wiley Blackwell; 2019 May 6.
10. Glanville J, Bayliss S, Booth A, Dundar Y, Fernandes H, Fleeman ND, Foster L, Fraser C, Fry-Smith A, Golder S, Lefebvre C. So many filters, so little time: the development of a search filter appraisal checklist. Journal of the Medical Library Association: JMLA. 2008 Oct;96(4):356.