#HEORInsights

During the course of any research, most of the relevant literature pertaining to a research question is retrieved though searching of recognised databases. However, in addition to this, searching of grey literature can add value to the depth of the research by providing information from varied sources. Grey literature search is an important, but often ignored, part of systematic literature review and data synthesis, especially in the medical research.

Grey Literature: Definition, Types, and Sources

The Grey Literature International Steering Committee (GLISC) defines grey literature as “Information produced on all levels of government, academics, business and industry in electronic and print formats not controlled by commercial publishing i.e. where publishing is not the primary activity of the producing body”.[1] Grey literature is often self-published, and the sources of grey literature can include Government agencies, research institutions, organizations, companies, and associations.[2]

Grey literature can be classified under various categories: [2,3]

• Regulatory Information: this includes information available from archives of regulatory bodies such as the USFDA, CDSCO, EMA, and NICE. Regulatory information can also be sourced from stakeholder organizations and pharmaceutical companies, and include product information leaflets, white papers, internal documentations, SOPs, and procedure briefs. Other examples include company and industry wide repositories and financially driven investor service websites.
• Government data: these include information available from government resources, such as notifications, guidelines, gazette notifications, judicial information, patent databases, policy briefs, etc.
• Unpublished material from clinical trials: these include prospectively registered clinical trial protocols in repositories such as ClinicalTrials.gov in the USA and the Clinical Trial Registry of India (CTRI). Pre-prints which are not ultimately published due to various reasons, unpublished dissertations and theses,
• Conference proceedings: abstracts, scientific sessions, and other conference proceedings provide a brief snapshot of contemporary research, which might not get published due to various reasons
• Internet resources: With the increasing presence of social media, newer forms of grey literature have also surfaced, such as blogs, internet forums, wikis, video lectures, lecture slides and lecture notes, educational videos, personal websites, and information posted on the omnipresent social media.

Importance of Including Grey Literature

Commercial publishers are guided by interests and priorities, and all information which do not conform to these are often ignored and not published. This unpublished information forms the bulk of grey literature.

A research which focuses solely on published material has a risk of missing out a comprehensive view of the topic under research. Grey literature provides valuable information about emerging or less popular research areas which are not published. Including a grey literature is also found to be useful in validating the results of a research-based literature search.[4]

Grey literature bypasses the time-consuming peer-review process. Also, because of a quicker publication, the time delay between research and its formal publication is also bypassed. As a result, the information in grey literature can be more recent and up-to-date than in a formal publication. This is especially important in situations of public health emergency, such as the COVID-19 pandemic.

There is a growing interest in using grey literature in systematic reviews and meta-analysis.[5] A study conducted by McCauley et al. analysed that 33% of meta‐analysis included some form of grey literature, accounting for 4.5% to 75% of studies in the meta‐analyses, and contributed significantly to the estimates of the intervention effects. The authors concluded that excluding grey literature from meta-analyses can result in a falsely exaggerated estimates of the effectiveness of intervention.[6] In several cases, information, and data disclosed at conference presentations is never published.[7] These observations have led to an opinion that the confounding effect of publication bias (where negative findings are more often not published) can be mitigated to a significant extent by including grey literature.[5-7]

Challenges in Grey Literature Searching

It is certain that grey literature search adds value to a research; however, it is also not possible to ascertain whether such a grey literature search has been done comprehensively or not. This is because grey literature is not well organized. In other words, there is no way to ‘define’ a proper grey literature search. It might be possible that a researcher has done grey literature search and included only data that is favourable, while excluding unfavourable data. This is in stark contrast to the published literature, which is often found to be well-organized. Thus, while it is possible to duplicate a database search strategy to verify if the search has been performed properly or not, such a luxury is not available with the quite unorganized grey literature. This is also the reason why inclusion of grey literature is more often than not entirely dependent on the choice of a researcher. Further, grey literature is not ensured to be peer-reviewed, which brings in an inherent bias.

Conclusion

The importance of transparency in research findings cannot be over emphasized. Publication bias still continues to be a huge problem in medical research. Grey literature search has a unique potential to improve transparency in medical research as well as offer a solution for publication bias, by including the unpublished information, thereby improving the comprehensiveness of research. With the increased ease of access of the internet, the access to grey literature has also become easier. Considering the amount of new information that grey literature can bring to research, all researchers must consider including grey literature search in their works. Efforts are required to organize the grey literature so that the credibility and validity of grey literature search can improve.

Become A Certified HEOR Professional – Enrol yourself here!

References

1. Grey Literature International Steering Committee. Guidelines for the production of scientific and technical reports: how to write and distribute grey literature. Available from: http://eprints.rclis.org/7469/1/nancy.pdf. Accessed on Jun 20^th 2020
2. Royal Roads University. Grey literature: what is it?: What is grey literature. Available from: https://libguides.royalroads.ca/greylit/what. Accessed on Aug 12^th
3. Citrome L. Beyond PubMed: Searching the “Grey Literature” for Clinical Trial Results. Innov Clin Neurosci. 2014 Jul;11(7-8):42-6.
4. Benzies KM, Premji S, Hayden KA, et al. State-of-the-evidence reviews: advantages and challenges of including grey literature. Worldviews Evid Based Nurs. 2006;3(2):55-61.
5. Mahood Q, Van Eerd D, Irvin E. Searching for grey literature for systematic reviews: challenges and benefits. Res Synth Methods. 2014;5(3):221-34.
6. McAuley L, Pham B, Tugwell P, et al. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228-31.
7. Hopewell S, McDonald S, Clarke M, et al. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007(2):MR000010.