Standards and guidelines for observational studies: quality is in the eye of the beholder

open access

 The use of real-world evidence is critical to transforming health care, and policies need to address this issue. Although the availability of multiple standards/guidelines has the potential to improve the quality and credibility of observational studies, there may be unintended consequences. If those guidelines differ from one another, decisions based on one guideline may subsequently be found deficient if measured against a different one.

 

What is new?

Key findings

•

Nine major standards/guidelines for observational studies have areas of disagreement based on a comparison of 23 common elements.

What this adds to what was known?

•

Comparison of the standards/guidelines is presented, along with an assessment of how actionable each is, and a policy discussion of next steps is provided.

What is the implication and what should change now?

•

Lack of standard/guideline agreement may contribute to variation in study conduct.

•

Common standards/guidelines for conducting observational studies will benefit funders, researchers, journal editors, and decision makers.

•

A consensus process to determine a common set of standards/guidelines needs to be established.

 The nine standards/guidelines are as follows: Agency for Healthcare Research Quality (AHRQ): Developing a Protocol for Observational Comparative Effectiveness Research [10]; Comparative Effectiveness Research Collaborative: Observational Study Assessment Questionnaire [11]; European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) Checklist for Study Protocols [12]; ENCePP Guide on Methodological Standards in Pharmacoepidemiology [13]; The United States Food and Drug Administration (FDA): Guidance for Industry Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment [14]; Good ReseArch for Comparative Effectiveness (GRACE) Checklist [15]; GRACE Principles [16]; ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report (Parts I, II, and III combined) [7], [8] and [9]; and Patient-Centered Outcomes Research Institute (PCORI) Methodology Standards [17].