Insights from Experts Convened by the U.S. Food and Drug Administration (FDA) and the National Academies of Sciences, Engineering, and Medicine.

Historically, randomized controlled trials have been used to determine the effectiveness and potential uses of biopharmaceutical products and medical treatments. These trials offer many advantages for researchers, including the ability to replicate testing conditions and to create control groups. Real-world evidence (RWE) and real-world data (RWD) collection on a large scale also offer real help in determining the ways in which physicians and patients actually use therapeutic options and biopharmaceutical compounds. RWE and RWD are increasingly expected by payors as they make coverage and reimbursement decisions, set formularies, negotiate rebates, and establish value-based performance expectations for providers. Real-world evidence is also essential for demonstrating the clinical utility of advanced diagnostic tests.

Three workshops on the applications and benefits of integrating real-world evidence were held through a joint project of the U.S. Food and Drug Administration (FDA) and the National Academies of Sciences, Engineering, and Medicine. The presentations and information provided at these workshops were published in Examining the Impact of Real-World Evidence on Medical Product Development: Proceedings of a Workshop Series. One of the primary goals of these workshops was to create a framework that would allow stakeholders to make the most appropriate and relevant use of RWE and data in clinical studies and evaluation processes.

Setting the Stage for Success

During the first workshop, which was held in September 2017, participants were asked to lay the foundation for creating decision aids for the use of RWE and RWD in the design of studies and research into various medical products and their uses. These decision aids were divided into four separate topics:

  • Assistance in determining when RWD offers insights into eligibility for studies, treatment arrangements or potential outcomes for medical products
  • The identification and minimization of bias in observations and comparisons
  • Factors to consider when allocating interventions in trials intended to provide RWE for researchers
  • Points to keep in mind when taking steps to control treatment quality in real-world trials 

These topics were explored during the course of the three workshops, which also introduced case studies as ways to investigate the various issues that might arise during the course of implementing RWE studies and using RWD as a tool for decision making. The general issues to be addressed were incorporated into the introduction, which also offers a useful overview of the workshops and the contents of these proceedings.

Defining the Terms

Another important task addressed by these workshops was the defining of terms related to RWD and RWE in the medical product research field. According to Gregory Simon, RWE is defined by four basic characteristics:

  • Relevance: RWD and RWE is relevant to the research performed now and into the future.
  • General applicability: The information derived from RWE will provide useful insights that will hold true for larger groups of patients in similar situations.
  • Efficiency: RWE and RWD can be collected and compiled in less time and with reduced expense compared to formal research studies.
  • Adaptability: Because RWE is collected from providers and patients in an actual treatment setting, it typically includes a wider range of test subjects. This allows the results to be applied to a larger group of potential patients.

By defining RWE and RWD in these ways, the sponsors of the workshops hoped to create broadly applicable recommendations for the incorporation of these methodologies into the research and development of new medical products and treatment plans.

Best Practices for RWD

An ongoing partnership and cooperative agreement between the FDA and the Margolis Center for Health Policy at Duke University (Duke-Margolis) has identified several best practices that can ensure the best use of RWD and the most practical approaches for regulatory agencies and researchers alike:

  • Ensuring the most robust sources for primary data
  • Integrating randomization where it is appropriate
  • Assessing methodologies to make sure they are in accordance with the “totality of the evidence”
  • Creating transparent and proactive analytic plans
  • Collecting sufficient data to make practical and actionable recommendations

Making sure that these practices are in compliance with regulatory requirements and standards will be critical to the ongoing success of implementing RWE and RWD and incorporating it into the research and testing strategies used to determine the effectiveness and the practical uses for medical products.

Insights Derived From the Workshops

The participants in the FDA-sponsored workshops brought some of their own perspectives and observations of their patients to the forefront of the discussions. Some of the key messages identified by individual workshop participants were summarized in the following ways:

  • Self-interest: Patients were most interested in RWE and RWD that applied to their own treatment plans. These individuals were also prone to misconceptions about ownership of data and erroneously believed that their data was their own property.
  • Consistency and reliability: Participants indicated that they found value in the reliability offered by the current system, which includes clinical trials, reviews of current practices and data collection within delivery systems.
  • Budget issues: Money continues to be a key factor in determining whether organizations, manufacturers and pharmaceutical firms will be willing to invest in RWE and RWD technologies. A framework of value-based agreements might serve to create greater stability for drug prices.
  • Patient-centric approaches: One area that may show significant promise is the development of shared data pools for patients and for communities of patients and caregivers. While the information available from these sources may not be entirely accurate, patients may eventually be a primary source for information about new treatments and existing medical regimens.
  • A high bar for evidence: Most participants in the workshops agreed that the bar for RWE and RWD should be set comparably high to that for evidence derived from randomized controlled trials.
  • Fit for purpose: Widely accepted as one of the most important factors in the success of any RWE or RWD implementation, deriving relevant and usable data through these methods is essential to justify the cost and effort involved in this process.

Three primary groups who would be affected by the integration of RWE into current research evaluation and testing systems were identified:

  • Payors indicated that cost was an overriding concern both in providing access to newer treatments and in delivering some treatments to their patients. Uncertainties about the costs of newer drugs and their effectiveness could reduce willingness to participate in RWE activities among healthcare providers.

  • Patients were most concerned about the effectiveness of newer treatments and medical products. Rather than expressing interest in the way in which these medications are tested, patients typically took a “bottom-line” approach to determine whether a particular treatment plan represented a good chance for a positive outcome.
  • Delivery systems typically take an analytical approach to incorporate RWE and RWD into their evaluation process. These organizations usually must perform a balancing act between potential benefits for patients and the costs of testing and implementing new intervention methods.

Buy-in from all three of these groups must be achieved to make widespread implementations of RWE and RWD a reality in the regulatory and medical environment.

Success Stories Already Exist

The workshops also reviewed some of the existing success stories, including the Salford Lung Studies, the FDA’s own Sentinel product monitoring system and registries already in place for patients and caregivers.

The Salford Lung Study

In 2017, some of the initial results of the Salford Lung Study were published in the scholarly journal Pragmatic and Observational Research. This study evaluated the safety and effectiveness of an inhaler treatment that uses a combination of fluticasone furoate and vilanterol in a dry powder form for patients with asthma or chronic obstructive pulmonary disease (COPD).

By combining the results of randomized clinical trials with RWE collected from patients with COPD, researchers were able to evaluate the effectiveness of these treatments with much more accuracy. Some issues were encountered during the process, including the lack of resources to ensure data collection and to store it safely and properly. By creating relationships among stakeholders and establishing a cloud-based platform that allows easy access to RWE for all researchers, the Salford Lung Study continues to provide real assistance to researchers into asthma and COPD treatment options.

The Sentinel Initiative

The FDA established the Sentinel Initiative in 2008 to ensure compliance with new legislation regarding the use of electronic health data to evaluate medical products after their release. In practice, Sentinel serves as a central location for data regarding medical product safety and patient outcomes for data partners throughout the United States. The data collected is curated to ensure that it is easily searchable and can be cross-referenced to derive the most accurate and actionable information for researchers and FDA personnel.

A number of Sentinel programs were discussed, including the following six examples:

  • A study to determine the safety of Warfarin and Rivaroxaban
  • Assessment of risks of bowel obstruction in infants who had been vaccinated for rotavirus
  • A project to link mothers with infants in the Sentinel system
  • The use of data from electronic health records to identify subjects for randomized controlled trials related to coronary artery disease
  • Field-testing of a mobile app for pregnant women to collect data on physical symptoms and treatments
  • An ongoing study on the use of oral anticoagulants and their effectiveness

Practical applications for the Sentinel Initiative and the data collected through this system are still being explored by the FDA and their partner organizations.

Challenges and Barriers

Acquiring the right RWE data and putting it to practical use can sometimes present significant challenges for stakeholders in this process. Many of the obstacles present in the process of managing big data are also applicable to the RWE and RWD acquisition, identification, and implementation process. Some of the most critical challenges identified by participants in the joint National Academies and FDA workshops included the following:

  1. Lack of knowledge about the required steps for obtaining and sharing the information
  2. Security concerns
  3. Worries about the risk or inaccuracy possible with RWE compilations and the process of choosing the right research methodologies
  4. The work involved in integrating data sources from a wide range of different providers and stakeholders
  5. Incompatibility of RWE data collection systems with existing systems in the healthcare environment
  6. Inadequate or unavailable training in the complexities of RWE implementations
  7. A “wait-and-see” attitude adopted by many healthcare providers regarding RWE, RWD and new technologies in general
  8. A lack of clear regulatory guidelines that will provide a framework for the collection, storage, and sharing of RWE
  9. Patient concerns about data privacy and the ways in which their information will be used, which can suppress participation

The FDA and other regulatory agencies can pave the way for increased participation and easier access to shared data by participants in these programs. Additionally, by providing training options and providing practical demonstrations of the value and reliability of RWE and RWD to stakeholders, government agencies can promote the most holistic approach to the identification of clinical evidence in the healthcare field.

Creating New Clinical Paradigms

Some of the most important potential uses for RWE have been identified as the following:

  • The establishment of the right biomarkers before studies begin to ensure the right selection of participants in randomized controlled trials and other medical research activities
  • Access to a wider range of patient records for rare diseases to help researchers and physicians to predict outcomes more accurately for their patients
  • Reducing the approval times and increasing the accuracy of patient indications for use of new medical products, including pharmaceuticals and devices
  • Creating hybrid approaches that incorporate randomized controlled trials with RWE that can be collected as these trials progress, allowing researchers to enjoy the best of both worlds when evaluating available evidence and results

As more stakeholders engage with the RWE process and begin sharing their data in a practical and actionable way, the applications of these data sets will expand to ensure the most practical benefits for patients, healthcare providers and manufacturers in the pharmaceutical and medical equipment industries.

Determining Whether Data Is “Fit for Purpose”

One of the most essential tasks in the implementation of RWE is the identification of the relevant information in what can be a mountain of data. Data must be curated and identified with relevant tags and metadata to ensure that it can be located by researchers in connection with their current activities. Creating a common nomenclature and working to eliminate sources of bias in the database can ensure that the information can be sorted, searched and utilized to provide the most accurate results for researchers. Making sure that all collected data is as accurate as possible is critical to the success of RWE and RWD implementations in the research environment and can ensure that the data is suited to the purposes for which it will eventually be used.

Controlling Treatment Quality in Real-World Trials

Another question that will affect the success of RWE implementations is the degree to which researchers should attempt to control the types and quality of treatment received by patients. This requires the establishment of standards of care that are broadly applicable across a wide range of patient situations and care conditions. Determining which patients will be eligible for treatments and which should be excluded from RWE and data acquisition will be a continuing challenge during the implementation process.

The Benefits of Blind RWE Studies

One of the primary advantages of randomized clinical trials is the ability to avoid patient and provider bias during these activities. Finding ways to provide these same benefits when conducting RWE research can ensure the most useful results. Conducting blind RWE studies can be challenging, however, because of the nature of real-world interactions between providers, researchers, and patients. By taking steps to eliminate bias and changes to outcomes that could be caused by patients and providers who are aware of the goals of ongoing studies, researchers can ensure the most accurate information and conclusions when dealing with RWE and data.

Establishing Greater Confidence in RWE and RWD Results

According to participants in the FDA workshops on RWE implementations, the degree of confidence achievable with RWE and RWD is directly related to the type of information being sought in the trials and studies:

  • Comparisons of two treatment types can often produce useful information for medical personnel in determining which of these options is best suited for patients across a spectrum of illness severity and physical health.
  • Treatments that produce measurable effects are also best suited for augmentation or study through RWE methods.
  • RWE can be used to support existing randomized clinical trials and to fill in gaps in knowledge that can sometimes arise from these research activities.
  • Insights into rare disease can be derived more easily with a wider group of patient experiences on which to draw.
  • Maintaining transparency in the RWE process is critical to ensure the greatest degree of engagement and buy-in by researchers in the field.

One advantage of RWE and RWD is the ability to use data collected for one purpose to support other activities over time. For instance, the information collected to support insurance claims or to itemize treatment costs can also be used to support some types of RWE for a specific illness or physical condition. This could potentially allow researchers to access a complete range of information that could serve to supplement or to complement randomized controlled trials in creating the most effective treatment options for patients.

As the infrastructure and robust data sources for real-world evidence continue to grow and develop, more researchers, life sciences firms, and healthcare providers will take advantage of the testing and research capabilities of these curated collections of information. This will improve the quality of care for patients and will help to streamline the regulatory approval, payor coverage decision making, and technology assessment processes for new drugs, biologics, diagnostic tests, and medical devices.