USA

Currently, the USA is the only country where RWE is explicitly mentioned in its legislation, resulting in the publication of a formal RWE program by the FDA [17]. The American regulatory Agency was also among the first to identify the need to provide concrete guidance for the use of RWD/RWE, not only for medicines, but for MDs as well. In 2017, the Agency released the first of as series of guidances in an attempt to define and form a new path in regulatory decision-making for MDs that would include the use and evaluation of RWD and RWE [18, 19]. The framework is divided into four main parts:

• Definitions of RWD and RWE and the scope of application under the 21^st Century Cure Act.
• Use of RWD to generate RWE.
• RWD/RWE evaluation framework for regulatory decision-making.
• FDA’s internal and external involvement with relevant stakeholders in the development of RWE.

FDA defines both RWD (“data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources”) and RWE (“clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of RWD”) (see Table 3), but does not define the planning, conducting, and reporting processes pertinent to the collection of RWD and RWE.

The submission of study protocols, including RWD on drugs and biological products and their statistical analysis prior to conducting a study, has also been addressed by the FDA. The protocols focus on the early discussion of relevant patient populations, study exposures/outcomes, study duration, the continuity of data coverage, study plan amendments, and potential confounds related to the study design and data collection processes [20]. RWD/RWE quality is affected by the data life-cycle (e.g., data source-curated, data-transformed, data-analytic dataset) and the FDA stresses that all actions and mechanisms should be documented and evaluated accordingly throughout the study period. The importance of taking into account the “well-captured” (i.e., well-measured data) and “not well-captured” (e.g., smoking, lifestyle, disease history, nutrition, etc.) confounders that may compromise data quality remains one of the major goals for the FDA.

The FDA acknowledges various sources of RWD (see Table 4) in the MD context, including electronic health records (EHRs), claims/billing activities, product/disease registries, patient-generated data (including in home-use settings), and data gathered from other sources that can inform health status (i.e., mobile devices). The main aspects to consider when assessing a RWD source’s eligibility should be accuracy, completeness, integrity, adequacy, data consistency, and a well-tested hypothesis. Overall, data accrual and data assurance/quality control are essential to provide a stable ground for the creation of good quality RWD data. Nevertheless, according to the FDA, the quality of the supported data must be critically assessed to determine whether the RWD sources are reliable, clinically relevant, and sufficient, which also depends on data quality levels required for a given regulatory decision.

The FDA recognizes two major challenges of RWD collection that may compromise data quality; one is the inherent bias included in these data sources (e.g., bias that compromises data quality and creates issues in drawing conclusions on MD exposures and outcomes), and the second is the actual data collection process. However, patient preference information, unique device identifiers (UDI), paper/electronic patient records, etc., can act as verifiable sources of documentation. With respect to registries, there is an ongoing discussion on how they can be used and the level of relevance and reliability they introduce [21]. The relevant draft guidance from the FDA invites stakeholders to discuss the following:

• The ability to accurately define and evaluate the target population based on the planned eligibility criteria.
• Which data elements will come from the registry (versus other data sources) and their adequacy.
• The frequency and timing of data collection.
• The planned approach for linking the registry to another registry or other data system (if required).
• The planned methods to ascertain and validate outcomes, including diagnostic requirements.
• The level of validation or adjudication of outcomes for which a consensus must be reached.
• The planned methods to validate the diagnosis of the disease being studied.

On the other hand, RWD sources may include, but are not limited to, randomized trials, large simple trials, hybrid and/or pragmatic trials, and observational studies (prospective/retrospective). The FDA will consider RWE as sufficient to support a regulatory decision for an MD when, a) the methodologies used to generate RWE are scientifically sound (i.e., methodology/analysis of RWD, clinical relevance, statistical significance, etc.), and b) the RWD data used to generate the RWE data are also evaluated as sufficient, relevant, and adequate (e.g., with regards to population, source verification procedures, timelines, sources/technical methods, etc.). However, the threshold for data sufficiency is dependent upon the regulatory decision that needs to be made.

Along with the RWD/RWE, the FDA also requires the evaluation of the overall device safety and effectiveness endpoints via the Objective Performance Criterion (OPC) and the Performance Goal (PG). Both OPC and PG act as numerical target values, providing information on the technology/performance and safety of the MDs, with OPC being more robust and trustworthy. The FDA guidance provides only limited insight on data/patient privacy/confidentiality concerns. Nevertheless, it has been among the pioneers with respect to establishing patient privacy as an important factor when working with RWD/RWE.

The latest document issued by FDA on September 2022 adds a practical perspective to the use of RWD/RWE by outlining the need to include cover letters with their submission, to explicitly identify the use of RWD/RWE in support of product labelling [22].

Europe

The European Medicines Agency (EMA) and the European Commission have been discussing the use of RWD and RWE in relation to medicinal products for the last ten years. As early as 2015, the use of RWE for the regulation of medicines was explored in the context of the EMA Adaptive Pathways Pilot [23]. A retrospective analysis of clinical trials submissions to the EMA during 2018 and 2019 revealed that RWE was included in two-fifths of initial marketing authorization applications, and in one-fifth of indication extension applications [11]. A more recent analysis by Eskola et al. revealed that nearly all European Public Assessment Reports (EPARs) included RWE in drug discovery (98%) and life-cycle management (100%), and nearly half included it in the development phase (49%) or to support regulatory decisions at registration (47%) [24].

In 2019, the EMA published the Operational, Technical, and Methodological (OPTIMAL) framework for regulatory use of RWE in regulatory decision-making, which aimed to set out the appropriate use of valid RWE for regulatory purposes (e.g, safety, efficacy, benefit–risk monitoring), to highlight the operational, technical, and methodological challenges for use of RWD to generate acceptable RWE, and to propose potential solutions [12].

In 2020, the EMA published the Regulatory Science to 2025 strategic document, which vividly promotes the use of high-quality RWD in regulatory decision-making to generate complementary evidence across a medicinal product’s life-cycle. This includes the development of a framework for rapid and safe access to the latest RWD, and the incorporation of training into the regulatory processes based on EHRs and other routinely collected health data [25]. EMA’s Regulatory Science to 2025 also emphasizes the key role of harmonization across international stakeholders to enhance data sharing and sets the stage for the embedment of data analytics and digital tools for everyday analysis of healthcare data, aiming to promote standardization of RWD. Furthermore, the EMA and FDA reinforced their collaboration on the regulation of medicines with the use of RWE in 2018 [26].

More recently, in August 2021, the EU put RWE in the wider context of big data and revisited their approach, guided by the priority recommendations of the Big Data Task Force, implemented through the Big Data Steering Group [27]. In November 2021, EMA published a document describing their vision for the use of RWE in the European Union, aiming to establish the use and value of RWE across a spectrum of regulatory use cases [28].

As part of the EMA-HMA Big Data Steering Group workplan, EMA has recently launched the Data Analytics and Real World Interrogation Network (DARWIN EU), which is part of a wider 11-workstreams EU policy context, most notably the European Commission’s plans for a European Health Data Space [29, 30]. Darwin EU will operate through the co-establishment of a coordination centre, currently under supervision by the Erasmus University Medical Centre. It aims to support the roles of EMA and the European medicines regulatory network in their regulatory decision-making by conducting studies to access and analyze healthcare data from across the EU, to increase the use and receptiveness of RWE in regulation of medicines, and to reduce the time and cost of drug development. At the same time, Darwin EU will become the managing depository of RWD by maintaining a catalogue of data sources along with their metadata for their use in regulatory processes.

DARWIN will deliver 4 types of observational analyses and studies:

• Routine repeated analyses. These analyses, based on a generic study protocol, could include periodic estimation of drug utilization, safety monitoring of a drug product, or estimation of the incidence of a series of adverse events (AEs).
• Off-the-shelf studies. These studies will take a generic protocol and adapt it to a specific research question, such as estimating the prevalence or characteristics of exposures.
• Complex studies. These studies require the development or customization of specific study designs, protocols, and Statistical Analysis Plans, with extensive collection or extraction of data. For example, studies that look at associations between exposures and outcomes while adjusting for confounding factors.
• Very complex studies. These studies require complex methodological work and would rely on more than just an electronic health care database source.

During the project’s last meeting in July 2022, the DARWIN EU advisory board aimed to increase the number of acceptable RWD sources and to propose a set of guidelines to improve the efficiency of RWE studies. [31] A technical workshop has been planned to identify common “use-cases” that could provide regulatory insight for all involved stakeholders, including EMA, HTA bodies, and payers [32].

The RWD sources for DARWIN EU will include data on primary and specialist care, hospital care electronic health records, claims databases, disease registries, patient-reported outcomes, and drug prescription and dispensing data. The European stakeholders acknowledge that RWD may arise from registries, e-wearables, and EHRs, all of which are susceptible to challenges related to quality, introduction of bias, and heterogeneity, due to non-standardized collection practices. Up until the time this article was written, EMA had not published a comprehensive clear guidance with best practices on RWE collection. However, there are 6 ongoing pilot studies under Darwin EU and their results will be used to inform the forthcoming decisions of data harmonization and metadata collection and subsequent reporting, as well as regulatory decision-making [12].

On November 2022, EMA announced the first data partners to collaborate with DARWIN EU. The Data Analysis and Real-World Interrogation Network aims to provide timely and reliable evidence on the use, safety, and effectiveness of medicines for human (including vaccines) from real-world healthcare databases across the European Union [33]. In 2024, the first year of operation, DARWIN EU will increase its capacity to routinely support EMA and NCA National Competent Authorities (NCAs) by delivering studies and maintaining data sources. The network will be fully operational by end of 2025- beginning of 2026, when it will be integrated with the European Health Data Space.

As part of its efforts to support the use of RWE, EMA also established the Patient Registry Initiative in 2015, which aids data harmonization within different disease areas and across different national registries [1, 34]. Europe is privileged with an affluence of healthcare data, and for this reason patient registries are often used (see Table 2). However, heterogeneity, restriction in access due to national laws, as well as data protection laws, such as the General Data Protection Regulation (GDRP), challenge the potential for data integration and harmonization. EMA has repeatedly underlined the need for data standardisation and expansion of RWD data sources, proposing the use of legally acceptable consent and data anonymization techniques to conform with data privacy requirements. However, so far, the impact of GDPR has hampered the expansion of the use of RWD/RWE for regulatory purposes.

While EMA has stretched its intention to ‘promote use of high-quality RWD in decision-making’ as published in “EMA regulatory science to 2025”, it has yet to develop a framework that includes concrete guidelines regarding the generation, reporting, and handling of RWE big data, including statistical considerations. It is important to note that EMA has been working with RWE only for medicines and pharmaceuticals and, to the best of our knowledge, there is no mention of RWD/RWE use for MDs.

UK | MHRA/NICE

In December 2021, the United Kingdom’s Medicine and Healthcare products Regulatory Agency (MHRA) published a consultation document on RCTs generating RWE to support regulatory decisions [35]. The guidance covers simple and hybrid trials and sets out the factors that need to be considered when collecting RWD as part of a clinical trial. More specifically, it highlights the importance of accuracy, validity, variability, reliability, and provenance of the data source. Confirmation that the data quality is considered sufficient for the intended use should be included in the protocol. According to the British Agency, there is no barrier to using RWE to gain an initial approval or approval of a new indication, providing the data quality is “robust” and the trial is “designed in a way which allows it to provide the evidence required to answer the regulatory question”.

A second guidance specific to clinical trials is currently under development and will be incorporated in the future [36]. Although the principles are applicable to clinical trials in any area, this guideline will refer only to pharmaceuticals. The guideline does not cover the clinical trials using RWD as a control arm. The RCTs using RWD should be of the same standard that would be expected for a traditional RCT, including pre-specification of the objectives, data to be collected, primary and secondary endpoints, and analysis methods. Evidence generated from RCTs using an RWD source is not generally considered of more or less value for regulatory decision-making than evidence from traditional RCTs, provided the data quality is robust and the trial is well-designed. Data from randomized trials using an RWD source are currently most used for:

• Label changes for licensed products (including drug repurposing).
• new populations (different age groups, different disease severity, etc. to what is already licensed).
• change in dose, or route of administration.
• adding a new indication (repurposing of existing medications).

RCTs using RWD should be sensitive and structured in a way that encounters the challenges arising from collecting data outside the controlled environment of a traditional clinical trial. In a real-world setting, differences in how the study is conducted could introduce noise or extra variability, which could be a result of clinicians not following the same protocol, differences in background care, or other factors.

Regarding the general framework for the use of RWD in clinical studies, MHRA considers it important to demonstrate that the data source is of sufficient quality for the intended use. Care should be taken to understand the origin of the source database, along with any transformation or manipulation that may have occurred during its processing. Users must be able to define the provenance of the source data, explain the mechanisms used to link data points, manage discrepancies, and describe any limitations or considerations associated with the data. With respect to endpoints to be tested, MHRA prefers the ones that are well recorded in the UK (overall mortality is the most representative example, but other examples are available). There is no difference in reporting between the traditional clinical trials with those using RWD. However, some flexibility is possible around the requirements for safety reporting, with flexibility depending on the proposed population and expected AE profile compared to what is already known about the safety profile and this would be considered on a case-by-case basis. MHRA adopts the same data quality concerns that most authorities express with respect to credibility, assurance, and replicability, and requires that data quality processes and checks are in place to ensure accurate reporting, interpretation, and verification. According to the Agency, RCTs using RWD should follow the same rules for privacy/confidentiality as the traditional RCTs; patient consent is required before enrolment and it should be clearly stated that the privacy and security policies apply to the use of all databases as well as the restrictions related to the transfer, storage, use, publication, and retention of data.

In June 2022, the UK’s HTA appraisal body, the National Institute for Health and Care Excellence (NICE), published its own framework on RWE, reviewing the methods and processes applicable for HTA programs [37]. The framework is one of the most comprehensive documents available so far and provides in-depth guidance and tools to support the use of RWE for HTA-related processes and decision-making. While describing NICE’s expectations for planning, conducting, and reporting RWE studies, it assesses data suitability, describing the information needed to assess data provenance, quality, and relevance for addressing specific research questions. It also provides more specific recommendations for conducting non-randomized studies. An overview of the NICE framework is tabulated in the Supplementary material.

Australia

In line with the increasing realization of the importance of RWD and RWE use in regulatory decision-making, the Therapeutics Goods Administration (TGA) has recently requested a review from selected stakeholders to identify how RWE is perceived and utilised by both manufacturers (pharmaceuticals/biologicals/MDs) and TGA assessors [38]. The review focuses on the use of RWE/RWD and patient-reported outcomes (PROs), thereby separating these sources of evidence. However, in a guidance document by TGA on the collection of clinical evidence for MDs, RWD is considered to cover all types of clinical experience data “generated through any clinical use of the device that is not related to clinical investigation”, including EHRs, registries and PROs (“patient-generated data including in home-use settings”). Other sources of RWD identified in the guidance are claims and billing activities, as well as data gathered by Digital Health Technologies (DHTs) (e.g., mobile devices and applications collecting data on health status). Nonetheless, the review established that RWE and PROs are currently not commonly utilised in pre-market submissions and approvals, certainly in comparison to post-market submissions. Additionally, TGA identified that, whereas “critical use of RWE for emerging technologies, such as gene, cell and tissue therapies, and software-based medical devices is a critical and necessary component to understanding and enhancing the performance of such products”, there was a gap in the communication to manufacturers on the acceptance of RWE and PROs in regulatory submissions [39].

To overturn this, TGA initiated several activities, including promotion of the use of RWE for regulatory pre-market submissions, and the introduction of a requirement for clear communication on the contribution of RWE in the application or decision-making. Pre-submission meetings are also encouraged to discuss the types of RWE included, as well as their quality and acceptability for regulatory purposes. Additionally, for MDs (particularly for software-based devices), in accordance with the 2019 Action Plan for Medical Devices, TGA is committed to publishing more information on how RWE and PROs are used in the regulatory submission and how they contribute to decision-making, at least for higher risk devices [40].

Implementation of UDIs for MDs is anticipated to enable better traceability and, therefore, support the collection of higher quality RWD that could push forward the use of RWE for MD regulatory submissions. In support of higher data quality, TGA advises the establishment of registries with a high number of included patients and device types. Finally, the collection of RWD is expected to follow protocols to ensure elimination of bias and Good Clinical Practice (GCP) principles.

TGA acknowledges that pharmaceuticals and MDs have different development pathways; formal clinical studies (clinical investigations) are mainly conducted for higher risk MDs. Additionally, clinical investigations are usually smaller and often “simpler” compared to RCTs for pharmaceuticals, as in some cases it is not feasible to use randomisation and/or blinding in the study design. Within this context, the Australian Agency deems necessary to develop different guidelines for the generation and use of RWE in regulatory submissions for medicine and MDs. Constraints in resources, however, appear to preclude the generation of guidance documents by the TGA. Therefore, TGA tends to follow other regulatory authorities (such as FDA, EU, Health Canada) or the International Medical Device Regulators Forum (IMDRF) for MDs and to adopt already developed guidelines.

Health Canada

Health Canada has recently expressed their intention to optimize the use of RWD/RWE in the regulatory decision-making process as part of the R2D2 (Regulatory Review of Drugs and Devices) project and respective follow-on initiatives. Health Canada and the Canadian Agency for Drugs and Technologies in Health (CADTH) held a joint workshop in 2018, launching an initiative to integrate RWE across the life cycle of drugs [41]. The 2 stakeholders co-decided that the full integration of RWE was necessary to help overcome marketing authorization and reimbursement challenges in the country [42, 43, 44].

Following this preliminary framework, Health Canada published a guidance document on April 16, 2019, acknowledging that the use of RWE in regulatory decision is increasing globally for the assessment of drug safety, efficacy, and effectiveness [45]. An enclosed document identified that certain diseases/disorders (such as rare diseases) could not be sufficiently studied with the conduct of RCT and, as such, RWE studies could be the answer to the unmet medical/regulatory need [46].

Health Canada currently limits discussion of RWE primarily to medicines. However, CADTH participates in several international initiatives such as CIOMS, INAHTA, and ISPE/RWE that consider MDs as well (see Table 3). The proposed framework requires that a protocol should clearly describe the data sources utilized and the appropriateness of these data to capture all relevant exposures, outcomes, and covariates of interest. RWE leverages data collected in the routine care of patients through sources such as electronic medical records, healthcare claims data, or disease registries. The study population should be well-defined, and the protocol should provide information on the sampling framework, demographic characteristics, clinical exposures, and duration of follow-up. Inclusion and exclusion criteria and a description of the generalizability of the study may also be included. For both prospective and retrospective collection of RWE, the protocol should describe how the outcomes are relevant to the claim and the approaches taken to demonstrate the robustness of the findings to any sources of bias. Outcomes should be clearly defined and measured. The protocol should not only address the validity of any measures but predict the collection of items that could modify the effect and how they will be included in the analysis.

Expanding data and evidence sources to include RWD/RWE may facilitate generalizability by incorporating data from rare diseases and/or sensitive patient populations. The Canadian authorities point out the need for clear data collection methods that include pretesting and identify the need to ensure data quality and consistent data management as major challenges.

Latin America

Despite the growing interest in RWD/RWE in Latin America, significant gaps exist compared to other regions in the world. To the best of the authors’ knowledge, the regional local authorities have not issued formal guidances, therefore there are no official definitions for RWD and RWE, and the terms are used based, mostly, on FDA’s definitions. Moreover, the collection and use of RWE for regulatory and healthcare decision-making is inconsistent across countries, providers, and insurance stakeholders, while RWE initiatives are unfavoured due to lack of resources and concrete guidance on how to collect and manage RWD. Cultural and socioeconomic diversity, and the uneven access to technology and health resources across the continent, also hamper data quality and the ability to ensure representativeness of data.

The ongoing discussion about the impact of digitalization aims to generate a framework for the facilitation of access to RWE. Yet, the emergence of various HTA agencies across the continent has not been able to promote incorporation of RWE in regulatory processes until now, mainly because the available sources of RWD are limited and lack reliability, thus rendering the use of data from RCTs more attractive and realistic [47].

A recent study by Justo et al. identified that the commonest RWD sources in Argentina, Brazil, Chile, and Colombia are clinical databases, including EMRs patient registry databases of observational cohort studies, and health information systems, including surveillance systems and administrative databases [47]. The main uses of RWE in Argentina, Brazil, Chile, and Colombia align with those of most countries around the world (i.e., for pharmacovigilance purposes, post-marketing surveillance of pharmaceuticals, HTA decisions related to reimbursement and economical evaluations, and academic research). Interestingly, there seems to be limited use of RWD/RWE to support access of orphan medicines into the market and RCTs are still considered the main source of data for marketing authorization.

Various initiatives are currently ongoing aiming to increase awareness and understanding of how RWE can be used in Latin American countries to boost healthcare decision-making (see Table 2). ANVISA, Brazil’s Regulatory Agency, is also working to revise Resolução da Diretoria Colegiada 200/2017, which sets outs the requirements for marketing authorization of medicines containing synthetic and semisynthetic active ingredients. The goal is to enable access to effective and tolerable treatments in the absence of traditional RCTs and it is expected that incorporation of RWE in this approach will play an important role [48].

Greater China

The development of RWE in China has been promoted significantly in the past few years and has taken a distance from the evaluation of outcomes and comparative effectiveness of the traditional Chinese medicine interventions. During 2020, the Chinese National Medical Product Administration (NMPA) published a series of preliminary guidance documents including a technical guideline for RWE supporting drug development, and an interim technical guidance for using RWE to support research, development, and regulatory review of pediatric drugs [49]. Chinese Authorities were among the first to issue a guidance for the use of RWE to support the evaluation of MDs, which was later supported by a more generic guidance on how to use RWD to generate RWE [50]. Overall, China scopes the use of RWE around drug development and support of post-marketing studies. Different case studies are discussed in which RWE may support drug development and regulatory decisions, including the use of external controls for approval of rare disease treatments and orphan drug assessments where the conduct of an RCT is neither feasible nor ethical, as well as the use of RWE for expansion of indications of currently marketed medicinal products. The Chinese approach focuses on medicines for pediatric use and how RWE can support pediatric drug development. In particular, the NMPA specifies circumstances for which RWE could introduce unique benefits both for product development and approval processes, specifically:

• RWE studies allow for greater flexibility in patient recruitment and trial design for diseases with small patient populations and scattered cases like rare diseases.
• For infantile diseases, or life-threatening diseases with rapid disease progression, RWE studies offer timely treatments of trial therapies when control group enrolment may not be ethical.
• For certain surgical MDs in which it is technically difficult to perform RCTs, RWE studies afford alternatives to sham surgeries for efficacy and safety evidence.

Given the size and complex infrastructure of China, one of the major challenges for Chinese Authorities is to determine reliable, reproducible data sources that can be used for epidemiology and health economics purposes, as well as research outcomes that are comparable and acceptable by the international community. Within this context, Xie et al. [51] grouped the majority of RWD used in China into 6 types: a) administrative claims databases (including national and regional levels), b) EHR (including regional EHR databases, multicenter, and single-center EHR databases), c) databases linking EHR with claims, d) cohort or registry data, e) medical chart review studies, and f) surveys of the general population and patients. The first 2 types are relatively new in China and their use is subject to limitations, such as limited data access/sharing potentials, lack of longitudinal follow-up data, and sub-optimal data quality. In addition, lack of regulations related to patient privacy protection is also hindering the use of RWD/RWE, as bureaucratic requirements, and lack of a central ethics review board in China complicate the use of the above sources. On the other hand, the potential to collect data from large-size cohorts creates research opportunities for general and rare diseases.

In July 2020, the Taiwan Food and Drug Administration (TFDA) finalized a guidance document on the points to consider for using RWE to support the research and development of drugs. This was followed by the November 2020 guidance on the use of EMR data in clinical investigations to improve data accuracy, and to promote clinical trial efficiency and increased interoperability [52]. TFDA generally tends to align with FDA’s rationale and promotes early engagement and regular communication with regulatory authorities to establish transparency in real-world study design. The agency is investigating acceptable uses of RWE to support a wide range of regulatory decisions, including label changes and premarketing safety profile evaluations, the use of periodic safety update reports and periodic benefit-risk evaluation reports of pharmaceuticals from other countries, and premarketing effectiveness assessments (e.g., clinical evidence of orphan medicines) as RWD sources of safety data.

Japan

Japan has a long history of initiatives related to the use of RWD/RWE and the RWD Working Group of the Pharmaceuticals and Medical Devices Agency (PMDA) has issued a series of relevant documents (see Table 1) [53]. In previous years, focus was mainly on drug safety assessment at the post-marketing stage and the efficacy evaluation of orphan drugs, but recently there is ongoing remediation regarding the acceptability of RWD and RWE for regulatory submission in earlier stages in the lifetime of a medical product. The goal of the Agency’s Clinical Innovation Network Working Group is to develop a framework that will incorporate appropriate registry data and use of EMRs to ensure data reliability for new and supplemental drug applications [54]. The 2 most recent guidelines address the basic principles for the use of registries for approval purposes and the points to consider for ensuring reliability during their use [55, 56]. PMDA emphasizes the importance of reliability and quality of the RWD in terms of accuracy, consistency and comprehensiveness, and argues that inclusion of RWD/RWE in regulatory decision-making should be determined by the appropriateness of the analytical methods used in a study. This is because unless both reliability and appropriateness are present, the interpretability of results cannot be guaranteed. The Japanese authorities point toward sufficient framing and specification of the regulatory purpose before deployment of any activity involving RWD/RWE, to ensure that the collected data are sufficient for the regulatory setting they are being used in. Currently, PMDA receives feedback from multidisciplinary working groups on all issues relating to RWD/RWE, including, but not limited to, data reliability standards, methodological approaches, and the potential to accept new technologies, such as data collection by wearable devices and data handling and screening by artificial intelligence.

South East Asia

Collection of local RWD is of high interest to South-East Asian countries as it can be used to close the gap in data collected in RCTs or observational studies conducted in a foreign-country context without being able to inform policy-making in the local Asian context, and often, without considering differences in regional clinical practices, reimbursement practices, ethics, and judicial systems. Indonesia, Malaysia, Philippines, Singapore, South Korea and Thailand all accept RWD/RWE as part of regulatory submissions, while Malaysia and Philippines do not even require justification for their use [57, 58]. The REALISE guidance [59], a collaboration between global experts and leaders from HTA agencies in Asia, indicates that primary RWD sources in South-East Asia include registries, EMRs, claims databases, and hospital administration documents (e.g., discharge records, prescirption archives and health surveys). Within this context, reimbursement decisions in many South-East Asian health systems (e.g., Indonesia, Malaysia, Philippines, Singapore, Thailand), may take years to issue, which renders the need for ‘’imported’’ clinical feedback even more critical.

However, despite the vivid interest by various stakeholders in the use of RWD/RWE, there is still no clear consensus, neither on national, nor regional level, about how to incorporate them into existing HTA processes [57]. Although there are a few available frameworks [58], HTA systems in South East Asia remain extremely variable. Alignment of practices within a region on how to generate and use RWD/RWE is hampering effective implementation. HTA institutionalization has progressed well in Thailand, where it has been formally integrated into reimbursement decisions, including the development of the National List of Essential Medicines and the Universal Health Coverage Scheme benefits package. [60, 61] Similarly, Philippines have moved to a major reform in 2019 by signing the Universal Health Care Act, which also included the establishment of a HTA unit, which is supported by the International Decision Support Initiative (iDSI) to develop capacities to generate and use relevant HTA evidence [62].

India

Indian Authorities are currently performing an extensive reform of healthcare services and regulation of medical devices/medicines. Although there is not yet any concrete regulation or guidance of the field, progress is made in digitalization of the healthcare system and regulation of patient data with the Digital Information Security in Healthcare Act (DISHA) [63].

DISHA defines Real-World Data as […] information concerning the physical or mental health of individuals; information around health service provision; information concerning organ or bodily substance donations; information on testing or examination of organs and bodily substances; information collected during the course of health service delivery; and information on the clinical establishment accessed by the individual […]. The Indian Chapter of ISOPR has already proposed a set of guidelines for the country, while various HTA initiatives are promoted. [57, 59, 64, 65, 66] According to a recent survey [65], the main actors/user of HTA in India is the government itself, while public organisations including autonomous, research institutions, are considered as the main generator. The same survey revealed that there is limited or no access to data, such as input from pharmaceutical use, cost of service delivery, hospital-level data and health outcomes, which is consistent with missing infrasturctures, the diverging profiling of the country and the nascent leveraging of digital healthcare.

Middle East & Gulf Counties

Currently, Middle East and Gulf countries are falling behind in generating and using RWD/RWE for regulatory purposes, while grappling to transform local healthcare systems. However, a limited number of case studies showcasing the use of RWE is indeed available [67]. To address the challenges hindering effective use of RWE in the region, the Gulf Corporation Council (GCC) came together in December 2020 to identify the inherent regional restrictions and propose actionable solutions. Based on the working assumption that major challenges include the public’s fear and mistrust to share personal health data, the lack of collaboration and partnership between institutions, variability in data sources also due to ethnic differences, and suboptimal technological expertise [68], GCC stakeholders identified the need to develop a consistent governance framework on RWD/RWE taking into account ISPOR-ISPE’s work in progress and proposed a set of 7 actionable points for leveraging RWE, namely:

• Treating “real data” properly and collecting and regulating data.
• Ensuring patient confidentiality and respecting the integrity of patient data: putting ethical consideration into place.
• Establishing a continuous healthcare plan, and customizing health plans and beneficial packages to suit countries’ needs.
• Adapting high standards for evidence generation
• Establishing a common language for RWE in the countries; bridging gaps between multiple stakeholders; educating and orienting stakeholders
• Investing in more robust databases, apart from UAE; Cerner; Daman claim database; Dubai claim database; Epic
• Enhancing technical capabilities to design, execute, analyze, and interpret RWE studies for all stakeholder and ensuring interoperability across institutional, national, and countries’ jurisdictions

Currently, the focus lies heavily on digitalization of healthcare systems via the incorporation of HTA applications but since requirements for registration, pricing & procurement stakeholders are highly variable across countries of the region, infrastructural changes in delivery of healthcare services must precede local RWD generation.

To the best of the authors’ knowledge, the major HTA actions on the region include

• The publication of recommendations for reporting pharmacoeconomic evaluations in Egypt by the ISPOR Egypt Chapter [58] and the Pharmacoeconomics Unit of the Ministry of Health. [69]
• The establishment of an independent HTA committee, the National Instance for Accreditation in Health Care (INASanté) in Tunisia, [70] which has a significant role in reimbursement decisions.
• The issuing of Lebanese pharmacoeconomics guidelines [71] co-developed by the Ministry of Health, the National Social Security Funds and the Lebanese University.

Africa

Similar to the Middle-East/Gulf regions, low-and middle-income (LMICs) African countries are significantly lagging in the generation and use of RWE in regulatory and reimbursement decision-making for practically the same reasons. Due to the particularities of the regional healthcare systems related to limited budgets and prioritization of healthcare needs based on prevalence of medical diseases, focus has been on the establishment of pharmacovigilance systems aiming to promote epidemiology-related services. For example, the African Union NEPAD Smart Safety Surveillance (AU-3S programme) [72] has adapted the principles of Habitat International Coalition (HIC) to fit local needs and capabilities in sub-Saharan African nations, while in 2020 and 2021, Ghana, Nigeria, South Africa, and Ethiopia leveraged pharmacovigilance technology provisioned by MHRA to establish an ICH-compliant RWD data repository in a secure UK cloud environment [66]. Nonetheless, there is a growing interest in the use of HTAs even though there are currently no examples of independent, established HTA [73] institutions. South Africa and Ghana seem to be pioneers in the field. South Africa is the only African country, which has introduced a legal framework to protect the country’s residents from harm by protecting their personal information (South Africa’s Protection of Personal Information Act (POPIA)) [74]. Ghana has already conducted multi-criteria decision analysis studies to set up the feasibility of various interventions to be used for the country’s healthcare sector reform [73].

Statistical Considerations

Non-randomized, evidence-related confounding factors and biases are discussed in all available guidances and frameworks since they add challenges to the statistical evaluation, mainly due to uncertainties and limitations for assessment and interpretation of causal inferences [6, 75, 76]. A concrete statistical analysis plan is essential when using RWD to estimate treatment effects and clinically meaningful outcomes. The statistical analysis should be appropriate to anticipate all challenges that may occur. A broad spectrum of statistical tools may be used for the analysis depending on the nature of data and the corresponding type of study (RCT, registry, prospective cohort study, survey, etc.). When conducting an RCT using RWD, the statistical analysis plan and protocol should follow that of a traditional clinical trial. Where available, most guidelines focus on techniques (traditional or more elaborative such as machine-learning) for managing bias and confounding factors that are likely to occur when using RWD, due to discrepancies across the diverse sources of data. Another aspect often mentioned is the use of a power calculation analysis to obtain a representative sample size for the study. Only FDA mentions that RWD can be used in later analyses to conduct an informative prior probability distribution using Bayesian statistics (see Table 5).

Statistical analysis techniques may vary according to the corresponding type of study and should be planned on a case-per-case scenario. More detailed guidelines are expected from authorities to provide a more thorough framework around statistical analyses and the use of RWD in different types of studies.

RWD/RWE for the regulation of medical devices

The new Medical Devices Regulation in Europe, and the profound revisions in the regulations of Australia, Canada and elsewhere, have significantly increased the requirements for substantiative clinical evidence throughout the lifecycle of MDs (see Figure 3). [9, 77, 78] In particular, EU Regulation 2017/745 requires that manufacturers who wish to market/continue to market their MDs within Europe must develop and regularly update a process that will enable the monitoring of the real-world performance and safety of their MD. This should not only be via clinical investigations but also through the proactive, dynamic monitoring of feedback collected from patients, end-users, and healthcare professionals. Post-market surveillance requirements under EU Regulation 2017/745 are risk-based and part of a continuous process that updates the substantiative clinical evidence of clinical evaluation reports, aiming to confirm the positive benefit-risk ratio and the clinical benefit of any given MD. Within this context, the Post-Market Clinical Follow-up (PMCF) is defined as the proactive collection and evaluation of clinical data from the use in, or on, humans of a marketed device, with the aim to confirm the device’s clinical safety and performance throughout its expected lifetime, to ensure the continued acceptability of identified risks, and to detect emerging risks on the basis of factual evidence [9]. Without explicitly stating it, EU Regulation 2017/745 necessitates the inclusion of RWD for the regulatory approval of MDs. MDCG 2020-6, a European guidance on the amount and type of clinical data needed for MDs that were marketed before the coming into force of EU Regulation 2017/745 (‘legacy devices’) to demonstrate conformity with the relevant General Safety and Performance Requirements (GSPR), states that RWD, such as registries and information deriving from insurance database records, can be used to demonstrate indirect clinical benefits [79].

In contrast to pharmaceuticals, where RCTs seem to remain the standard option for substantiation of claims, ‘’traditional’’ clinical data may not always be able to serve the requirements of PMCF under EU Regulation 2017/745. This is primarily due to the extended scope of PMCF, which aims not only to confirm the approved indications, targeted population, and potential restrictions of use (contraindications, warnings, precautions), but also to identify the real-world use of a device and therefore to highlight potential off-label uses, either in terms of clinical applications or the targeted population. RWE that feeds into the clinical evaluation of a MD is drawn from the entire population exposed to the device. For this reason, RWE, by definition, is non-comparative and focuses on the performance of the device in scope, rather than making comparisons to other devices or treatment options. Since PMCF is intended for devices already on the market, there is no experimental exposure involved, no upper limit on included subjects, and no eligibility criteria. Furthermore, PMCF is not necessarily conducted in clinical settings, as data can be collected directly from patients and/or end-users. Vigilance data i.e., data collected from databases managed by Competent Authorities for the reporting of (serious) incidents, field safety corrective actions (FSCAs) and recalls, are among the sources of data used in PMCF, underlying the growing role of RWD in the regulation of MDs around the globe.

In this context, development of medical device registries is significantly favoured. The IMDRF defines a medical device registry as an […] an organized system with a primary aim to increase the knowledge on medical devices contributing to improve the quality of patient care that continuously collects relevant data, evaluates meaningful outcomes and comprehensively covers the population defined by exposure to particular device(s) at a reasonably generalizable scale (e.g., international, national, regional, and health system) [80, 81]. […] In the previous years, patient registries have been restrictively used for the marketing authorization of medical products with low-disease prevalence or high-disease severity intended for small and heterogeneous patient populations, especially when the level of clinical evidence submitted were not deemed sufficient. Lately, under the generalized reform of healthcare by the introduction of innovative new technologies, registries are considered a major source of RWD/RWE, receiving global attention. The International Medical Device Regulators Forum (IMDRF) is exploring the possibilities of prospective national data collection in multiple countries, with the potential for subsequent data linkage to provide information from very large numbers of patients longitudinally. [80, 81]

Typically, registries [82] become an option for MDs on a post-market surveillance level for the collection of long-term safety data. In case of implantable MDs, the flexibility of registries to accommodate RWD in a dynamic manner is gradually establishing their role as a key supplementary method for the evaluation of long-term safety. Indeed, registration of all implantable devices under the EU Regulation 2017/745 is compulsory and both notified bodies and manufacturers are required to consider registry data, not only to ensure traceability but also as part of their strategy for continuous collection of data about patient characteristics and clinical outcomes. Registries that collect a predefined, limited set of variables with an easy and quick procedure may be more effective in motivating data providers and, therefore may be triggering a higher level of data quality. However, when assessing the level of evidence collected through a medical device registry, the overall design and rationale of device categorisation, the type of selected patient-reported outcomes, and the ability to link and/or verify the registry’s data with other data sources play a decisive role. The International Medical Device Regulator Forum Registry Working Group has defined 15 registry requirements [83], grouped into six elements, to assess the suitability of registry data for regulatory submissions, namely:

• Governance: Transparent governance structure and processes.
• Quality management system: conforming with legal requirements for data collection, access to data and patient data protection.
• Data gathering: consideration of relevant variables and ability to link to other sources, explicit device identification through the UDI system.
• Data storage: protection against hacking, altering, deleting, or stealing data.
• Methodology/data analysis: multifactorial analysis and data interpretation.
• Transparency/display/distribution: publicly available reports and accessible website and web-reporting.

Therefore, when considering the use of registry data for regulatory submissions, it is necessary to provide evidence of systematic and complete coverage transparency of quality assurance, clear policies for data access and sharing, registry sustainability and conformity with ISO 14155:2020 requirements.

From a PMCF perspective, a concrete sustainability plan laying down short and long-term strategies for the collection of RWD, along with monitoring and interpretation is critical to establish the continued acceptability of the benefit-risk ratio as well as to monitor the emergence of previously unidentified risks for MDs through a MD registry.