Agile vs. Regulatory: How the two coexist and contribute to successful Medical Device Software Development
Agile vs. Regulatory: How the two coexist and contribute to successful Medical Device Software Development
Software is gaining relevancy in a broad range of medical devices, as it either enables the control or influence of their operation, or because Software as Medical Device (SaMD) itself has the potential for detection, diagnosis, treatment and alleviation of diverse diseases/disabilities.
General Safety and Performance Requirements (GSPR as per Annex I of the Medical Device Regulation MDR 2017/745 and In Vitro Diagnostics Regulation IVDR 2017/746) are applicable for all SaMD manufacturers. The regulation states in section 17, chapter II of Annex I that:
“17.2. For devices that incorporate software or for software that are devices in themselves, the software shall be developed and manufactured in accordance with the state of the art taking into account the principles of development life cycle, risk management, including information security, verification and validation”
State of the Art
The IEC 62304:2006 – medical device software – software life cycle processes standard is considered the state-of-the-art for software development. Manufacturers shall leverage this standard to comply with the GSPR.
Although this standard is process-oriented and defines a set of specific requirements for manufacturers, Quality Management Systems (QMS), Standard Operational Procedures (SOPs) and lifecycle, it is not essentially different from other standards for software development.
Furthermore, it is not incompatible with Agile methodologies for software (SW) development. In fact, there is a Technical Information Report (TIR45:2012 – Guidance on the use of Agile practices in the development of medical device software) providing some insights and unification of terms for Agile development of SaMD.
In this post, we highlight some differences and correspondences in software development terms between Agile SW and SaMD development regulations, in order to help project teams work better together and channel their efforts in the right direction:
1. Lifecycle
The IEC 62304 requires manufacturers to define the lifecycle and detail the entire process from requirement collection (pre-market) to problem resolution (post-market). Three lifecycles are mentioned in the standard: waterfall, incremental and evolutionary.
Agile is considered incremental/evolutionary and is therefore recognized by the standard as an adequate methodology.
Manufacturers should make it clear, from the beginning, if and when they are using the Agile development methodology. They should always provide this information in the Technical Documentation to enable the software development team and regulatory bodies to understand their development process.
It’s important, however, that the methodology is aligned with the company’s QMS procedures for design and development and embeds the requirements of IEC 62304!
2. The concept of “Done”
When Agile teams have completed an activity, this activity is considered finished and no further actions are expected once it is delivered.
Similarly, section 5.5 of IEC 62304 lists very similar requirements. However, the concept of “Done” should be expanded to include conditions such as:
1) review of requirements,
2) approvals, tester teams and testing conditions,
3) expanded acceptance criteria
4) documentation level required.
In other words, the concepts are similar and the “Done” concept and the underlying process can be widely leveraged to build compliance against IEC 62304, however, additional steps need to be checked before moving an activity to “Done”.
Techletter
Medical Device Software Incorporating Artificial Intelligence
Generating Sufficient Evidence under the MDR
3. Moving to Verification & Validation (V&V)
According to the regulatory requirements as per IEC 62304, integration, hardening and V&V activities need to be performed before the software becomes shippable (final release). However, this is quite difficult for each sprint outcome. To manage it, the Software Development Plan (and most particularly its Verification Plan subpart) must define a “Done is Done” criteria in order to collect a minimum set of increments/epics that could be considered adequate to undergo V&V testing.
Validation according to the regulation is a more complex concept including not only technical performance but also clinical association and clinical validation. For further details on Clinical Validation of Medical Device Software, check our dedicated webpage.
4. Release
Before software is released, all Verification & Validation activities need to be completed and the results evaluated as per the IEC 62304 section 5.8.
Agile methodologies are aligned with these requirements. Indeed, the activities performed at increment level are documented using software development tools, and then at the end of the processes, Agile teams must ensure proper documentation of the consistency and acceptance criteria of all tests that have been performed.
Another requirement set in IEC 62304 is to document known residual anomalies. In Agile, this is created by conducting a review of each increment and searching for the potential known anomalies/bugs to be addressed.
Finally, manufacturers need to ensure that a new release is assessed against the criteria of significant or substantial change, as per Article 120 and Annexes related to Conformity Assessment procedures of the MDR and IVDR. Further details to determine whether an evolution (update/upgrade) might fall under the significant changes as per Article 120 under MDR are found in the MDCG 2020-3 guidance. No guidance related to the concept of substantial change has been published to date.
The examples provided above represent just a thin part of the parallelisms between the Agile development process and IEC 62304 requirements. Based on our experience, most of the software development processes that are IEC 62304 agnostic are very often just a few adjustments away from compliance with the regulatory requirements.
At Medidee, now part of Veranex, we have been working with numerous software development teams, helping them leverage their pre-existing tools, working practices and methodologies to build compliance against IEC 62304, and other related standards, in a pragmatic fashion.
If this is something you are currently working on, do not hesitate to check our full offering of Digital Health Services, or to reach out directly for expert support: https://www.medidee.com/contacts/
This article was written by Dr. Gustavo Hernandez and Dr. Nuria Gresa.
[ARTICLE] IVD Manufacturers' New Year's Resolutions: 6 Reasons to Start the Performance Evaluation of your Device Now
Once a device is used for diagnostic purposes on human specimens, the European Union expects, in accordance with Article 5(3) of Regulation (EU) 2017/746 (IVDR), a performance evaluation of the device as a demonstration of compliance with the relevant General Safety and Performance Requirements (GSPRs) and this regardless of the device’s risk class.
As defined within Article 2(44) of the IVDR this means an “assessment and analysis of data to establish or verify the scientific validity, the analytical and, where applicable, the clinical performance of a device”.
The performance evaluation shall be based on a continuous process and follow a defined and methodologically sound procedure, based on an established device-specific plan, i.e., the Performance Evaluation Plan (PEP). The identified performance and safety data, which shall allow to demonstrate compliance with relevant GSPRs related to the device’s performance characteristics, is then consolidated in the Performance Evaluation Report (PER). The PER is a part of the device’s technical documentation and shall be reviewed by the Notified Body during the conformity assessment procedure. De facto, manufacturers of legacy IVD devices will need to invest resources in evaluating the performance of their devices prior to their transition to IVDR.
Here are 6 reasons why manufacturers should not put off this endeavor and start the process now
1. It ensures alignment with the current "state of the art"
In accordance with the requirements of the Regulation (Annex XIII), a PEP must include “a description of the state of the art, including an identification of existing relevant standards, Common Specifications, guidance or best practices documents”. Implementing the PEP as soon as possible will ensure that the device is still aligned with the current state of the art and does not require major design improvements or additional testing according to the latest guidelines. This is especially important because “the performance evaluation of an IVD must consider the benefit-risk ratio in light of the state-of-the-art” and as further stipulated in MDCG 2022-2, “risks for IVDs are often generated from a series of events which may involve several factors such as inadequate design characteristics as well as immature technology”. Therefore, establishing the state of the art is the first step in the preparation of the performance evaluation. As it involves a systematic and methodological approach to the literature review, it requires time and resources.
2. It allows for an early assessment of the level of clinical evidence required
The level of clinical evidence needed to fully demonstrate the performance and safety of the device as claimed by the manufacturer will depend on the characteristics of the device and its intended use. Manufacturers transitioning their devices to IVDR shall assess as soon as possible the level of clinical evidence required. This includes verifying that all performance and safety claims, including those used for marketing purposes, are supported by an adequate level of evidence. Doing so will either allow for timely follow-up actions to collect additional performance data in case some claims are not yet sufficiently supported by clinical evidence, or to reword the device’s intended use taking into consideration the existing clinical evidence.
3. It helps to identify and react to gaps in the analytical performance
Conducting a performance evaluation in advance will allow the manufacturer to identify gaps in the data supporting the analytical performance of the device in a timely manner. If gaps are identified, the manufacturer will be able to generate new evidence in accordance with common specifications, guidelines, or applicable standards, to demonstrate that the IVD in question is capable of reliably, accurately, and consistently detecting the analyte, and thus GSPRSs linked to the analytical features of the device are adequately addressed.
4. It determines if sufficient clinical performance data is available
It is also important to promptly determine whether sufficient clinical performance data is available to support the device's intended use, as well as the claimed indications. The level of clinical evidence should be consistent with the clinical strategy defined in the PEP. As per MDCG 2022-2 “the manufacturer should demonstrate that the IVD has been tested for the intended use(s), target population(s), use condition(s), operating- and use environment(s) and with all the intended user group(s)”. This last aspect is particularly important when it comes to devices intended for point-of-care testing, where manufacturers need to ensure that the clinical data reflects the device’s use environment. Available clinical data may come from the manufacturer’s own clinical performance studies, Post-Market Surveillance (PMS) or peer-reviewed literature. Identifying gaps in advance will allow time to plan and eventually conduct a clinical performance study in compliance with IVDR and ISO 20916:2019, if necessary. It is important to remember that under Article 56(4) of the IVDR, “Clinical performance studies in accordance with Section 2 of Part A of Annex XIII shall be carried out unless it is duly justified to rely on other sources of clinical performance data”.
5. It supports risk identification and management
MDCG 2022-2 (6.2) states that “the risk management system should be carefully aligned with and reflected in the performance evaluation process of the IVD, considering the clinical risks to be addressed as part of the performance evaluation, performance studies, and post-market performance follow-up(s)”. Hence, manufacturers must ensure that all risks identified through the performance evaluation process are adequately addressed within the device’s risk management documents. Timely completion of the performance evaluation will allow the manufacturer sufficient time to address newly identified risks, if necessary.
6. It facilitates robust Post-Market Surveillance and Performance plans
In the event that the time frame is still too short to address all identified performance gaps through appropriate V&V testing or clinical performance studies, the manufacturer will be able to work on a robust PMS and Post-Market Performance Follow-up (PMPF) plan aligned with the performance evaluation outcome. Additionally, in case new PMS and PMPF data were collected after the initial performance evaluation, the PEP and PER should be updated in light of this new data prior to the device’s conformity assessment. As this always requires a certain amount of time for proper implementation, it is best to plan ahead.
In conclusion, the performance evaluation may identify some gaps in the quantity or quality of available clinical evidence or in the completeness of the device’s risk management documentation or PMS/PMPF plan. Depending on the extent and nature of these gaps, it may take time to address them, especially if a clinical performance study is needed, which may require months or years. Additionally, manufacturers should keep in mind that a conformity assessment procedure can take 18-24 months, which means that for legacy IVD devices classified as Class D or C according to IVDR, compliance with GDPR is expected by May 26, 2025 and 2026, respectively.
With this in mind, we advise you to place the performance evaluation of your device as a top priority on your to-do list for 2023. Contact Medidee today to discuss your needs and how Medidee supports you in addressing them: www.medidee.com/contacts
Don't miss any of our upcoming IVD content! Follow Medidee on LinkedIn
This article was written by Dr. Julianne Bobela.
