Quality Assurance When Developing Software with a Medical Purpose

Abstract

In the field of development of scientific or medical software, questions may arise, such as how we define if software has a “medical purpose,” what regulations may apply and how they influence the (projected) pathway. We may find ourselves embroiled with the new “In Vitro Diagnostics Regulation” (IVDR) and its implementation in organizations. In this chapter we will attempt to summarize and order key bits of information, as found in these standards and related publications, that seemed relevant along the way in our software development processes. After this we will try to expose possible pitfalls that could be encountered. We also reach out to (existing) methodologies that may aid in the endeavor to the realization of software. Top-down risk approaches consider hierarchical ordering of priorities based on process levels where context and meaning play a more significant role over content and documentation. To honor the different sources, we will seek to outline how this led to a form of understanding that allowed the development of software. Maintaining a high standard of risk control while keeping focus on product realization. Hopefully these outlines and referred source materials may bring slight relief to others on a similar quest.

1. Introduction

Science may be seen as something that is involved in measuring what a thing is not. Science may be about hypotheses where it may be a form of communication, which is in a constant flux, challenging earlier stated definitions. Because the hypothetical does not exist, we set limitations and perform measurements according to systematic methodology, that may then lead to understanding of possibilities. When enough views are examined to concern that what is observed is certain within as set of specifications, then we may have a better understanding of what a thing may be. Techniques that investigate copy number aberrations may be no different, they typically measure that a test sample is not equal to a control reference. Controls to assure quality are also similar, in that they typically aim to measure that a sample it not according to certain standards. In practice, such standards may lead to empirically defined acceptance criteria, that could for instance be used to determine that the DNA concentration of a sample is not within the expected range. These ranges or acceptance criteria are often initially determined by predefined logic, but they may be adjusted following evaluation of discernable observations that via empirical logic may lead increased accuracy. Laws may be similar in origin as well; in that they typically focus on what should not be done, though they not as often may be adjusted based on empirical logic. Consequently, conscious law may invite its subordinates to examine “what is too much,” but often fails to provide understanding and transparency about its purpose.

When developing software that may have a medical purpose, we may find ourselves embroiled in the practical application of regulations that are currently being enforced, such as the IVDR. We may also find ourselves in contact with these regulations, for instance via implementations of these standards into company policy. While standardization certainly has positive effects: to the quality of care, the consistency of outcomes and reducing costs, in other areas it may be experienced as impeding endeavor. Research may be such an area where regulations based on predefined logic block innovation. Innovation in science may leads to progressive leaps by discovery via intuitive understanding of discernable observations. Understanding of possibilities may for instance have led to the introduction of Multiplex Ligation-dependent Probe Amplification (MLPA) in 2002 [1]. When developing devices such as software, that may have a medical purpose, we may find ourselves stuck in the morasses of regulations as well. These standards and their implementation in organizations typically tell us what to do, or rather what we should not do, but they do not seem to focus on explaining how we should get things done. One-to-one implementations of regulations often lead to documentation cultures where the intended purpose of implemented regulations may have been lost altogether.

In this section, we like to discuss and consider strategies involved in the development of software that may or may not lead to a product with a medical purpose. We will try to use the new IVDR standard to provide guidance and consider obstacles that may be prevalent in commercial endeavor. Can we develop high quality products and fulfill the requirements of the IVDR harmoniously? By evaluation of the different pieces that make up the IVDR and observation in scientific writing of the effects of its implementation in practice, we will attempt a discussion to see if the IVDR really requires huge piles of documents and rigid structures. Perhaps the IVDR does value research and innovation, but it just takes time to understand the law, by gaining experience applying the law. We may then also learn how such standards are to be implemented in crossover design, where research and diagnostic are very near to each other such as clinical trials. Finally, we will discuss a methodology to quality assurance when developing software, that is designed to react quickly to changes, which tends to be essential in a dynamic field, while also upholding IVDR standards for quality assurance with patient safety as continuous mindset. We will then use this methodology and apply it in a few examples on a software project currently in development, that involves the analysis of digital Multiplex Ligation-dependent Probe Amplification (digitalMLPA) data to establish chromosomal aberrations [2].

2. Barriers to effective implementation of regulations

Since the release of MLPA, the technique has been become a widely used tool for copy number detection and methylation status analysis on DNA samples in both a research and diagnostic setting [3]. The increased use of MLPA in diagnostics, for instance where detection of copy number variation has prognostics significance [4, 5] has led to increased demands for more diverse products, as well as their certifications for In-Vitro Diagnostic (IVD) usage. MLPA has great properties to be used in routine applications and for screening [6] to detect, treat or prevent many more serious early-onset health conditions, such as when applied for newborn screening for Spinal Muscular Atrophy (SMA) [7]. Conventional MLPA reactions are however limited to about 40 targets, and thus broad-spectrum screening may need many different conventional kits, while some diseases may even need multiple kits for a single diagnosis, such as with Duchenne Muscular Dystrophy (DMD) [8]. Also, investigation of complex tumors, such as Multiple Myeloma (MM) often requires a more detailed mapping of Copy Number Alterations (CNAs). To respond to this demand, MRC Holland (Amsterdam, the Netherlands) developed digitalMLPA, a technique combining MLPA and Next-Generation Sequencing (NGS), to detect disease-related CNAs. This technique combines the advantages of NGS and MLPA and has the potential to include up to 1000 probes in a single reaction [2] requiring only a minimum of 20 ng. digitalMLPA already showed that the method is reliable in different studies and can provide comprehensive profiling of disease-related unbalanced genetic aberrations as well as to detect specific point mutations. Due to the ability to focus on a specific range of targets, relative low cost, and short turn-around time [9], digitalMLPA could represent a valuable addition to diagnostics and screening tests.

The increase in demand to product variation and product accreditation may be seen with more products in the biochemical arena and urges companies that are involved in manufacturing to upscale. The departments that are involved in production, support, testing, sales, and shipping for instance will all need to expand. Next to that, the demand for Conformité Européenne (CE) marked products requires accreditation, which often requires further expansion of departments that are involved in quality assurance, quality control and regulatory affairs, though we may find an overhead for nearly all departments, including software development. Today, patient safety is an increased priority to both patients, providers, payers, and policy makers alike [10]. This shift requires also changes to production systems that demand higher quality standards, as well as continuity in performance and delivery. For European countries the European Union rules enforces such standards by stating that medical devices or equipment intended for a medical purpose must undergo a conformity assessment to demonstrate they meet legal requirements to ensure they are safe and perform as intended. “Software is a medical device as well if the manufacturer has made the software for medical use…,” states the website of the Dutch Government. For medical devices the Medical Devices Act (MDA) becomes applicable, and it is then important to determine in which class a device falls if one wants to continue in that direction. Depending on the class, certain requirements will apply for accreditation. This may for instance determine if the manufacturer may obtain a CE marking directly or whether an external audit need to be performed by a competent authority. Under the new Medical Device Regulation (MDR) the rules will be even stricter with regards to classification of a medical device and software has a high chance of being classified as a class IIa or higher. Though the European Union enforces such standards, there are no EU-harmonized rules governing the pricing and reimbursement of medical devices, member states are therefore responsible for establishing national social security schemes, including healthcare policies to promote the financial stability of their healthcare insurance systems [11]. One way of promoting social security systems is by expanding public health insurance aiming to reduce the risk of expenditures and improving health outcomes [12]. As a result, some healthcare providers in certain member states may only reimburse medical expenses in case the used products are CE marked. But in a commercial environment there may not always be interest to invest great amounts of time and money to obtain such markings, if a product is not economically viable to generate profit, for instance in the case of product intended to diagnose rare diseases. Even if a product for a rare disease is successfully developed, obtaining reimbursement from healthcare systems can be challenging. Pricing and reimbursement decisions often consider the cost-effectiveness of treatments, and with a limited patient population, it can be difficult to demonstrate cost-effectiveness compared to more prevalent conditions.

As regulations grow in scope and impact, understanding their precise role, effects, and limitations will become increasingly important for companies that make products and the hospitals using them. Especially for health providers there is a growing demand for transparency with the patient and the public in terms of how treatment decisions are made and when and why errors and unexpected outcomes occur [12]. So, while regulation may potentially provide a variety of benefits for instance in terms of maintaining safety and quality in patient care. At the same time, we may recognize that due to the extent, variability, and fragmentation of the regulatory system it is hard for regulations to act effectively, and it places a massive burden on provider organizations [13]. Design and production of biotechnological products and its related software may be an industry that is best understood by those who deliver it. It may thus be questionable whether there are competent authorities that have the expertise to provide dedicated investigations into quality systems that target specific areas where professionals are scarce to be found. Next to that with the new regulation being far more comprehensive than IVDD and it will require an estimated 80–90% of IVDs on the EU market to undergo a conformity assessment by a Notified Body (NB) [14]. Will there even be enough NBs to handle the increasing demands from the industry? When competent authorities cannot provide a dedicated system evaluation that provides insight into the effectiveness of such a system, then what will be their focus? Is it enough to merely evaluate that all generic protocols are in place and that version numbers on protocols and check forms match, or will they allow the profession to set the appropriate standards because it is best positioned to determined what is needed to improve safety.

Even if the profession itself may determine its protocol, then still the question arises if fast growing industries are equipped to effectively apply fast changing regulations internally while also expanding its internal machinery, especially when governments quickly apply new rules but do not have a framework yet in place to provide support and understanding on those legislations. The drive for companies to comply certainly exist, since care providers that use their products often may only reimburse their spendings in case the products used are CE marked. The drive to comply may therefore become a goal on its own and often is resolved within companies by expanding procedures and regulations. Clinical research may be an area that is being subjected by increasing regulation, research ethics committees and research and development offices are responding to social and political pressures that may lead to compromising clinical care [15].

Since progress must continue and patient care comes first, we may see in the industry that care providers find ways around the jungle of regulations to provide dedicated care for their patients. At the same time, we may see that in certain countries, individuals working in the industry completely shy away from care positions altogether where following regulations have overtaken the main priority above patient care. Shortages in dedicated trained staff are currently seen in all health care branches [16], subjecting patients to insufficient care, delays, and medical errors. Adequate staffing is essential for the delivery of quality care and safe nurse working conditions, which in turn are associated with better patient outcomes [17], A similar trend may be felt in research where overregulation is counterproductive, as it tends to discourage researchers. In other areas of endeavor such as software development or areas of technical production, companies may even outsource the required piles of papers needed to be sent to competent authorities. Outsourcing may have a negative effect, as it increases the gap between different entities involved in quality assurance, blocking harmonization of processes and feedback into design and production.

Next to that standards are often implemented by individuals that do not have an active role in the processes to which they apply. If we need to comply, then we may want to comply to patient safety and we would need to understand on an individual level how these standards may be implemented to further that goal. Enforcement to comply to regulation for its own sake may lead to atrophy of consciousness [18], as well as being a hinderance in intuitive progression via discovery. Why would this be any different when working in the medical sector where we are dealing with the health and lives of individuals? Caution comes from wisdom, not from fear (of non-compliance). Prudence requires awareness of the pitfalls to be avoided. Controls that are a threat to awareness, are a threat to patient safety and public health and therefore must be investigated on their effectiveness and improved or removed. Governments and managers ought to aid developers in providing the tools they need to properly perform the task at hand and to be established with minimal risks to patient safety. They may want to attempt to oversee and provide recommendations or key bits of information that leads to harmonization of different areas of endeavor rather than to disperse and divide. Consciousness itself may need to be our highest priority in establishing patient safety when developing products that are used as medical devices. Consciousness itself is self-regulating [19] and self-correcting which may often not be the case for ever-expanding systems that may become mostly self-serving. How can we then navigate through the morasses of regulations as developers while focusing on progress and prioritizing patient safety?

As an individual it seems irresponsible to attempt to vanquish the negative of regulation as it may lead to reductionism ignoring the social values which clearly may also play an important role in science [20]. Instead, we may apply the positive norm opportunely which practically exists only as spirit of the determining lawmaker, spirit which gives formal seriousness to some requirements which she/he considers rightful according to asset of values which contains the requirements and the interests historically determined [21]. As a scientist it may be easy to ignore the intangible and through creativity and logic deduction, dualistically reason that our intentions are good, thus as such we may avoid in clear consciousness, attempting to get deeply acquainted with new regulations. We may however also attempt to resolve the issue as a scientist and stimulate awareness and aim to create procedures that attends toward self-correction and allowing natural evolution of these procedures while considering both the product or goal and the human factor.

We could even draw this concern further and wonder why authorities themselves do not focus more on investigating legal consciousness, which is an area that can be defined as the attempt to make sense of the decisions of applicants. If even in areas of welfare, it may be clear that applicants ‘know the law’ and (ab-)use it [22], such may also be expected in other areas. In the end, when considering standards and regulations, as well as most integer companies, we may consider that they both aim for the same, with respect to patient safety. The new IVDR for instance, increases its focus on measuring effectiveness, accurate description and strengthening post marked research, all of which have been introduced for the benefit of the patient. We may also see that agencies are set in place to measure the real-world pertinence and relative clinical effectiveness of the IVDR [23]. We may further see that the IVDR as a key point aims to preserve innovation, safeguards quality, safety, and accessibility of innovative diagnostics.

When we want to design and develop medical software in a time of fast changing technologies and regulations, where demand often exceeds availability, whether related to development of products, or being knowledgeable about the exact meaning of new regulations, we may often just want to move forward and gain experience along the way if we keep our intentions always to act to the benefit of the patient. Manufacturers must strive to do the best as is possible with the information available at that time and evolve along the way as more information becomes available. Prudence, or acting in accord with what is real, is crucial when developing anything that is used in diagnosis, but we must also strive to have patience and understanding when working in teams. Stress, due to high workloads and unrealistic deadlines, rarely is an indicator that results in improvement of patient care [24] and productivity on the work floor. The virtues that we extent to patients, to some extend must as such also be applied to staff, disposing both parties to act well in relation to the end of medicine. Next to this, we may state that virtues of different people involved in medicine may be correlated with the goals of purposes of the medical encounter [25]. However, without a common agreement about goals and values, virtue theory cannot survive in health care [26]. In practice we may be more willing to get into a position of agreement, when the path ahead becomes clearer, and focus may be placed on well-defined shared objectives.

3. The pathway

The IVDR introduces a risk-based approach to classification, it is like the MDR in that it takes a lifecycle approach to IVD regulation. It also incorporates many elements of the current European guidance documents and concepts such as performance evaluation (with a major emphasis on clinical performance), vigilance and post market performance follow-up. The first step in the development of medical software, is to determine the class as it will highly influence the pathway to obtain accreditation which in turn influences how software should be developed and e.g., determines post market requirements. Besides, one may choose specific strategies that may make the progress easier and take steps that help to integrate the process into the organization. As such, the demonstration of conformity through technical files and records, quality systems and clinical evidence & performance may become a wide carried process and includes input from business executives, members of the data management team and workers throughout the whole line of development. By harmonizing the development of software and the required documentation they may work in synergy and time and effort may be spared.

3.1 digitalMLPA & software

To classify digitalMLPA software, it’s important to understand its usage alongside digitalMLPA products. digitalMLPA probe mixes consist of probes targeting specific genomic sequences, comprising a Left Probe Oligonucleotide (LPO) and a Right Probe Oligonucleotide (RPO). LPO contains a P7 Polymerase Chain Reaction (PCR) primer and a DNA hybridization sequence, while RPO includes a sequence for distinct digitalMLPA read identification and a DNA hybridization sequence. Hybridized probes that can bind to target sequences are ligated and amplified with a single primer pair, making MLPA robust for copy number assessment. The MLPA reaction involves mixing a minimum of 20 ng DNA sample with 2 ul of a unique barcode solution. After denaturation at 98°C for 10 minutes, 1.25 μL digitalMLPA probe mix and 1.25 μL buffer are added to each sample. Probes are then hybridized overnight at 60°C, saturating all target sequences in the DNA sample. Oligonucleotides are ligated enzymatically, incorporating a barcode oligo for sample identification. The ligase is inactivated by increasing the temperature to 98°C for 5 minutes and incubated at 65°C for 20 minutes to reduce background. Ligated probes are amplified using a single P7/P5 primer pair in a multiplex PCR, removing the need for unbound and non-ligated probes. The PCR-amplified products are quantified using an Illumina sequencer by utilizing the Rd1 sequence, which serves as the Illumina tag. The resulting data is stored in FASTQ files, a text-based format for biological sequences and quality scores. An average FASTQ file size is 23 Gb, containing approximately 25 million read sequences in a 7.5 Gb file. Specialized data analysis tools are required to analyze such data and provide a viable user experience that allows interpretation and give meaning to the results.

3.2 Classification

Here we describe possible classifications based on deductions using the provided information the EU has granted on the IVDR. Classification rules may vary based on regional regulations, intended use of the software and the different countries or regulatory bodies that may have their own classification systems. It is thus recommended to consult regulatory authorities or guidelines in the relevant jurisdiction for a more precise classification, we only wish to give an outline here. The IVDR uses seven rules for classi4ication along with guidance as provided by the Medical Device Coordination Group (MDCG 2020-16 Rev.2) on implementing the rules to help manufacturers to identify the risk class of the IVD devices. Assuming a risk-based position, classification starts by assuming the highest: Class D (detection of the presence of, or exposure to, a transmissible agent and/or working with infectious loads of a life-threatening diseases) or products that pose a high public health risk. The standard then tones down to Class A, products for general laboratory use which possess no critical characteristics (receptacles, buffer solutions, washing solutions, and general culture media). Guidelines to software classification follow more specific guidelines as laid out by MDCG 2019–2111. The starting point is setting out the devices, intended purpose, from which it can be determined whether it is a medical device and if it is, what the classification may be.

Software itself can be defined as a set of instructions, that processes input data and creates output data. Medical software is software intended to be used, alone or in combination, for a purpose specified in the definition of a “medical device” in the MDR or IVDR, regardless, of whether the software is independent, driving or influencing the use of a device. Medical devices are products or equipment intended for a medical purpose. This seemingly, simple question can be quite complex, and heavily related to the intended purpose that is assigned to the product, which a manufacturer should describe in detail to understand whether there is a medical action involved. In the European Union’s Regulation (EU) they rely on No. 2017/745 on Medical Devices Regulations (EU MDR) and Regulation (EU) No. 2017/746 on in vitro diagnostic devices (IVDR). Software may be considered a medical device if it’s intended by the manufacturer to be used alone or in combination, for human beings for one or more medical purposes such as: diagnosis, prevention, monitoring, prediction, prognosis, treatment or for providing information by means of in vitro examination of specimens derived from the human body, including organ, blood, and tissue donations. Not all software in healthcare is qualified as medical software, simple search and retrieval of records does not qualify as medical software, but software intended to process, analyze, create, or modify medical information may be qualified as a medical device software. We may also need to consider whether the software is a companion diagnostic or an accessory. Companion diagnostics only provide information for the use of a product and accessories indicate that the medical device can be used independently even in the absence of an accessory; neither seem to be true for software intended to analyze digitalMLPA data. MLPA data analysis software may be described as a molecular diagnostic tool as it aids in the interpretation of genetic data for diagnostic purposes. MLPA is often used in clinical genetics, cytogenetics, and research settings to identify genetic mutations or chromosomal abnormalities associated with various genetic disorders.

Once it is established whether the software is a medical device, then it needs to be considered whether it is an In Vitro Diagnostic Medical Device Software (IVD MDSW) or Medical Device Software (MD MDSW). IVD MDSW falls under the in vitro diagnostic medical devices Regulation (EU) 2017/746. In case the MDSW creates information based on data obtained by a vitro diagnostic medical device or if information provided is based on data, obtained solely from in vitro diagnostic medical devices, then it is an IVD MDSW. Software that takes raw data outputs and uses clinical algorithms for diagnostic and/or prognostic purposes also qualifies as a IVD MDSW. The IVDR requires that all existing and new IVDs are (re)classified based on a risk-based classification system [27]. Finally, the risk class may be considered that is applicable. This categorizes the risk of software, based on the combination of the significance of the information provided by the software to the healthcare decision and the healthcare situation or patient condition. We may for instance consider, that the software is used in combination with a medical device, that provides information on the statistical predisposition of Down syndrome (Trisomy 21), may be classified as Class IIb, since it may lead to an irreversible deterioration of a person’s state of health. If the software is used to determine the etiology of intellectual disability and developmental delay to inform neurologists, geneticists, pediatricians, and patients’ families, then the class would likely be Class IIa. The IVD class influences the requirements and assessment routes, and the required development process documentation needed to demonstrate that all requirements have been fulfilled, as for instance indicated by IEC 62304. A supporting approach to the documentation process that may be interesting for software development is Good Automated Manufacturing Practice (GAMP), GAMP 5, which is a science-based approach to understanding and managing risk for computerized systems that resembles the International Electrotechnical Commission (IEC), IEC 62304. These standards both suggest that for Class IIb and Class C devices should have detailed software design documentation while this is not necessary for Class IIa and Class B devices. Class I and Class A devices on the other hand only require planning, setup of user requirements and validation of those requirements and do not require architectural design specification, unit testing and integration testing, which is necessary for Class IIa and Class B devices and up.

If we want to make software for a product that is currently Research Use Only (RUO) oriented but as a future product has the intend to be a medical device, then one may argue that prudence entails both long-term preparation and short-term, goal-oriented planning. Quality assurance comes from due diligence in all stages of production. As such, we may choose to take up a higher classification initially as it will be easier to come down later, rather than to first built a high mountain and then attempting to climb it. Such decisions are however highly dependent on purpose, context, meaning, organizational structure, availability, priorities, and support by a wide range of members of the data management team, stakeholders, business users and business executives. If a team prefers to build a tool that will never be used for a medical purpose, then extensive documentation may be superfluous. Problems with documentation may become a complication that brings frustration and struggle. While documentation and code serve different goals, without documentation code is not a full-fledged product. Choosing an approach that is well thought through and supported through the whole line of development is therefore essential to the success of any project.

3.3 Innovative diagnostics

The IVDR recognizes the need for different types of products by admission of the clinical need for laboratory developed tests, In House IVD tests (IH-IVD), modified CE-IVDs, RUO tests, and off-label use, if their benefit for patients is obvious and can be proven. IH-IVD can be used by diagnostic laboratories if they meet several conditions and requirement as specificized by IVDR article 5.5. The IVDR thus explicitly states, that its requirement does not apply to IH-IVD, with exception of the relevant General Safety and Performance Requirements (GSPR) if several conditions are met [27]. Health institutions must justify that the specific needs of a group of patients cannot be met, at a desired level of performance by an equivalent device on the market that is a CE-IVD. Judgment of equivalence is up to health institutions, and they should have documented justification for their use of IH-IVDs, competent authorities only oversee compliance of health institutions notified bodies with respect to article 5.5. Justification, however, is a continuous responsibility and thus requires laboratories that have implemented IH-IVD to regularly monitor and evaluate new CE-IVDs. This extra work on its own, may be an incentive for replacing or marking IH-IVD through the industry. How equivalence is exactly determined, and what is expected in terms of comparison of performance, is open to speculation. In the absence of exact specifications what needs to be done, we may get started without delay. Epistemologically, for abstract ideas and beliefs to become more concrete, experience with the matter is necessary, which can only be obtained by doing the actual work. The implementation of the regulations may thus follow a similar iterative process where the final interpretation may only become clear while the law is applied. Innovative diagnostics are therefore typically expected to be first developed and applied locally, in healthcare institutions with specific expertise and then taken over by IVD manufacturers that produce above certain volumes, to make the tests cost effective.

When considering MLPA in its early development, we may see that this technique was first introduced in 2002 by MRC Holland as a research technique which then, among others obtained validity as a classifier [28] and as a technique to investigate copy number aberrations [29]. Through the years of usage and by experience via many validation studies it was able to confirm its performance. The IVDR also mentions that: “if a device is in a performance evaluation phase, it can be made available to institutions or laboratories to be subjected to one or more evaluations studies, with the intend to gather information on performance evaluation parameters” as mentioned in Annex I of the directive. This information can then be used for its conformity assessment. This would require the manufacturer to draw up a statement that the device conforms to the requirements of the directive, apart from those specifically itemized in the statement and that every precaution has been taken to protect the health and safety of the patient, users, and other persons. When IH-IVDs developed and used by the academic diagnostic sector make it into commercialization, other problems may arise, such as lack of interest by manufacturers due to economic viability and/or access patient-derived material for controls. In-house devices shall not be transferred to another legal entity, which thus entails that health institutions that manufacture in-house devices may only use those device within that same legal entity. Since a legal entity may be a single hospital or several hospitals that are part of one health institution, collective development of IH-IVDs may greatly reduce regulatory burden, as well as increasing access patient-derived material for controls. This may in turn lead to improvement of quality and standardization of tests. Access to patient-derived material may be one of the main factors preventing transfer of IH-IVDs to CE-IVDs, especially for rare disease and why it is nearly impossible for manufacturers to directly develop CE-IVD tests. Other factors include complexity, suitability for automation, algorithm dependence as well as a variety of national diagnostic and reimbursement practices that affect economic viability [23].

3.4 digitalMLPA RUO tests

Products designated for RUO are exempt from MDR and IVDR regulations, and if used solely for research purposes, they are not considered in-house devices. Common genetic disease research has already established knowledge on copy number variations using methods like conventional MLPA, array Comparative Genomic Hybridization Arrays (CGH), Single Nucleotide Polymorphism (SNP), SNP genotyping, and next-generation sequencing [30]. RUO products developed for research purposes typically fall into the category of general laboratory use, serving as elements in diagnostic testing plans to assess their potential future use in IVDs. digitalMLPA, as a screening technique, offers advantages such as replacing multiple MLPA assays with a single test, reducing costs and turnaround times, and targeting specific genomic targets without extensive NGS kit requirements. It ensures deliberate and purposeful testing to reduce ambiguity surrounding patient problems and prevents misleading disease associations. With its simplicity, low cost, target specificity, rapid detection of CNAs in up to 1000 target sequences, digitalMLPA is well-suited for diagnostic purposes. Furthermore, once development and quality control processes are established, new digitalMLPA products can be easily developed, including targeting niche diseases overlooked by commercial companies.

digitalMLPA data results cannot be interpret without specialized software, that can import FASTQ files and convert the sequence data into understandable results, that include meaning and significance for a variety of context dependent situations. MRC Holland, as the internal user, needs software for data analysis and quality assessments of digitalMLPA products and are therefore an important group of target user for which the software should be developed. We may expect that digitalMLPA will target areas for investigative research, such as the usage of copy number data for genomic profiling and their correlation to medicine effectiveness. The potential for molecular-guided therapy to affect the treatment of an individual patient has been recognized [31] and digitalMLPA may be a product that may aid in recognition of patients where such therapies may be beneficial. Especially in understanding cancer development and progression, next-generation sequencing has played an important role, and therapies that target a cancer’s genomic dependencies has fueled the transition of genomic assays into clinical use [32]. Users of RUO products may therefore be, an important second line of users. While these products may initially be developed and used to investigate specific purposes, through usage and experience they may end up as IH-IVDs or as part of an IH-IVD device. We may also have users that already use conventional MLPA as a diagnostic tool or users that are in search of diagnostic tools and are evaluating how NGS devices may aid in lowering costs and improve turnover times. Typically, much of scientific software development, is carried out by scientists who create programs to support their investigations [33]. When software development is not done by scientists themselves, establishing requirements may be difficult considering that the goals in question are potentially unknown. In such cases it is particularly important to gather understanding of the environment, practices, technologies and wishes that potential and future users may have.

3.5 User surveys/pre-market research

Understanding of the user environment of potential customers is an essential step in an early phase of development since it may for instance lead to constraints to the resources a software program may depend upon. Software for digitalMLPA will need to work with FASTQ files which may very large files up to 100 Gb. If 8 GB is the standard amount of RAM for your average desktop computer in 2022, special operative processing capabilities will be necessary to support analysis of raw digitalMLPA data on an average computer system. These early requirements form the base for the operative and system requirements that need to be established early in development, as they may influence all secondary choices and design specifications. Failure in compliance to such requirements may lead to the development of software that either needs users to greatly invest in computer systems, or simply leads them to search into alternative choices. When we are dealing with a wide range of different user types, we also need to consider that different users may have different availability to computers systems. It is then thus imperative to design the software to support the group of users that have the lowest operative capabilities, though scalability is also a factor.

Another point that needs to be identified early in development is the mode of delivery of the software and the required configurability. Software for digitalMLPA eventually will preferentially be able to support all MLPA products over a longer period. Since digitalMLPA products are still under development, the exact requirements for the software are still changing. We may thus for instance recognize, that internal users at MRC Holland may have specific demands and want to use the software for quality control purposes. They also want to have an easy way to adjust and update the configuration, so its content matches the actual digitalMLPA products. While digitalMLPA product developers want flexibility in the software, users in a diagnostic laboratory often prefer products, software, and its configurations to be stable. This allows them to validate a technique which may then be used as an IH-IVD or part of an IH-IVD. Adjustments in software or its configuration will often an incentive for them to reperform the validation, completely or partially.

MRC Holland’s internal users, who are primarily focused on researching digitalMLPA product development, can be considered as RUO users. Their preferred output should support adaptation, filtering, and manipulation, enabling comparison of results over longer periods. Unlike field users who value stable products and changing samples, Internal users anticipate that the digitalMLPA products will change while the samples remain relatively constant. RUO users typically handle larger sample groups simultaneously and require a data presentation format conducive to integrative review. On the other hand, prospective users in the diagnostic field concentrate on individual patients and prioritize extensive quality control. Eventually, diagnostic field users may prefer CE-IVD products and software due to the added burden on clinicians when using RUO products in an IH-IVD setting. To develop software that caters to all user groups, specific requirements from regulatory affairs must also be considered. For example, traceability of patient reports may be necessary in a diagnostic setting, while results should remain unalterable to ensure data integrity. Addressing these contrasting demands calls for a flexible framework with extensive configurability, along with modes that manage risks by limiting adaptability for diagnostic-focused users.

We may furthermore need to recognize that the success of the evaluation of RUO product by users influences the progressive move of those products into IH-IVDs, which may then stimulate the production of CE-IVD products. The transfer of research into diagnostics is also more likely to occur within institutions as a consequence of trust rather than availability alone. Within diagnostics we may also have different groups of target users. A small institute for instance, is not expected to have much automatization and is more likely to be interested in software that can be operated using a Graphical User Interface (GUI). A single user of a digitalMLPA kit will likely execute the MLPA procedure as well as analysis of the data. A large service lab on the other hand that performs hundreds of analyzes each day will probably have a specialized department for the analysis of data and automated pipelines that revolve around Laboratory Information Management System (LIMS). This type of user is unlikely to be interested in GUI applications and will rather integrate a technique and related software components into their existing pipelines. To be able to support these kinds of features specialized software architectures are necessary. Early in development, conceptual, logical, and physical data models need to be created in a sequential process that involves all members of the data management team and business users. Agreement on what is needed is essential to establish, otherwise, the data models may not fully capture the business context of data or meet an organization’s information needs. Conceptual modeling may thus be an important part to get to a point of understanding of what is needed before building commences. Knowledge and understanding of a complex usage environment may result to the process of creating a more usable and useful tool. Another important part that needs to be considered early in development is the method of documentation, which may be the results of the risk classification, but proper documentation may also lead to transparency regarding evidence about its performance and algorithms. Choosing a fitting strategy early in development may make the development of software easier, as well as testing and integration.

3.6 Iterative behavior

To ensure a comprehensive and effective approach to software development, it is crucial to recognize and adapt to the varying needs of different elements throughout the development process. As depicted in Figure 1, attention to these elements should be iterative and correspond to specific stages of development.

Figure 1.

Elements that are expected to return on a cyclical basis.

By making well-considered decisions and obtaining team agreement on goals and timelines, significant time can be saved, and disappointment and frustration can be avoided.

It is important to acknowledge that these elements are interconnected and that the software’s scope may evolve over time as more information becomes available about the goals. Therefore, making decisive choices regarding the prioritization of development components and understanding their mutual influences will significantly impact factors such as cost, agility, simplicity, scalability, fault tolerance, performance, and extensibility. In this context, the creation of data models and workbenches assumes a crucial role in the development process. They provide valuable insights into the possibilities and often inspire early users to envision software implementation approaches. Consequently, these investigations may lead to the identification of new requirements that will influence the establishment of the software architecture. Overall, by recognizing the dynamic nature of software development, adapting to changing requirements, and leveraging tools such as data models and workbenches, developers can ensure a more informed and effective development process.

3.7 Software architecture

The architecture style chosen for an application plays a crucial role in determining its characteristics and behavior. Certain styles are particularly well-suited for highly scalable systems, which are often necessary for accommodating a large volume of users, as seen in service labs. On the other hand, some architecture styles enable developers to respond quickly to changes in requirements. Each style possesses its own strengths and weaknesses, and selecting the appropriate one that aligns with specific business needs and goals will greatly impact the subsequent stages of application development, as well as its extensibility and usability. Prior to embarking on significant software projects, companies or institutes intending to develop software may find it beneficial to initiate modeling projects. During this phase, data modelers or architects interview stakeholders to gather comprehensive requirements about the processes involved. Business analysts may also contribute to designing conceptual and logical models, which can subsequently be employed to convey technical requirements to designers. Skipping these crucial steps can result in implemented software architectures and data models that fail to fully capture the organization’s and users’ needs. In recent years, the field of software architecture has witnessed significant advancements, leading to the emergence of new styles such as microservices and domain-driven design. Architecture styles provide an overview of the system’s macro structure, while patterns offer structural building blocks that can be utilized within architecture styles to address specific problems. Well-designed patterns have the potential for reuse in applications and company infrastructure. This section aims to explore various considerations in architectural styles during the early stages of software development for digitalMLPA, with the objective of meeting project business needs. The choices made in architectural styles have far-reaching consequences, impacting the overall development process, including quality assurance and the pathway to certification.

3.8 Requirement drivers

3.8.1 Useable software without installation

Often some desired functionality will be the main drivers in choosing software architecture. When starting development of software for digitalMLPA it was seen as an advantage to create the opportunity for users to be able to use the computer that is used to run the NGS system. These computers are often powerful systems that could make the conversion of FASTQ files much faster as opposed to using their personal computers. However, users in institutes will often not have installation rights on such systems, as such one of the main requirements was to develop software that could be used without the need of installation.

3.8.2 Data sharing

The software system supporting digitalMLPA experiments requires the capability to handle data sharing among multiple user groups. To optimize costs, users can utilize a central facility for NGS runs, enabling the merging of digitalMLPA experiment products for sequencing. Consequently, the centralized unit must be capable of redistributing specific experiment-related portions of the data from a FASTQ file. To fulfill this requirement, the software should employ a file-based data storage system utilizing a customized format that allows for data updates throughout each phase of the analysis. In this setup, a FASTQ file can be divided into separate files containing data specific to individual samples. These files can then be grouped together within separate folders associated with each experiment, facilitating redistribution to the respective experiment owners. For example, on a MiSeq device from Illumina, it is possible to combine approximately 64 reactions for probemixes with around 600 probes, with single reads of 110 nucleotides or longer. On a Nextseq device, it is feasible to combine 192 reactions for probemixes with a similar probe count. Combining data in this manner can lead to significant cost reductions in NGS runs by minimizing the number of plates and reagents required. Automation, when coupled with data combination, further enhances sample processing efficiency while reducing variation and waste. While file-based data systems may have certain disadvantages, such as potential limitations in performance and scalability, they offer notable advantages in terms of data retrieval, backup, and sharing. The simplicity and efficiency of file-based systems make them an attractive option, eliminating the need for installing third-party components. Overall, the software’s file-based data storage system, coupled with the ability to merge and redistribute specific experiment data, enhances collaboration and cost-effectiveness in digitalMLPA experiments, without the complexity of additional software installations.

3.8.3 Reusing of components

In the development of components for digitalMLPA analysis, it is worth considering their potential for reuse in future software programs, including updates to the analysis software for conventional MLPA data [34]. By accurately identifying the components that can be shared between conventional and digitalMLPA and designing them in a way that allows their availability as framework components within the company’s infrastructure, economic benefits can be realized. By reusing validated and verified components, a significant portion of the work required for certification may already be completed even before the entire solution is finished. This approach not only saves time and effort but also improves the overall quality by standardizing procedures that are recognized as part of the general functionality for MLPA. For instance, techniques like normalization and population statistics are commonly employed in MLPA, and their standardization through the reuse of components can enhance quality and reliability. Moreover, by establishing a framework of reusable components, the company can leverage this infrastructure for other software programs and projects in the future. This approach promotes efficiency and consistency across different applications, reducing redundant development efforts and fostering a more streamlined development process. In summary, by identifying and designing components for digitalMLPA analysis in a way that enables their reuse and incorporation into a framework, economic benefits can be achieved. Reusing validated components saves time, effort, and potentially contributes to meeting certification requirements. Furthermore, standardizing common procedures enhances quality and establishes a foundation for future software programs within the company.

3.8.4 Continuous integration/continuous delivery

Continuous integration and continuous delivery (CI/CD) are important aspects of software development, especially when integrating software into an organization. In scientific software development, the end goal is often uncertain. A monolithic software architecture, which consists of a single deployment unit, may be cost-effective and easier to design and implement initially. However, it lacks scalability, fault tolerance, and elasticity. Domain-Driven Design (DDD) offers an alternative approach, focusing on the function of the domain rather than the workflow or technical components (Figure 2).

Figure 2.

Domain structures software architecture.

In DDD, all functionality related to a specific domain is grouped together, allowing changes within that domain to be self-contained within a specific area of the system. Microsoft has been promoting their version of domain architecture, which is based on developing microservices-based applications and managing them using containers. Microservices, as single-purpose, separately deployed units of software, excel in scalability, agility, and fault tolerance. They also facilitate risk management due to their focused functionality. However, implementing and maintaining microservices in an environment with rapidly changing data sources and extensive database content across multiple functionalities can be challenging. Domain-driven design offers advantages in validation and verification testing, as separate components have well-defined inputs and outputs, enabling quality control and integrity testing. These components can be easily integrated into automated pipelines. DDD also allows close collaboration between development teams and domain experts, focusing on specific key parts of the system. Over time, the individuals involved in development may change, depending on the scope of the development cycle, which can limit team sizes while enabling specific testing and a focused approach. DDD also facilitates the gradual integration of components into the organization for evaluation purposes while development on other components continues. This iterative approach allows for feedback and evaluation at different stages, ensuring a smoother integration process. CI/CD is crucial for software development, and DDD offers a beneficial approach that emphasizes domain functionality. Microservices-based architectures provide scalability and fault tolerance, although they can be challenging to implement and maintain. DDD enables efficient validation and verification testing, close collaboration with domain experts, and gradual integration of components into the organization for evaluation.

3.8.5 Separation of concerns (design)

Separation of concerns in software design, particularly in layered architecture styles, ensures that each component within a specific layer focuses solely on the logic relevant to that layer. Each layer has its own role and responsibility in the application, which enhances its effectiveness and testability. For example, the presentation layer handles user interface and communication logic, while the business layer applies specific rules associated with the application’s domain. Each layer forms an abstraction around a particular request, isolating its functionality from the other layers. The advantage of closed layers is that they provide independence to each layer, ensuring that updates or changes made in one layer do not affect the others. This isolation makes risk management easier and allows for specific scoping of development cycles or sprints, as changes are contained within isolated parts of the code. Understanding the impact of these changes enables better definition of associated risks and improves quality assurance. Additionally, the separation of layers enables the design of presentation layers to be deferred to a later stage while other components are being tested and integrated into the organization. In scientific software development, where functionality often takes precedence over usability, separating end-user presentation design features such as GUI and reporting options from the analysis modules proves beneficial. Analysis modules produce visual output primarily for verification purposes, allowing their development, documentation, and testing to be carried out independently, possibly even by separate groups. The output design can be optimized for easy automated integrity testing, promoting stability, and simplifying quality control while other parts of the software are being developed. Back-end specialists can focus on developing the functionalities of analysis modules, while design specialists can handle front-end design and report generation. Usability is an essential aspect that often requires close interaction with the intended user groups to ensure that the software efficiently and effectively achieves the set goals. By separating presentation design features and analysis modules, software developers can engage with design specialists and users to create a user-friendly and goal-oriented application. Separation of concerns, particularly in layered architecture styles, improves testability, risk management, and quality assurance. Separating presentation design features from analysis modules allows for independent development, testing, and documentation. Design specialists can focus on usability aspects while back-end specialists work on functional modules, leading to a more efficient and effective software application.

3.8.6 Interfaces

Interface design may be a process more akin to an art than a science and requires a different skill set and possible several rounds of iterations with communication between different end user groups and the developers to reach a satisfactory result. Challenges involved in usage of interfaces are often of a “more subjective order.” The applications that a scientist works with are often less advanced, less secure and deliver an inferior user experience compared to the applications the same scientist uses on their smartphone after work [35]. Though a digital revolution is taking place biotechnology companies are slow to adapt and many companies still believe that software must be developed and maintained in-house, and that data is more secure on internal servers than on the cloud. Cloud services are often avoided being believed to be less secure than local storage which requires companies to spend both capital and operation expenses on in-house servers and in-house software development. By investing in components and codes that are encapsulated and independent, movement to a different platform in later stages may be made easier. Thus, when the overall organization has gained more trust in cloud services, much of the previous work can be easily transferred and documentation, testing procedures and risk assessment information are more readily available.

3.9 Life cycles

3.9.1 Principles of coding

1. Keep it simple: Write code that is easily understandable, self-explanatory, and divided into smaller, manageable pieces. Use clear variable names and separate bigger problems into smaller components.

2. Risk management: Prioritize patient and user safety by exercising risk management throughout all phases of development.

3. Epistemology: Invest in understanding the theory of knowledge especially regarding its methods, validity, and scope, and the distinction between justified belief and opinion and apply methods for testing and verifying conformance. Establish a shared nomenclature within the organization to ensure accurate descriptions and prevent confusion.

4. Research: Conduct thorough research before starting coding to identify user groups, their requirements, and constraints. Create separate components for evaluation and testing, allowing for early feedback and ease of reusability.

5. Meaning and context over content: Focus on implementing functionality that aligns with the intended use and minimizes patient risk. Emphasize the importance of context, intention, and meaning in the code.

6. Writing good requirements: Craft clear, unambiguous, testable, and correct requirements that serve as blueprints for the design. Good requirements facilitate understanding, minimize misunderstandings, and aid in creating acceptance criteria.

7. Tools development: Consider the development of tools that support various processes such as traceability, documentation, testing, configuration management, and deployment. Use available software tools to streamline testing and improve code quality.

8. Documentation: Maintain accurate and concise documentation tailored to the target audience. Keep code and documentation closely linked and updated together. Consider integrating code and documentation in the same file to enhance understanding.

9. Order of development: Start with macro structures and main functionality that can be easily validated. Then focus on implementing configurable options to meet specific user requirements. Finally, address user interfaces and user experience, which are subjective and require iterative development.

Adhering to these principles may promote effective communication, collaboration, and quality assurance throughout the software development process.

3.9.2 Agile

A life cycle is a process used to design, develop, and test high quality software. It consists of planning, requirement analysis, defining, designing, coding, testing, deployment, and maintenance. Agile approaches emphasize on continued collaboration and improvement. Tasks for a project are typically placed on boards to produce comprehensive overviews. The project is then divided into phases that each consist of planning, executing and evaluation. The advantage is that a large, complex project then consists of manageable chunks. Evaluation after each sprint improves collaboration and invites adjustments in the processes whenever this is necessary. As with the interpretation of the law, optimizations to processes are often recognized while executing them. Quality assurance departments recognize the need for corrective and preventive action but in practice only execute these after non-conformities are found. Though there is significant evidence that agile methods can be applied for developing software in the safety critical industry [36] and that this has significant benefits, organizations in highly regulated environments often lack suitability to accommodate fast response to changes. Processes may be dictated by standards rather than adapting methodologies to the company. The time and the energy that is needed to implement changes in processes is discouraging. So why would we still want to encourage agile methods? The obvious paradox is quality assurance itself.

Though the documentative culture caused by the need for compliancy to regulations and the fear for regulatory inspections [37] may prevent agile working, developing teams may still employ these methods to ensure the quality of the product. The idea is to create a process based on empirical logic that in the end fits into the defined logic of the company standards. This may create overhead for developers and lead to two track systems that then often requires syncing. Failure in doing so tends to make integration more difficult. The IVDR however is clear that we must attempt to take every action possible to minimize risks to the patient even if it means avoiding processes that are set in place due to that standard. Working agile in shorter sprints promotes quality because it limits the number of changes. If the software architecture has a strong domain-oriented architecture, it may prevent changes in one domain from affecting others. This focus minimizes the risks of introducing bugs because the scope of the changes is limited, and the effect are easily recognized. The scope of a cycle may for instance completely focus on interfaces, no changes in result data are then expected. In case discrepancies are found, they may be easily traced since the change set is limited and confined to a single domain. In comparison, if we choose to make a wide set of changes that affect analysis methods, configuration, and interfaces, then the complexity increases exponentially. The source of an issue may then be the result of changes in any of the domains or even be the result of unexpected interactions between these areas. Traceability may require rigorous investigation of possibly hundreds of changes to find the source of an error. Next to that, due to the increased length of a cycle, feedback from official testing may take such extensive periods of time that developers will need to invest time to get reacquainted with the topics at hand. Organization may also urge developers to continue working on a project before test results are produced. In those cases, changes may be placed on top of changes making traceability nearly impossible.

Shorter cycles may furthermore prevent scope creep, that is when new project requirements are added after the project execution has started. This has a major impact on project management and increases risks because these changes are often not properly reviewed, and developers are expected to complete the tasks with the same resources and in about the same time as the original scope. This in turn leads to stress and frustration, it crushes enthusiasm and invites quick-and-dirty programming. This tactic of programming leads to serious deficiencies also known as the technical debt. We want to obtain all objectives, but we do not have the resources needed so priorities are reassigned, and certain objectives are postponed. We may for instance assume that, like in traditional development, QA teams run their tests and code is expected to have bugs and that as such developers only need to ensure that the code runs. When developing software for a safety critical industry cutting corners should be avoided. If QA becomes agile as well, they can work closely with developers, and each group can focus on different aspects of testing. Harmony and balance may reduce work and stress for both groups leading to higher quality products. Limited cycles also make it more acceptable for new project requests to be executed in the next cycle of development. Another way to prevent scope creep is by defining the requirements for a project, making a schedule for their execution, and then verifying the scope with the stakeholders. In that way an agreement may be reached that is verified by team members in stakeholders, in practice we may notice that unless officially verified, agreements tend to be easily mended.

3.9.3 Software validation

An important part of ensuring patient safety is sound risk/hazard analysis. Also, in software development it is an essential and required process to logically process for risk management from risk identification through evaluation, rating, and mitigation. International Society for Pharmaceutical Engineering (ISPE), ISPE’s GAMP5 is a framework that provides a structured approach to validation that can be applied to agile development processes, as well as more traditional development patterns. GAMP is an acronym for Good Automated Manufacturing Practices, suggesting a scientific approach to enable the development of high-quality software that meet regulatory requirement and ensure patient safety. Standards such CE and FDA describe what needs to be done, but they do not describe how this must be accomplished. Protocols that are placed into effect by companies based on these standards often are translated one-to-one leading to inflexible systems that are document based on predefined logic that are inflexible to new implementations of context. The protocols are often also not defined by those who use them, leading to operational deficiency. Methods that are not adapted and made suitable for those who work with them, may lead to conflict, prevents innovation, decreases efficiency, and takes focus away from the core essential of the regulations: patient safety and product quality. The problem often lies with the absence of knowledge and experience with the specific matter at hand leading to demarking everything with the same priority and the idea that everything must be documented. The evaluation of risk to quality may instead be based on scientific knowledge and linked directly to the risk to the patient. In GAMP5 the level of effort, formality, and documentation of each process commensurate with the level of risk associated with that process. One could argue that this is the same as the benefit/risk analysis for all risks and the overall residual risk as required by the IVDR. GAMP5 has also been endorsed by the FDA as an acceptable approach to risk-based software development.

GAMP5 has a top-down approach that looks at processes before systems and operates from the notion that the risks associated with the computerized system cannot be greater that the risk associated with the processes it supports. When considering software for digitalMLPA, then the risk category for the software is thus directly linked to the digitalMLPA product that has the highest risk classification. When considering the relative risk level within the software, an initial assessment should be done that is based on an understanding of the processes, typically derived from the user requirements, design specifications, operating procedures, regulatory requirements, and functional areas as well as understanding of the behavior of the users. This information is then used to identify the specific functions that have impact on patient safety, product quality and data integrity. These functions can be identified as possible hazards and may require extra controls to minimize their harm. The rigor of the risk can be based on the impact of the function, note that no function can have a higher risk than its related process. Risk assessment can typically be done using the Failure Mode and Effects Analysis (FMEA) derived risk assessment processes, where severity and probability are set out against each other to obtain a risk class. The risk class is then plotted against detectability to obtain the priority class. High impact functions will often require mitigation, for instance by adding requirements for control systems, but they may also be warnings or constraints to limit behavior. Correct functional risk assessment requires a strong knowledge of the system, as well behavior of end users. It is also advisable to communicate mitigations with the final users to identify if the controls are suitable in practice. For example, a warning message that out of habit is directly closed by users without reading the information may not be a suitable control. Such behavior can only be discovered by direct contact with final users.

3.9.4 Implementation and testing

Based on the identification of the functions that have a certain severity and risk, tests may be developed to evaluate if the implementations meet expectations. For testing low level impact functions, we may suffice with validation of a user requirement. For digitalMLPA software we may want to validate that a user is able to produce a log report, such low impact requirement meets compliancy if they match normal business expectations. Medium impact level functions need consideration of failure mode and typically involve the creation test case scenarios. These test scenarios describe use cases and their expected outcomes. For digitalMLPA we may for instance want to have a test case scenario where a user has specified incorrectly which digitalMLPA product was used. The digitalMLPA software has a control that matches the set digitalMLPA product to the automatically recognized digitalMLPA product. The test in the implementations when executed by a user must then show that a user was able to correctly identify the problem when using the software. Such outcome may involve subjective reasoning, since a user need to identify that there is a problem and what that problem entails. This may thus lead to the need to install extra controls or adjustments to the chosen implementation. For high impact functions specific risk scenarios must be tested. For digitalMLPA software, we may for instance expect that there are probes in probe mixes that are used to detect DNA mutations that are used when considering the diagnosis of a patient. Note again that the highest risk class cannot be higher than the risk class of the software. Under the IVDR that would automatically indicate that use cases that involve diagnosis of a patient are only appropriate if that software was distributed and intended to be used with a digitalMLPA product that has an intended purpose to influence the diagnosis. High impact level functions should be rigorously tested to ensure that the appropriate response is generated. For digitalMLPA this may involve creating test scenarios that involve evaluation of the appropriate response when reference signals are absent for these mutation probes or what happens if the settings in the configuration are incorrect. Based on the outcome of the tests, controls may need to be applied to increase robustness. Controls and their test results should be retraceable to their original risks. Risk assessment is typically repeated periodically and in case the risk class changes, for instance due to the installment of a higher class digitalMLPA product in the configuration of the software, then appropriate measures and controls need to be installed that match the impact level. The higher the risk level of the function, the greater the controls that should be applied. For digitalMLPA software when analyzing a mutation probe that has the highest impact level, we may for instance set in place a requirement that to analyze that probe in a test sample, we need in that same experiment both a positive sample and a negative reference sample. Such samples may then be used to establish the expected outcome under the same circumstances allowing higher confidence when calling a result in an unknown test sample.

An alternative strategy to estimate the necessary level of testing and investigation of the impact level of each function is by using the V-models (Figure 3).

Figure 3.

Example of a product validation pattern.

In V-models often the depth is related to the risk class. As such low impact user requirement only need performance qualification. This usually entails requirement analysis, unit implementation and system testing (Class A – Level 1). Medium impact user requirements obtain functional specifications and perform both performance and operational qualification. Here design documentation is extended with architectural design and testing is extended with unit verification and integration testing (Class B – Level 2). High impact user requirements then also include the lowest layer of technical design specification and verification of these specifications (Class C – Level 3). Each step is followed by risk assessment, as well as evaluation of test results and installment of controls whenever needed.

It may be worth mentioning here that software designated for digitalMLPA cannot produce any data on its own, similarly a digitalMLPA product cannot produce useful output without specialized software. The product digitalMLPA, thus consist of the probe mix kit, the software and the service or support provided. A user requirement of a product may then for instance be the ability to detect certain types of Duchenne muscular dystrophy. Associated risks when performing such an analysis may be that incomplete denatured DNA can show patterns that resemble deletions of copy numbers. Control mechanisms may thus be set into place to measure incomplete DNA denaturation by adding specific probes that are targeting GC% rich areas that will show early signs of denaturation issues. The functional tests to ensure such a control system works appropriately could be established by calibration series for that product using samples with increased salt concentrations. These salt concentrations will make it incrementally harder for the DNA to denature. We then want the control system to pass off a warning long before entering the process of copy number estimation related to disease diagnosis. However, the only way to measure this feature is to have specific software that can fulfill the needs related to the specific control mechanism. In this case the software is thus in direct association with the user needs of the overall product and the validity of the control mechanism can only be tested by the combination of the two elements. Controls should always be traceable to their original identified risks, and residual risk evaluation must be performed after installment of the selected controls for all high-risk functions. We may thus recognize that due to the associative actions between a digitalMLPA product and its related software, there may be many different V-models and that the depth of these models may be multilayered. Such multilayered systems can be executed in stages that have their own specific models for evaluation. The required depth should however be associated with impact level which is always related to the overall risk level associated with the intended use of the product as specified by the manufacturer. digitalMLPA software should fulfill the data analysis needs of many digitalMLPA products that may have different demands depending on the type of product. To do this, the system does not alter its original functionalities but rather allows adaptation of values that are assigned to certain variables to fulfill the needs of different users in the context of their processes. The risk level of the software itself should thus be related to the risk level of the digitalMLPA product that has the highest risk level. Please note that even though digitalMLPA products may use the software as a type of control that exists on a single level for the digitalMLPA probe mix, that we would still recommend multilayers testing and installment of controls for the software itself.

3.9.5 Review and monitoring

Once software requirements are verified and controls are implemented, they require monitoring. Implemented controls can reduce testing efforts, eliminating the need for calibration series with different salt concentrations for each digitalMLPA probe mix once we confirm that the installed controls function effectively across multiple mixes. Assuming the control configuration remains unchanged, residual risk analysis should be conducted to ensure the adequacy of controls. Periodic evaluation is necessary to identify new, unrecognized risks and determine if recognized hazards, their estimated risk levels, and controls remain appropriate.

4. digitalMLPA software

The analysis of digitalMLPA data involves assessing signals representing the relative number of target sequences in DNA samples. Conventional MLPA utilizes fluorescent tags connected to PCR primers for signal measurement [1]. Capillary electrophoresis separates and measures relative fluorescent units of probe PCR products based on length. In digitalMLPA, NGS devices sequence the probe products [2]. Analysis of resulting FASTQ files involves counting reads associated with specific probes in samples. Barcode sequences correspond to samples, while target sequences identify the probes. Normalization compares read counts to reference values, providing probe ratio values for copy number estimation. Probes can be clustered and sorted to identify genomic patterns for disease recognition, prognosis, and correlation with treatment effectiveness in precision medicine research.

4.1 The analysis workflow

The proposed software for digitalMLPA organizes the analysis of FASTQ files into different domains with specific tasks (Figure 4).

Figure 4.

digitalMLPA analysis processes flow.

Each domain has its own executable subprogram, which can be accessed through a provided GUI that executes all steps sequentially. Alternatively, these subprograms can be integrated into existing pipelines and connected to LIMS. Incorporating LIMS can enhance automation, but it requires on-site experts familiar with the program’s input and output requirements. The analysis workflow begins with FASTQ conversion, which requires information about the NGS run (Coffalyser Definition File, CDF file) and a configuration containing digitalMLPA probe mix and barcode collection information. FASTQ conversion generates proprietary file formats for each sample barcode, serving as input for subsequent analysis stages. This allows users to demultiplex large FASTQ files at one location and continue experiment-specific analysis at a different location. In institutes where multiple groups share an NGS device, this behavior is often expected as each group may only be interested in a subset of the FASTQ file. FASTQ conversion can be automated through a tasked operation system that triggers the conversion when the FASTQ file is generated by the NGS device. As FASTQ conversion can be time-consuming, automation is beneficial. After conversion, the resulting sample files undergo fragment analysis, which performs sample-specific quality checks to determine their suitability for subsequent analysis steps. Several steps implement controls for higher processes, including NGS machine performance, DNA sample quality, sample identity, MLPA reaction quality, and context-related factors such as reference sample and probe quality factors. Quality control also ensures reliability of the overall results, aiding in quality evaluation when discrepancies occur. Following fragment analysis, the sample files are updated and used as input for data normalization, which applies specific instructions from the CDF file and configuration to normalize samples within the same experiment. After normalization, the sample files are updated again and serve as input for results analysis. This component sorts and clusters the results based on sample type for comparative statistical analysis, enabling determination of significant changes in sample probe results compared to reference collections. The results analysis facilitates confident result calling and customized recognition of genomic disease patterns. Results are stored in the sample files, which can generate various types of reports. The report format is highly configurable, allowing digitalMLPA probe mix designers to customize the products for customers. Each component produces debug files to facilitate validation and stability testing. This enables automated unit testing when updating a single component, ensuring stability in other components. This methodology supports quality assurance by facilitating estimation of the effects of changes within a single domain.

Domain-specific changes enable focused and specific verification, allowing scrutiny of individual software components while verifying the stability of others. This approach optimizes large software projects by integrating each part into the organization while other processes continue in parallel. Although domain-organized software may initially have a longer development path, it remains more agile throughout its lifetime. Documentation around quality assurance can benefit from this structure if organizations are prepared for this work style. Importantly, this structure lends itself well to a top-down risk approach that centralizes focus on the requirements of the highest processes, which may have an impact on patient safety. This approach allows for the implementation of dedicated controls to minimize risks effectively. The effectiveness of controls and regulations in minimizing risks to patient and user safety should be continuously verified through implementation and measurement in the field, encompassing the entire scope. These measurements are essential for establishing the effectiveness of controls and ensuring the usability of regulations.

In summary, the main requirements for digitalMLPA software involve the analysis of signals representing the relative number of target sequences in DNA samples, using NGS devices and FASTQ files. The analysis workflow encompasses several domains with specific tasks, allowing for automation, quality control, data normalization, and results analysis. The domain-specific approach facilitates focused verification and the integration of software components into the organization. Furthermore, this approach aligns with a top-down risk perspective, emphasizing patient safety and the implementation of effective controls. Continuous measurement and evaluation are crucial for assessing the effectiveness of controls and maintaining regulatory compliance.

4.2 Quality assurance when developing software

4.2.1 Initial development of digitalMLPA software

How do we approach quality assurance when developing software with unclear goals and intended purpose? Here are some considerations to guide us. The software’s classification depends on the intended purpose of the supported probe mix, which may require us to anticipate future usage scenarios. During software development, we can incorporate configurable functionality to cater to various product needs. These configurations become a separate domain, requiring thorough testing and evaluation alongside other product aspects. When determining the risk class, we have two options: assuming the highest possible risk class throughout the software’s lifetime or aligning it with the current risk level and upgrading when higher-risk products emerge. The choice depends on factors like project size, available resources, and documentation practices. Timely documentation, preferably done alongside coding, ensures synchronization and avoids outdated information. Identifying controls is often best achieved during method development. When the exact goal of the software is not completely defined, then we may search for guidance to quality assurance from the highest applicable authority. Authority comes from excellence, which may be defined by dedication to the highest standard. This may mean IVDR or FDA, or how its regulations are implemented in the organization, it may also involve some inner conviction and experience with the matter at hand. The main requirements of the company, such as MRC Holland’s goal to provide affordable and user-friendly products for detecting chromosomal aberrations, can shape the software’s objectives. Supporting input from measurement machinery and calculating MLPA probe ratios are key requirements. These ratios, in combination with specific MLPA products, assist in copy number estimation and disease diagnosis. The risk class depends on the potential impact on patients and its associated probability. To address uncertainty, initially assigning the highest risk class and gradually lowering it through control implementation and verification of validity can be pursued. Use cases that assess severity and probability help determine the initial risk class.

4.2.2 Use cases

4.2.2.1 User case one

Conventional MLPA as a technique focuses on the detection of genetic diseases, that are typically tested for when a phenotype is observed. The intended use may then be described as a function that is employed to know something that could lead to understanding of its genetic underpinnings. Genetic test kits could also be used to understand the probabilities of certain inherited genetic traits being passed on to offspring. For instance, we may have a scenario where a user has a genetic test result and the intended purpose of the probe mix is to provide understanding in underlying conditions of a patient, but not treatment. Positive (false) results may be expected to be confirmed by a second method (as stated per IFU), but even without that confirmation, the product is not expected to harm the patient directly. In case of false negatives, patients are typically expected to remain being monitored due to their already observed phenotype and to be subjected to alternative types of testing. The severity to the patient is therefore low as well as the probability to do harm, the risk class may therefore be expected to be a IIa risk class.

4.2.2.2 User case two

We may also consider a use case where product has the intention to test for genetic diseases on newborn. In this case, we may deal with specific syndromes, where an outcome may be used in a chain of a decision-making apparatus, that ultimately leads to application of medicine, for instance to delay progression of disease. In that case, a false negative may lead to harm to the patient, but not death. A false positive would still be expected to be confirmed, especially since the pharmaceutical industry is expected to adhere to higher risk classes under FDA or the IVDR. The severity in this case may also be expected to be higher than with the previous case since an extended delay influences the patient sovereignty. The initial risk class for such a product may be IIb. We need to keep in mind however, before such product obtains such an intended purpose, they usually must pass extensive rounds of testing by external users. The post market survey data from these tests are important to understand how products are implemented and what role they play on different locations. This data may result in changes to risk class as well as help to provide information about its performance in the field. Such data may be generated by allowing a new genetic test to be used for an extensive period alongside current implemented technologies or so-called gold standards. Information about its application, context of usage and performance in the field may then lead to better insights that can change the severity and probability leading to a lower or higher class. Developers are usually limited to verification and unit testing, that have their own importance, but do not grant validation of a devices multi operational implementation. To determine to the risk class of medical software we thus must adhere to the risk class of the known process that lies above its application. For software for digitalMLPA it’s the product itself. Processes above that may be the design and production of the product, but without data about its usage in the field, no verifiable validity can be determined. Verifiable validity may mean that the intended purpose has compliancy by operation and evaluation of its performance in the field. Please note that we do not intend to dictate an importance to a certain risk class here, merely to examine and invite discussion to the outcome of certain case scenarios.

4.2.2.3 User case three

If we would consider digitalMLPA in its current employment, we may state that its focus relates mostly to research in cancer genomics [2]. It’s very suitable to investigate the copy number changes that investigate specific genomic locations and combinations of changes that lead to effects in biochemical pathways. Such understanding can lead to better insight into prognosis and patient specific medicine. Software intended for research may be classified as a general laboratory purpose device and does not need to undergo external validation by notified bodies. However, if we also consider the commercial interest in such products, then there is also a need to develop trust with customers in the field that a product including its software version and configuration results in a stable element in a process they use to investigate cancer diagnostics. Considering commercial endeavor, then a positive outcome of a research trial may lead to the establishment of a diagnostic product for a manufacturer. This may be a screening test or a device that is part of standard operating procedures in clinical diagnosis. Data that may lead to insight into performance in the field also requires a stable supply of a product and its specific software and related configurations. The risk class that may be considered in this category under the IVDR may be IIb, since the deployment of the product may influence diagnosis related to cancer genetics. The obvious problem here is that data that may lead to insight to its implementation, does not exist before it is implemented. When developing software, we may thus better be prepared for the road ahead and invest in accuracy and stability to attempt conformity as best as possible to the company’s main requirements. To create software that may remain stable over periods of 10 years broader considerations need to be evaluated. We may for instance consider to the decisions I want to be accountable for, when that period has passed. We may then conclude that we should be prepared for its ultimate usage in practice. If we accept a high-risk classification early, we may have the option to choose a suitable documentation style that matches its purpose and is an accepted company standard. We could for instance choose to comment in such a way in code, that it leads to automated description of processes and/or understandable diagrams and workflows. It may be considered that using such data in combination with Artificial Intelligence (AI) linguistical models could amount to great potential in this area, as well as leading to an easier flow of data considering support to QA documentation and support material.

Note that risk class is often dependent on its verifiable validity of the software implementation in the world, and that there is a dependency on that data to establish and optimize controls. This however does not mean that we cannot attempt to mitigate risks, via case scenarios that lead to controls that allow adjustment of the probability. Controls also require testing and evaluation of these results to understand whether a mitigation has been properly implemented and that the final risk is acceptable. Sometimes we may also choose to mitigate risks with lower probabilities, because their detectability is high and easy to implement. Such situations are typically based on verifiable data and consider the detectability of a failure mode. We may also imagine failure of the highest possible processes that may influence the overall intended use and highest risk class to which applicable detection controls may be set in place. Risk priority is determined by the risk class combined with the detectability. That thus means that via the severity, it may be relevant to also work on low probability failures, due to availability of the mitigations. It’s not in the scope of this piece to consider all scenarios, but we may attempt to evaluate a few that may rank the high.

4.2.3 Use scenarios

Prioritization by combining detectability with the risk class allows focusing on implementation of controls to ensure its endured overall intended purpose, while considering safety to patient and user alike, in every step of the process. While the severity may be directly linked to the intended purpose of the product, the probability to different failure modes may be a different component altogether. Probabilities may be adjusted by increasing the detectability of failure of the main processes. As such, in discussion we adhere to the highest severity imaginable, within confines of the context at the time of implementation of the controls, to the highest processes that are available for the software to investigate or support. In this section we discuss some examples of failure modes that may exist in practice, and the types of control that have been implemented to mitigate risks.

4.2.3.1 Case scenario one

Failure of NGS machine measurement device. Failure mode: The NGS machine measurement device fails to obtain an accredited accuracy rate. Very low read quality results in changes in probe ratio, as reads are not recognized or assigned incorrect identities.

Severity: Complete failure of the device may lead to false positive results. Verification: The validity of the software’s methodology should be verified by provable specificity and sensitivity in real-world implementation. Logical reasoning and aiming for the highest degree of accuracy can also be employed. Specificity and sensitivity: Read recognition is a critical requirement of the software. With digitalMLPA, read identification is simplified as it only needs to match against a limited probe set. By implementing a method that distinguishes barcode sequences and links them to specific sample barcodes, probe mix identification is established, increasing specificity. Further segmentation and setting of thresholds enhance read recognition. Controls: Establishing control systems to measure data outside standard limits is important regardless of severity and probability. Controls for NGS data may include sequence quality, detection range, signal amplitude, sample signal bias, and more. New controls can be developed based on observed measurements, which may impact the probability of risk. Priority and detectability: NGS run quality and library preparation are prioritized due to their effect on outcome and high detectability. Improving NGS run quality can enhance the detectability of controls in later stages of data analysis. Rerunning digitalMLPA products can improve data quality without repeating the experiments. Adjusting probability and detectability: The effectiveness of implemented controls determines whether adjustments can be made in risk classification. Actual measurement under configured conditions is necessary to evaluate the impact of controls.

4.2.3.2 Case scenario two

Copy number estimation failure. Failure mode: High salt concentration in certain samples negatively affects the specificity to discern chromosomal aberrations from poorly denatured DNA, resulting in incomplete probe hybridization. This behavior may introduce patterns originating from chromosomal locations, increasing the probability of a false positive result. Severity: High salt concentration can mimic a false deletion pattern, leading to a high severity level. Probability: The strain on discerning copy number estimation via probe ratio and genomic pattern recognition increases due to the influence of high salt concentration, affecting the probability of accurate estimation. Detectability: digitalMLPA products incorporate smart controls using specially designed probes to test for different quality aspects, such as proper denaturation and DNA fragmentation, enhancing detectability. Priority: Supporting the requirement of proper denaturation and minimizing false positives becomes a high priority. It is prioritized due to the increased probability of false positives, the high detectability provided by the product, and its potential impact on later processes. The software should exclude such results from copy number estimation processes.

4.2.3.3 Case scenario three

MLPA reaction failure for copy number estimation. Failure mode: Poorly closed tubes in the MLPA reaction may result in sample volume evaporation, leading to changes in salt concentration that negatively affect probe hybridization completeness. Severity: Similar to Case Scenario Two, this scenario affects larger groups of probes and leads to poorer distinction of chromosomal aberration patterns, resulting in false positives. The severity is therefore high. Probability: Since this scenario can mimic genomic aberration patterns, there is a higher probability of an end user incorrectly calling a result aberrant while observing a false positive. Detectability: The digitalMLPA product includes specially designed control fragments that measure the extent of the hybridization reaction, enhancing detectability for identifying such issues. Priority: Given the high severity and probability of false positives, along with the high detectability facilitated by the product, addressing this scenario becomes the highest priority. The software should aim to prevent end users from attempting copy number evaluation on identified cases related to this failure mode.

However, the exact implementation of the control system’s effects cannot be solely attributed to the software itself. It should align with the intended purpose and risk class of the overall product. For example, a risk IIb class product should not produce probe ratio results but instead issue warnings and error reports to eliminate the chance of false positives. In contrast, if the software is distributed with digitalMLPA products intended for research purposes only, a less strict control system may be applied, and end results could include warnings. The behavior and effectiveness of these control systems can be observed in practice, providing insights on how to improve the accuracy of the product to achieve its intended purpose. Adjustments to the control systems can be made based on their effectiveness, ultimately enhancing the overall product’s accuracy through harmonious implementation of all processes and control systems.

4.2.3.4 Case scenario four

Controls on software processes for compliance and measurement. In this case scenario, we focus on implementing controls on the main software processes used to establish compliance with company design, specifically related to signal measurement and probe ratio estimation. It is important to recognize and prioritize these processes based on their influence and significance in achieving the intended purpose of the product. Failure mode: One possible failure mode is when part of an experiment becomes unreliable due to structural variations, such as insufficient master mix during the MLPA procedure. This failure can introduce changes in the biochemistry, leading to aberrant patterns and false positives. Severity: Such changes may lead to aberrant patterns and thus false positives. The severity of this failure mode is determined by the severity of the product and its specific configuration. It is important to address this issue to prevent misleading results. Probability: The probability of this failure mode occurring may be considered low since laboratory staff is trained to avoid such situations, and the usage of multichannel pipettes is recommended. Detectability: To establish control systems for such scenarios, specific recommendations can be provided through the information for usage (IFU) that consider different experimental contexts. For example, recommending the inclusion of reference samples at the start, end, and intermediate positions of an experiment sample population. The software can then measure the variation over these reference samples, although the effectiveness of this control may be context-dependent and influenced by assumed limitations of higher processes. Priority and effectiveness: The priority of implementing control systems should be determined by the severity and detectability of the failure mode, with a focus on higher processes that have a greater influence. While the potential effectiveness of measuring the variation over reference data can help prevent false positives, its actual value can only be established through field experience and data analysis. Acceptance criteria and definable logic can be implemented in the software to prevent false positives, and adjustments can be made based on data in the field to improve the control’s effectiveness.

Harmonization and collaboration: The effectiveness of control systems relies on the harmonization of all underlying processes and collaboration between departments, manufacturers, and users. Sharing data and experiences can lead to improvements in control systems. The increased focus on Post-Market Surveillance (PMS) in the IVDR is understandable considering the importance of ongoing improvement. It is important to note that these examples do not cover all possible failure modes but provide an examination of a few scenarios and their hierarchical order. Controls can be established based on machine manufacturer guidelines or by adhering to controls designed by the digitalMLPA product itself. A top-down risk-based approach should prioritize implementing controls for higher processes with higher probability and detectability. The severity of the software processes should align with the severity level of the product, and comprehensive testing of the product, including software, configuration, and support material, is necessary to ensure conformity to the intended purpose. By implementing appropriate control systems, such as evaluating variation in reference data for copy number estimation in MLPA, the accuracy of probe ratio measurements can be improved. Strict control criteria should be initially researched for effectiveness, and adjustments should be based on empirical logic and observable data. Stricter controls may be applied for specific probes or arbitrary borders can be tightened to reduce false negatives. However, it is essential to always prioritize the effectiveness of controls on higher processes over lower processes.

5. Discussion

When working on scientific software, encountering the new IVDR regulations can initially seem rigid and defining, while the implementation regarding patient safety and effectiveness may appear vague and transitory. Standards are often based on existing devices and knowledge, and while they set limitations to minimize outcomes, they rely on comparing and examining observable data. However, even with definitions in place, the practical implementation of patient safety measures can be challenging.

There seems to be a correlation between the focus on patient safety and the level of connection between the provider and the recipient of care. Direct patient contact allows tangible correlations, but in other cases, the ethical implications remain a question. It’s important to reflect on how our day-to-day actions and decisions influence patient safety, regardless of our level of direct interaction.

Standardization has its advantages, such as uniform results, lower product costs, and predictability. However, it can also invite commercial interests and potentially lead to monopolization. Conformity to standards can become a goal, conflicting with personal integrity, and potentially causing resignation. In the context of patient care, caregivers should prioritize providing care rather than simply conforming to definitions. In diagnostics, focusing on delimitation of factors of categorization may be a strength leading to a more accurate result. In research this limitation may result in a blockage to discovery, that may lead to innovation.

While it may be easy to differentiate between these two categories in theory, the status of products and their implementations can be more fluid in practice. Implementing regulations in a way that ensures patient safety often lacks harmonization and may create discord and separation. Harmonization is crucial for effectively achieving the intended purpose of medical products while maintaining the highest standards, which should ultimately serve the well-being of the patient. To achieve this, there needs to be a personal understanding of how regulations translate into improved patient safety in day-to-day actions and decisions. In practice such an approach may be admirable, but vain.

A top-down risk approach therefore prioritizes the highest processes and their intended purpose. For companies, improving patient care should be aligned with the goals of the IVDR. It is therefore surprising that the EU does not invest more in improving communication between medical institutions and manufacturers to exchange information on standards’ implementation and their effectiveness. They are in a prime position to advise on implementation since they can measure the effects of standards and could consider endorsing a legal entity for products and their users. Such collaboration would enable the placement of patient safety controls at a high-level process and allow PMS to focus on measuring product accuracy and the effectiveness of implemented controls.

Empiric logic, however, does require a vantage point. Lawmaking on the other hand often seems necessary and is often met with deadlines. In an urge to control content there may be a drive to document everything. It could also be so that the extensive nature of the regulations invites to make a set of procedures that are of equal size making sure that every delimiting box is checked to showcase conformity to the standard. Enforcing controls to promote patient safety is great but we need to take an active stand, taking interest to measure its accuracy and effectiveness in practice.

The extensive nature of regulations may tempt the creation of equally extensive procedures to ensure conformity, potentially overshadowing the goal of promoting patient safety. It is essential to take an active stance, measuring the accuracy and effectiveness of enforced controls in practice.

The extent to which the IVDR will be implemented and actively practiced remains unknown. In this piece, we discussed how applying such standards to software development in the intersection of research and diagnostics can have implications. We explored how software architecture choices contribute to quality assurance and how domain-based programming can establish quality control points. While we can attempt to safeguard quality through predefined logic and controls, the proof of a product’s intended purpose can only be measured through discernable observations. Methods and controls should be subjected to empirical logic and adjusted based on newly obtained information to improve accuracy. However, it’s important to recognize that harmonious implementation of products and controls often comes with its own set of limitations. Using a top-down risk-based approach, we can prioritize and establish hierarchical logic to ensure quality. These controls remain subject to change if they stimulate the accuracy of the intended purpose or increase the detectability of failure modes. However, for the sake of patient safety, we must acknowledge our limitations and recognize that processes and context beyond our own, offer higher value in effectiveness but require harmonious implementation.

Acknowledgments

We thank MRC Holland for its support and goal to provide affordable and user-friendly products for detecting chromosomal aberrations as well as for its ongoing support to the MRC Holland Foundation with the aim to contribute to the development of a better world.

Notes/thanks/other declarations

Thanks to Jan Schouten for his ongoing support in the medical and science world and providing a pleasant a comfortable working environment. Also thanks the MRC Holland staff for aiding in development, providing feedback and support. A special note to all referenced authors, their valuable work provided insights that helped to try to make sense of it all, without that it was not possible. Thanks to Joost van den Berg for many years of excellent programming work and providing understanding into the world of coding. Very grateful for Lydia de Burger, her unwavering support and unending understanding have made a great difference. If any references were left out, sincere apologies, this work was intended to summarize and attempt to provide some understanding and describe different considerations we went through when developing scientific software. This is also intended to invite discussion, share experiences to implementation of regulations to move forward.