Draft Guidance for Industry and Food and Drug Administration Staff

Highly Multiplexed Microbiological/Medical Countermeasure In Vitro Nucleic Acid Based Diagnostic Devices

This draft guidance, when finalized, will represent the Food and Drug Administration’s (FDA’s) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.

1. Introduction

FDA is issuing this draft guidance to provide industry and agency staff with recommendations for studies to establish the analytical and clinical performance of highly multiplexed microbiological/medical countermeasure in vitro nucleic acid based diagnostic devices (hereafter referred to as HMMDs) intended to simultaneously detect and identify multiple pathogen nucleic acids extracted from a single appropriate human specimen or culture. For the purposes of this draft guidance document, the multiplex level that is used to define HMMDs is the capability to detect â‰¥20 different organisms/targets, in a single reaction, using a nucleic acid-based technology and involves testing multiple targets through a common process of specimen preparation, amplification and/or detection, and result interpretation. HMMDs are used to aid in the diagnosis of infection.

FDAâ€™s guidance documents, including this draft guidance, do not establish legally enforceable responsibilities. Instead, guidance documents describe the Agencyâ€™s current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidance documents means that something is suggested or recommended, but not required.

2. Background

This document recommends studies for establishing the performance characteristics of HMMDs. FDA considers these recommended studies to be relevant for premarket notifications (e.g., 510(k) or de novo) that may be required for a particular device.

A manufacturer who intends to market a device of this generic type must, in addition to any other applicable requirements,

conform to the general controls of the Federal Food, Drug, and Cosmetic Act (the FDC Act), including the premarket notification requirements described in 21 CFR 807 Subpart E, and
obtain premarket clearance or de novo classification prior to marketing the device (sections 510(k), 513, 515 of the FDC Act; 21 U.S.C. 360(k), 360c, 360e.

This document is intended to supplement 21 CFR 807.87 (information required in a premarket notification) and other FDA resources such as â€œPremarket Notification 510(k)â€¹.

Guidance on the content and format for abbreviated and traditional 510(k)s can be found in the guidance entitled â€œFormat for Traditional and Abbreviated 510(k)s.â€²

Information regarding the use of standards can be found in section 514(c) of the FDC Act (21 U.S.C. 360d(c), and in the FDA guidance entitled â€œUse of Standards in Substantial Equivalence Determinations.â€³ FDA recommends that developers of HMMDs use this draft guidance for information on FDAâ€™s current thoughts regarding the regulation of these devices.

4 In addition, if you plan to seek clearance or approval for a microbiology multiplexed device that detects less than 20 organisms/targets, please contact the Agency prior to undertaking any clinical or analytical validation studies to discuss whether the recommendations found within this draft guidance may be applicable.

5

By contrast, if your device requires the use of 95% ethanol, and any type of 95% ethanol will allow your device to achieve the performance characteristics provided in your labeling, then 95% ethanol is a general ancillary reagent for the purposes of this document.

If the instructions for use of your device specify one or more specific ancillary reagents, you should address how you will ensure that the results of testing with your device and these specific ancillary reagents, in accordance with your instructions, will be consistent with the performance established in your premarket submission. Your plan may include application of quality systems approaches, product labeling, and other measures.

Manufacturers should address the elements described in detail below. FDA will evaluate whether your plan will help to mitigate the risks presented by the device to offer reasonable assurance of the safety and effectiveness of the device and establish its substantial equivalence.
Â

You should include in your 510(k) a risk assessment addressing the use of specific ancillary reagents, including risks associated with management of reagent quality and variability, risks associated with inconsistency between instructions for use provided directly with the specific ancillary reagent and those supplied by you with your device, and any other issues that could present a risk of obtaining incorrect results with your device.

Using your risk assessment as a basis for applicability, you should describe in your 510(k) how you intend to mitigate risks through implementation of any necessary controls over ancillary reagents. These may include, where applicable:

User labeling to assure appropriate use of ancillary reagents.

Plans for assessing user compliance with labeling instructions regarding specific ancillary reagents.

Plans for alerting users in the event of an issue involving specific ancillary reagents that would impact the performance of the HMMD.

Material specifications for specific ancillary reagents.

Identification of reagent lots that will allow appropriate performance of your device.

Stability testing.

Complaint handling.

Corrective and preventive actions.

Any other issues that must be addressed in order to assure safe and effective use of your test in combination with named ancillary reagents, in accordance with your deviceâ€™s instructions for use.

In addition, you should provide testing data to establish that the quality controls you supply or recommend are adequate to detect performance or stability problems with the specific ancillary reagents.

If you have questions regarding identification, use, or control of specific ancillary reagents, you should contact FDA for advice.

6

ii. Fresh vs. Frozen Specimens

Detection sensitivity can change for frozen specimens as compared to fresh. In developing your test, we recommend you consider and evaluate whether this is a concern for any of the pathogens detected by the particular multiplex panel (i.e., they should not be frozen and thawed). You should assess the effect of storage temperature and repeated freeze/thaw cycles on the yield of the nucleic acid and its influence on the assay performance. The evaluation of fresh versus frozen specimens can be considered during the determination of the limit of detection (LoD) providing an early indication of assay performance with specimens that have been frozen as compared to fresh specimens. The freeze-thaw conditions tested should capture the actual conditions encountered in a clinical setting and those used to establish the enrollment criteria for frozen archived specimens that could be used in the prospective analysis.

If significant differences are observed with the LoD, then additional studies will not be necessary and product labeling will indicate the use of fresh, unfrozen specimens only. If developers still wish to pursue the use of frozen specimens, then performance should also be established for frozen specimens by conducting a study consisting of a total of 60 positive clinical specimens tested fresh and frozen. In certain circumstances, well-designed mock specimens, made by spiking cultured organism into individual negative specimens could also be considered. However, FDA feedback for this approach is recommended prior to undertaking the study. The positive specimens should be representative of the test panel composition. If it is the case that fresh and frozen specimens are not equivalent, then developers should ensure that all studies are conducted with specimens that have been handled properly.

iii. Nucleic Acid Extraction

Different extraction methods may yield nucleic acids of varying quantity and quality, and thus the extraction method can be crucial to a successful result. Purification can be challenging because biological specimens may contain low pathogen signature sequences in a background of human genomic DNA, normal flora, as well as high levels of proteins and other contaminants. Additionally, the lysis conditions may differ depending on physiochemical nature of the organisms detected by the device.

For these reasons, you should evaluate the effect of your chosen extraction method on the performance of the assay with respect to satisfactory nucleic acid quantity and quality for the intended use of the assay. In addition, you should evaluate your assayâ€™s clinical performance characteristics using the entire pre-analytical process (including extraction procedures) that you recommend for use with your assay. If multiple extraction methods are recommended for use with your assay, you should demonstrate extraction quality and efficiency, as well as analytical and clinical performance of your assay with each extraction method. Specifically, you should demonstrate LoD and reproducibility for each pre-analytical method indicated for use by your device. You may be able to combine the extraction method variable with each site performance variable. For example, if you recommend three different extraction methods, you can design a reproducibility study by evaluating one of the three extraction methods at each of three testing sites: test extraction method A at site 1, test extraction method B at site 2, and test extraction method C at site 3.

However, if the studies from the three sites indicate statistically or clinically significant differences in assay performance, the reproducibility study should be expanded to include testing each extraction method at three study sites (e.g., site 1 extraction methods A, B and C, site 2 extraction methods A, B and C, and site 3 extraction methods A, B and C).

In addition to the analytical limit of detection and reproducibility, each extraction method should be utilized in at least one clinical site during the clinical studies to generate clinical performance data. If results from the expanded reproducibility testing indicate a significant difference in efficiency among the extraction methods, the data from each clinical testing site (using a different nucleic acid extraction method) are not considered equivalent and should not be combined, but rather should be analyzed separately. As a consequence, additional prospective clinical specimens may be called for in order to support the claimed extraction method.

You should provide your recommendations for assuring specimen adequacy for the different specimen types for which your assay is indicated. For example, the quality of the nucleic acid can be assessed using internal controls that determine presence, quality, or both presence and quality of the nucleic acid.

iv. Library Preparation

For devices that use library preparation steps you should address the variability on assay performance for all claimed preparation methods and reagents used. Different library preparation methods may yield nucleic acids of varying quantity and quality, and thus the extraction method can be crucial to a successful result. You should consider the steps involved in the construction and normalization of the specimen libraries, which could impact the reproducibility and reliability of the sequences generated (e.g., primers, amplification efficiency, reagent lots, hybridization, etc.). These factors could impact the assayâ€™s performance.

v. Specimen-to-Specimen Cross Contamination with Automated Extraction Systems

If automated systems are used or recommended for nucleic acid extraction, you should include a check of potential well-to-well cross contamination as part of the performance qualification of the extraction instrument. You should provide a software hazard analysis for automated extraction systems as part of your 510(k). A validation study of the extraction process can be designed in a pattern such that a nucleic acid-containing specimen with a concentration at the highest anticipated clinical level is surrounded on all sides by negatives. The results should demonstrate that well-to-well cross-contamination does not occur.

vi. Performance Study Quality Controls

Evaluation of assay performance should include appropriate controls for the duration of the analytical and clinical studies. The results should also include any positive and negative controls provided with your assay as well as appropriate external controls recommended but not necessarily provided with the assay. If a rotating control scheme is used throughout the evaluation process, the results should be presented for each control panel member.

Class II Special Controls Guidance Document: Instrumentation for Clinical Multiplex Test Systems.â€⁷ If the multiplexed assay uses instrumentation for clinical multiplex test systems and obtains FDA clearance, the instrumentation for clinical multiplex systems will be cleared for use with the assays included in the device. Simultaneous clearance of the assay and the instrumentation can be accomplished by either (1) the sponsor of the assay submitting the information for both the assay and the instrumentation for clinical multiplex systems within one 510(k) or (2) the instrument manufacturer simultaneously submitting a separate 510(k) for the instrumentation for clinical multiplex systems by itself. The amount of information required for clearance of the instrument depends on whether the instrument has been previously cleared for a similar intended use, and whether there are software modifications to accommodate the new assay(s).

If your system includes software, you should submit software information detailed in accordance with the level of concern. More information can be found in the document entitled â€œ Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices.â€⁸ You should determine the level of concern prior to the mitigation of hazards. In vitro diagnostic devices of this type are typically considered a moderate level of concern because software flaws could indirectly affect the patient and potentially result in injury due to inaccurate information.

Below are additional references to help you develop and maintain your device under good software life cycle practices consistent with FDA regulations.

General Principles of Software Validation; Final Guidance for Industry and FDA Staff⁹

Guidance for Off-the-Shelf Software Use in Medical Devices; Final Guidance for Industry and FDA Staff¹⁰

Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices¹¹

21 CFR 820.30 Subpart C â€“ Design Controls of the Quality System Regulation.

ISO 14971-1; Medical devices – Risk management – Part 1: Application of risk analysis.

AAMI SW68:2001; Medical device software – Software life cycle processes.

Guidance on Informed Consent for In Vitro Diagnostic Device Studies Using Leftover Human Specimens that are Not Individually Identifiable¹²
In Vitro Diagnostic (IVD) Device Studies -Frequently Asked Questions¹³

14, clinical specificity would be demonstrated to achieve a point estimate 99.9% with a lower bound of the 95% CI greater than 99%.

Also note, for each specimen from the prospective study that has positive results by the subject multiplexed device for a pathogen, this specimen will also require testing by the corresponding CM. Information about CM results that were driven by a positive result from the HMMD would not be used directly in calculation of sensitivity and specificity as it introduces bias into estimation of the multiplexed device performance. However, this information is useful to understand the overall performance of the HMMD, especially in terms of co-detections, and should be presented in a separate table. Comparative performance of the specificity of the HMMD should be established using FDA cleared or approved devices, if available. Use of cleared multiplexed devices, when appropriate, is recommended.

When FDA cleared or approved devices are not available or appropriate, a composite reference method of two well validated PCR based assays followed by bidirectional sequencing could be used. See section 6.D.ii for further clarification.

Evaluation of Positive Percent Agreement

The analysis will include a minimum of 50 positive specimens (as determined by the CM) per claimed organism. The number of positive specimens for each pathogen to include will be driven by the point estimate of positive percent agreement and the lower bound of the 95% two-sided confidence interval. These values can vary, depending on the intended use of the device, and would require discussions with the FDA to determine the appropriate clinical sensitivity levels for each pathogen of the multiplexed device.

For example, a multiplexed device with an assay menu composed of upper respiratory organisms would include a sufficient number of archived/retrospective specimens for each claimed analyte to generate a result with at least 90% positive percent agreement with a lower bound of the two-sided 95% confidence interval (CI) greater than 80%. Assuming a point estimate of 90.2% is achieved; a minimum of 61 positive specimens (55/61) will need to be included to surpass the indicated lower bound of the 95% CI of greater than 80%. Indeed, for 61 specimens, 55 out of 61 yields a point estimate of 90.2% with 95% CI: 80.2% to 95.4%. However, using the example of 60 specimens, with a performance of 54 out of 60 yielding a point estimate of 90.0%, the CI does not meet the minimum performance bar with 95% CI being 79.9% to 95.3%.

All positive archived/retrospective specimens (as determined by the method prior to banking the specimens) will be analyzed with the corresponding CM and the subject device. Verification by the CM is imperative to ensure that specimens were properly archived, no specimen degradation occurred during storage, and that the specimens are properly identified; thus, any specimens that are not confirmed as positive by the CM should not be included in the performance determination. Additionally, any positive determination for any other pathogen by the HMMD should also be verified by the CM as this provides additional information about multiplexed device performance, especially in cases of potential co-infection. Alternative approaches to confirming the positivity of specimens can be considered; however, we would encourage discussion with the FDA before executing the study. Retrospective positive specimens should be the same specimen type as listed in the intended use of the device, and have been collected from the appropriate intended use population. The specimens selected for inclusion in this study should represent the clinically relevant range of concentrations for the particular pathogen. In cases where extracted nucleic acids from positive clinical specimens have been archived, they can be considered for inclusion in the analysis provided that the appropriate intended use population was used, the indicated specimen type was collected and processed using the indicated pre-analytical steps, and confirmation by the corresponding CM was done.

We recognize that actual clinical human specimens, archived or otherwise, may not be readily available for biothreat pathogens. The use of mock clinical specimens, prepared by spiking cultured pathogen into individual negative clinical specimens may be used. For this analysis, 50% of the spiked specimens would be made at the LoD concentration, while the remaining 50% would span the expected clinical range of pathogen concentrations. For non-biothreat pathogens with extremely low prevalence, mock specimens should reflect the relevant clinical range. Justification of the expected clinical range through peer-reviewed literature references or feedback from subject matter experts should be provided by the developers for each specimen type indicated. Given the restrictions associated with the handling of many of the biothreat pathogens, arrangements to validate the clinical performance at qualified institutions with the capability to conduct the proper studies should be made. For biothreat pathogens, due to the logistical issues with this facet of the positive percent agreement validation, analysis can be conducted at a single site. Alternatively, if the biothreat pathogen does not involve a special facility, a multiple testing site approach can be used to evaluate positive percent agreement , and the archived specimens (positive and negative) should be randomly distributed evenly among three testing sites for analysis.

Prior to conducting any studies using mock specimens, you should consult the FDA for feedback. Your protocols should include a detailed test plan and justification.

iv. Data Presentation

You should present positive percent agreement (PPA) and negative percent agreement (NPA) (with 95% CI) separately for each pathogen identified by the HMMD. Also, you should present:

The results of your test for the specimens that have co-infections as obtained by the reference method. Note: This information may not be available for some of the prospective specimens due to prohibitive specimen volume.
The results of the CM for the specimens that have co-infections as obtained by the HMMD.
The results for CM measurements that were driven by a positive result from the subject multiplex device.

All specimens in the clinical study should be tested with the HMMD as described in the instructions for use of your device. For example, if specimens with initial indeterminate or invalid results are re-tested according to the instruction for use for the HMMD then the final result obtained from the indicated testing procedure for these specimens should be used in your statistical analysis. For the specimens in your clinical study, you should provide the percent of re-tested specimens because of initial indeterminate results (if applicable) and the percent of re-tested specimens because of initial invalid results (if applicable). In addition, you should present the percent of final invalid and final indeterminate results (if applicable) for each.

You should provide numerical result distributions of the HMMD for all prospectively collected fresh, prospectively collected archived, and banked pre-selected specimens shown separately, for each pathogen and for all pathogens combined.

v. Study Specimens/ Specimen Types

You should use clinical specimens from all specimen types and matrices you claim in your intended use to demonstrate that correct results can be obtained from clinical material. For specimens you use in your clinical studies, you should provide data demonstrating that storage and transport of any banked specimens have not affected assay results as well as the methods used to bank the specimen as positive for a specific analyte. If you have questions regarding the choice of appropriate specimen type(s) as well as specimen types that can be pooled, please contact the FDA.

7. Post Market Analysis

The potential for performance erosion over time for certain pathogens, especially those that are known to have a relatively high rate of mutation, is a significant concern for HMMDs. HMMDs are subject to the quality system regulation described in 21 CFR Part 820. As part of the quality system a developer should monitor the performance of their devices in the intended use environment and through their Corrective and Preventive Actions program. If a mutation occurs that compromises device performance characteristics that were used in the premarket evaluation for clearance of the HMMD, we strongly recommend that the developer contact the FDA for further guidance to modify the existing device. The validations studies for the modified device and the performance of the modified device will vary depending on the type of change.

8. Device Modification

The following information provided is meant to clearly define a pathway to incorporate new targets on an existing device in response to public health needs or an emergency situation and to ensure that performance characteristics of a cleared device are consistent over time. Addition of a new analyte to a multiplex panel will result in a new intended use, and thus will require the submission of a traditional 510(k) in accordance with 21 CFR Subpart E. Since many studies have been conducted to establish the performance of the previously cleared device and we presume that the assayâ€™s performance has not changed, only a subset of the evaluations would need to be repeated.

In cases where the inclusion of additional targets to address a public health need or emergency is necessary, the studies to substantiate performance will focus primarily on the additional analyte. It is important to highlight that certain types of evaluations may not need to be considered when adding a new analyte or modifying a device, including stability studies and the evaluation of carry-over and cross contamination. Additionally, the scope of the reproducibility study and clinical evaluation would be focused on the new/modified analyte and a representative panel for performance confirmation.

A. Limit of Detection

A LoD study using the recommendations found in section 6.B.i of this draft guidance should be followed for the new analyte. Briefly, confirmation of 95% positivity using a minimum of 20 replicates, including all pre-analytical steps, would be done. In addition, a confirmation that performance for previously cleared analytes has not changed in the new assay format (i.e., with the new analyte). For the confirmation study, a dilution series to bracket LoD concentration with three to five extraction replicates per concentration testing only a representative subset of previously cleared analytes. These specimens should be run on both the modified assay and the cleared assay. The selection of representative analytes used in the conformation study should be justified by the developer.

B. Analytical Reactivity

Evaluation of reactivity should be done for the new analyte. This evaluation should be based on the recommendations provided in section of 6.B.ii; however, the study should focus on the new analyte only.

C. Cross Reactivity

Evaluation of cross reactivity should be done for the new analyte. This evaluation should be based on the recommendations provided in section of 6.B.iii, however, the focus should be placed on the new analyte only.

D. Competitive Interference By Analytes /Interference By Other Microorganism s

Evaluation of competitive interference and interference from other microbes should be done for the new analyte. This evaluation should be based on the recommendations provided in section of 6.B.iii.b and 6.B.iii.c. The evaluation of competitive inhibition should only include relevant co-infections that include the new analyte.

E. Precision (Repeatability/Reproducibility)

Evaluation of reproducibility/repeatability should be done using a single site, which can be in-house, using three different instruments. The evaluation should include the new analyte and a subset of the previously cleared analytes. Developers should provide justification for the representative panel selected for this evaluation and are encouraged to consider the LoDs and types of targets when designing the test panel. See section 6.B.v. for appropriate test levels.

F. Limited Clinical Evaluation

A limited clinical evaluation should demonstrate the performance of the new analyte and reaffirm the clinical performance of the previously cleared analytes. Evaluation of device performance with clinical specimens can be done using a single test site and can be done in house. Positive clinical specimens can be either prospectively collected or can be archived clinical specimens as described in section 6.D.iii. Comparator testing for the new analyte should be done as described in section 6.D.ii. For the rest of the analytes, you can perform an evaluation of the cleared device (i.e., device without new analyte) and the modified version compared head-to-head using a representative panel of analytes. The test panel should be justified and include sufficient specimens to achieve the performance criteria addressed in section 6.B.iii for sensitivity and specificity. If there are stored residual specimens or extracted nucleic acids from the evaluation of the cleared device (i.e., prior to modification with new analyte) it is recommended that they are utilized for this part of the evaluation. If a sufficient number of specimens are not available, then additional retrospective specimens can also be used for the comparative evaluation.

¹ http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/ HowtoMarketYourDevice/PremarketSubmissions/PremarketNotification510k/default.htm

²http://www.fda.gov/downloads/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/ucm084396.pdf

³http://www.fda.gov/downloads/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/ucm073756.pdf

http://www.fda.gov/downloads/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/UCM071061.pdf

⁸http://www.fda.gov/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/ucm089543.htm

⁹http://www.fda.gov/medicaldevices/ deviceregulationandguidance/guidancedocuments/ucm085281.htm

¹⁰http://www.fda.gov/downloads/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/ucm073779.pdf

¹¹http://www.fda.gov/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/ucm089543.htm

http://www.fda.gov/downloads/ RegulatoryInformation/Guidances/ucm127033.pdf

¹³http://www.fda.gov/downloads/MedicalDevices/ DeviceRegulationandGuidance/GuidanceDocuments/UCM071230.pdf

¹⁴ For clarification of pathogens considered to be Select Agents please see the National Select Agent Registry.

Effective Medicine

Medication and Pharmaceutical news

Draft Guidance for Industry and Food and Drug Administration Staff

Preface

Additional Copies

Table of Contents