Final Device Human Factors Guidance – Ch-Ch-Ch-Ch-Changes?

By Melisa M. Moonan –

After more than 600 comments and four and a half years since the draft guidance (“Applying Human Factors and Usability Engineering to Optimize Medical Device Design” (June 21, 2011)) was published, FDA has released the final device human factors guidance, “Applying Human Factors and Usability Engineering to Medical Devices, Guidance for Industry and Food and Drug Administration Staff” (Feb. 3, 2016) (final HFE guidance). FDA considers the terms human factors engineering and usability engineering to be synonyms; we utilize the acronym HFE here for convenience.

The final HFE guidance provides a detailed framework for consideration and testing of user interface design and potential use error. FDA delayed effectiveness of the final HFE guidance until April 3, 2016, perhaps to provide industry some time to understand whether the numerous language and organization changes made since the 2011 draft guidance was published are just fine-tuning or represent significant changes in agency expectations. With some exceptions, we do not believe the final version represents significant overall change from the draft. Many changes appear to be clarification of terminology and addition of detail. Some changes (including the revision to the title to drop the words “optimize” and “design”) appear to emphasize an agency message that the goal of the guidance is the establishment of safety and effectiveness of devices for their intended users, uses, and use environments, not merely design optimization that may enhance safety and effectiveness. In short, human factors design and validation are critical components of a device’s safety and effectiveness, not merely an added bonus.

As we previously blogged, here, the agency simultaneously issued a complementary draft guidance, “List of Highest Priority Devices for Human Factors Review, Draft Guidance for Industry and Food and Drug Administration Staff” (Feb. 3, 2016) (draft List guidance). The draft List guidance addresses one issue raised by the draft HFE guidance, i.e., when will the agency expect or request HFE data in premarket submissions. In the draft List guidance, FDA provides a list of 16 classified device types and the corresponding product codes for which FDA will expect human factors data to be included in premarket submissions in an “HFE/UE Report”, as outlined in Appendix A of the final HFE guidance. As we discussed in our blog, FDA also sets out certain circumstances under which ODE reviewers will be empowered to ask for such data on a case by case basis for devices not on the list.

Comments and Changes

In the Federal Register announcement for the final HFE guidance, FDA stated that comments on the draft HFE guidance most frequently requested changes to the guidance’s language and structure, and clarification of the following topics: risk mitigation and human factors testing methods; user populations for testing; training of test participants; sample size determination; reporting for premarket submissions; and collecting human factors data as part of a clinical study. Considering the complexity of this topic, the significant impact on industry and the extraordinarily high level of interest, we believe this guidance topic may have been better served by notice and comment rulemaking. FDA conducted a webinar on the guidance on February 19, 2016, which can be found here: http://www.fda.gov/Training/CDRHLearn/. In addition, we believe an agency FAQ document may be beneficial for enhanced understanding of FDA’s thinking and its responses to industry concerns.

One of the changes made was to add definitions of some important terms. One of the new definitions is for the term “critical task,” and that term receives additional focus in the final HFE guidance. A critical task is “[a]user task, which if performed incorrectly, or not performed at all, would or could cause serious harm to the patient or user, where harm is defined to include compromised medical care.” Devices with identified critical tasks would therefore seem to qualify for submission of their HFE data to FDA in premarket submissions. Related changes to the guidance’s section on Preliminary Analyses and Evaluations provide additional detail on methods of identifying critical tasks, and an expanded section specific to critical task assessment methods and findings was added to the final “HFE/UE Report” outline provided in Appendix A.

Another difference from the draft guidance is found in the Conclusion section of the final guidance’s HFE/UE Report outline. The draft Report outline’s Conclusion section requested a statement from the applicant that the product had been found “reasonably” or adequately” safe and effective for the intended users, uses, and use environments, i.e., language akin to the PMA approval standard of “of a reasonable assurance of safety and effectiveness.” However, the final Report outline’s Conclusion section requests a statement that dispenses with the modifiers; a submitter’s HFE/UE Report conclusion now should be that the device “has been found to be safe and effective for the intended users, uses, and use environments.” The recommended conclusion language thus exceeds that of the PMA approval standard. We are curious how FDA will implement review of the HFE/UE Report in the context of a PMA approval, and are particularly interested in how the Report fits into 510(k) review and clearance, where the overall standard is substantial equivalence, and predicates may not have HFE data on file with FDA for comparison. We believe it would be helpful if FDA utilizes an FAQ document or the draft List guidance to clarify how review of HFE/UE Reports will affect premarket programs and processes.

We noted a similar shift in language by FDA in the final guidance’s Scope section. The draft guidance recommended implementing HFE testing where there was a “moderate to high risk” of use error. In the final guidance, FDA states that use errors are not easily anticipated and suggests that potential severity alone, i.e., without giving much, if any, weight to likelihood of occurrence of the harm, should drive HFE design activities and decisions whether a hazard (a “potential source of harm”) must be designed out to eliminate the risk. This is paralleled in the risk mitigation section (now section 7) where the title has changed from “Mitigation and Control of Use –Related Hazards” to “Elimination or Reduction of Use –Related Hazards.”

HFE Validation Testing and Residual Risk Analysis

In addition to critical tasks, another major emphasis in the final guidance is on HFE validation testing (now section 8). FDA encourages seeking review of draft validation testing protocols under the CDRH pre-submission program to ensure the methods used will be acceptable. There is also guidance provided on HFE validation testing for product modifications and/or corrections, and a new Appendix (Appendix C) that provides information on and examples of test results analysis.

The four key factors identified for HFE validation testing design are:

• the test participants represent the intended (actual) users of the device;
• all critical tasks are performed during the test;
• the device user interface represents the final design; and
• the test conditions are sufficiently realistic to represent actual conditions of use.

The recommended sample size remains a minimum of 15 from each distinct user population, and could be higher for certain device types. The final guidance also reiterates that subjects should be U.S. residents and should not be employees of the manufacturer, and training should approximate what would be given to actual users, including that it should be allowed to elapse somewhat before testing.

An area that the final guidance fails to clarify fully are the differences between HFE simulated use testing, HFE actual use testing, and HFE actual use testing that is considered clinical testing requiring an IDE or exemption. With regard to the latter, the final guidance states that when “actual use testing is needed to determine safety and effectiveness of the proposed device and the requirements of § 812 apply,” an IDE or IDE exemption would be needed. This is an unhelpful tautology. We believe examples of actual use HFE studies that do and do not require IDE or IDE Exemption would be helpful. This is particularly important for planning modifications to 510(k) devices, where the need to do a clinical trial to support equivalent safety and effectiveness may trigger a 510(k) submission requirement.

Like the draft, the final guidance states that testing results indicating that use errors that could cause harm persist in the design are not acceptable in submissions, unless there is a strong rationale demonstrating that “further reduction of the errors’ likelihood is not possible or practical” and the benefits of device use outweigh the residual risk of harm. A promise to address the use errors in subsequent versions will not suffice, and may invite further scrutiny of the HFE processes used.

Looking Forward

We recommend that device manufacturers prioritize review of the final HFE guidance and the draft List guidance. With the publication of the final HFE guidance, manufacturers can expect increased FDA scrutiny of HFE design processes during inspections focusing on design controls. FDA will expect to see medical device user interfaces thoroughly considered in product risk assessments, and will look for HFE processes for user interface evaluation, risk mitigation, and validation testing to have been implemented wherever a use error could result in serious harm. With regard to premarket submissions, agency expectations should remain status quo ante until the draft List and accompanying criteria for requests for such data are finalized. Nonetheless, manufacturers should conduct human factors risk assessment and consider the final HFE guidance’s recommendations for devices in development.