Human Factors / Usability Engineering Question:

QUES:  As you mentioned on the webinar, the usability engineering is done as needed based on the product. So how should we decide if we need the usability engineering on specific product? Take anesthesia needles as an example, we know this is a mature design on the market for a long time. Is there any way to prove that this similar design won’t cause any use error? By MAUDE or adverse events?"

 

ANS:  The question is -  "How similar is the new design to the old one, as to whether it raises new issues of safety and efficacy, especially around ease / safety of use?" Not -  that a current design precludes an occasional use error, as there will always be an infrequent use error with any device, even one with an optimized design for safety / intuitive use. If the new product's design is substantially unchanged, and has a long field use history, so that it's use / use interface issues would be the same as the predicate's design, then there's no need for a usability engineering study.  Your SOP on HF / UE should discuss and allow these approaches, then you have to follow your SOP. Document your rationale.

FDA's 2016 Guidance on HF / UE:  

"As part of their design controls, manufacturers conduct a risk analysis that includes the risks associated with device use and the measures implemented to reduce those risks. ANSI/AAMI/ISO 14971, Medical Devices – Application of risk management to medical devices, defines risk as the combination of the probability of occurrence of harm and the severity of the potential harm. However, because probability is very difficult to determine for use errors, and in fact many use errors cannot be anticipated until device use is simulated and observed, the severity of the potential harm is more meaningful for determining the need to eliminate (design out) or reduce resulting harm. If the results of risk analysis indicate that use errors could cause serious harm to the patient or the device user, then the manufacturer should apply appropriate human factors or usability engineering processes according to this guidance document. This is also the case if a manufacturer is modifying a marketed device to correct design deficiencies associated with use, particularly as a corrective and preventive action (CAPA)."  

--  https://www.fda.gov/media/80481/download , pg 2, para 1

 

Yes, MAUDE / adverse events would be one of several (see the UE File template example in my slides) possible sources to check.  If there could be a questions as the need to do a UE analysis, and you chose not to, you could do a one or two page plus "UE File" stating the limited analysis done to reach the conclusion that a full-blown study was not needed.

Patient / Device Risk Management (ISO 14971) is always done on new device and major change to existing devices.  Human factors / usability engineering is done as needed, per the above in red.  Each activity is described in a File, which is part of Design Control (or CGMP Change Control) depending on where or when in the product's life cycle the initial design or design change is initiated. 

 

-- jel@jelincoln.com 

To the first paragraph under "answer", added new last sentence on "document your answer." - JEL 05/01/24

 

Mil Std 105E  Sampling Plans Not Dead Yet

Mil Std 105 E was officially cancelled in 1995 and replaced by ANSI/ASQ Z 1.4 for sampling by attributes.  At that time I purchased a copy of Z 1.4 (for a fee ~$199.00, and copyrighted; whereas the Mil Std stated that its "distribution was unlimited" - no fees and no copyright worries), and started to switch my clients over to Z 1.4. But in checking I found that the Z 1.4  sampling plan was identical to the 105E's, so I went back to 105E with those of my clients who hadn't updated / wanted that type of plan, using the105E's plans copied in the SOP's I wrote for them, without having to purchase Z 1.4 / updates for every client and worrying about the associated copyright issues. 

However, one of my clients had an FDA CGMP compliance inspection, and was called out for using 105E, an "obsoleted standard" for acceptance sampling, and the FDA inspector was adamant in the need to change to an alternative. I couldn't convince the inspector that the plans were identical, so, rather than argue, we changed (in a couple of instances, to Nicholas Segura's C=0, also for a fee and copyrighted, and derived from Z 1.4).  

Recently, while updating an QC inspection SOP, I stumbled upon some older Quality Digest articles arguing for the advantages of the old Mil Std105E, and which referenced ASTM E2234, which "carries forward" that standard (e.g., see Wikipedia reference below).  So now I once again have Mil Std 105 E as an option for clients for its sampling plans for QC sampling in CGMP applications in medical device manufacturing, because they have always been and are still valid, but I now had a valid, easy to understand rationale for their continued use.       

"MIL-STD-105 was a United States defense standard that provided procedures and tables for sampling by attributes based on Walter A. Shewhart, Harry Romig, and Harold F. Dodge sampling inspection theories and mathematical formulas. Widely adopted outside of military procurement applications.

The last revision was MIL-STD-105E; it has been carried over in ASTM E2234 - "This practice establishes lot or batch sampling plans and procedures for inspection by attributes using MIL-STD-105E as a basis for sampling a steady stream of lots indexed by acceptance quality limit (AQL). It provides the sampling plans of MIL-STD-105E in ASTM format for use by ASTM committees and others and recognizes the continuing usage of MIL-STD-105E in industries supported by ASTM. This practice also establishes lot or batch sampling plans and procedures for inspection by attributes."

It was officially cancelled in February 1995 by a Notice of Cancellation. This Notice was updated in March 2001 and again in February 2008. The current Notice of Cancellation (Notice 3) recommends that future acquisitions refer to: MIL-STD-1916, "DoD Preferred Methods for Acceptance of Product", or ANSI/ASQ Z1.4, "Sampling Procedures and Tables for Inspection by Attributes"."

-- Wikipedia and ASTM E2234 description (Color added by JEL)

The above information is presented FYI only.

-- jel@jelincoln.com

12/25/2023 - Added additional to ASTM E2234. - JEL

 Recent questions on a webinar on Verification and Validation:

My responses:

 

Ques 1: 

In the handouts the term “pre-approval” is mentioned several times. Going through the recording did not fully make this term clear to me. Would be great to have some details on what´s behind this.

Ans 1:  I use the term when I add to a validation test report a pre-approval signature block.  The document is routed to the stakeholders for review and approval before the validation is run, to get changes and "buy-in" to the format and test cases beforehand.  It reduces the problems of getting post-approvals when the signatories didn't have a chance to agree to and sign off on the proposed validation prior to performing it. Your validation SOP would be written to address this feature if you chose to employ it. 

 

Ques 2: 

I would like your assistance with a question raised during a discussion regarding manufacturing process verification/validation:

In one of the implant’s identified critical processes, a visual inspection (under microscope by a technician) is one of the tools used to validate the process.

 The question we are facing is whether the visual inspection should be in a quantitative scale rather than qualitative.

Please bear in mind that the visual inspection is done by a technician who uses specific criteria for approval/disapproval of the implant.

 

Ans 2:  Quantitative is generally better that qualitative, as subjectivity / interpretation elements are greatly reduced.  But qualitative inspections are used frequently.  How you structure the visual inspection to reduce subjectivity / interpretation would be a key consideration.  One can also be used to complement the other for some inspection elements. And monitoring the results over time will provide the best indication of the effectiveness of a chosen approach, a CGMP requirement anyway.

- jel@jelincoln.com

Definition added 08/30/23:  Qualitative observation results cannot be measured while quantitative observation gives measurable data. Quantities like area, height, weight, temperature, weight, time, speed, etc., are examples of quantitative observation while smell, taste, texture, color, etc., are examples of qualitative observation.

Years ago, on specs on/in injection-molded plastic parts, we changed a qualitative "test" (a subjective "too many") to quantitative (mostly; an unacceptable volume to total black specs per set area per the chart) by implementing the use of a TAPPI Dirt Estimation Chart, with a pass/fail range.  See:

https://www.tappi.org/publications-standards/standards-methods/charts--datase/

- JEL 09/19/2023 

Additional justification for "pre-approval": 

 

FDA's Guidance:  "Analytical Procedures and Methods Validation for Drugs and Biologics", issued July 2015, beginning at line 264: 

     "Validation data must be generated under a protocol approved by the sponsor following current   

     good manufacturing practices with the description of methodology of each validation characteristic       and predetermined and justified acceptance criteria, using qualified instrumentation."

-- color added, similar statements can be found in other such documents.  E.g., Google "What is pre-execution approval of validation test scripts?"  -  JEL 10/13/2023