Risk vs. Risk

RISK  VS. RISK

"Risk-based ...", a source of confusion to the medical products industries.

Some say this should be a generalized approach throughout a company. This is partially correct.  If a company is primarily addressing ISO 9001, then they are focused on ISO 31000, Risk Management, which addresses all manner of business risk.      

   

However, if we are dealing with medical products, the U.S. FDA want to see “risk” tied specifically to to patient (and user) risk.

I have always recommended that companies tie such key medical product risk-based decisions to a Product Risk Document -  ISO 14971 Risk Management File / Report, or ICH Q9;

     -  Cite specific line items, e.g., from a FMECA;

     -  Include “Normal” as well as “Failure / Fault” in Hazard List / FMECAs.

“Risk” in FDA-regulated industries usually means patient risk, not business, IT, legal, etc., risks, though some are obviously tied together.  If you are marketing medical products both in the U.S. and EU / overseas, then your documentation will have to clearly address both types of risk, product / patient, and business.

ISO 14971 patient risk / safety vs. ISO 31000 business risk / “safety”.

Understanding such patient “risk” will determine how far to proceed on test cases, failure investigations / root cause analysis, degree of documentation, etc., needed to resolve a medical product risk issue.

Additional references:

"Pharmaceutical cGMPs for the 21st Century - A Risk-Based Approach", September 2004: 

"FDA has identified a risk-based orientation as one of the driving principles of the CGMP initiative.  The progress outlined below reflects FDA's commitment to the adoption of risk management principles that will enhance the Agency's inspection and enforcement program, which is focused on protecting the public health."  (emphasis added)
 -- https://www.fda.gov/drugs/developmentapprovalprocess/manufacturing/questionsandanswersoncurrentgoodmanufacturingpracticescgmpfordrugs/ucm137175.htm#_Toc84065737 

John E. Lincoln        jel@jelincoln.com

FURTHER USABILITY ENGINEERING Q & A

Ques: The first question is applicability of usability engineering on all products including legacy.  Do we need to go back and remediate all legacy product files that do not have usability? Is there an expectation to remediate or on a forward moving basis as we make changes to legacy products.

Ans:  I'm not aware of any expectation on the part of the FDA to require such for legacy products until they undergo sufficient change(s) to warrant such (which should be defined in your SOP, as determined by your company's analysis of the usability issues your products pose; and a discussion / rationale written - almost a memo to file or "one page" UE File).  For CE Marking, you'd have to talk to your N-B. 

Ques:  Is the expectation that all devices go through usability? For example for a simple device like a syringe where the risk is well understood and low would we conduct usability? Do you have recommendations for how to proceduralize which products we apply usability to?

Ans:  I would analyze all, even if only to have a UE File that basically says it's not necessary, e.g., your example of the syringe.  Of course if it has some non-stick component, then it would probably require such.  Your company would have to make the decisions as to how to proceduralize / decide which to apply UE to.  In my webinar, I basically stated that some products, like needles (not non-stick), have such a field use history over many decades that they probably wouldn't need it, but that decision / rationale should be defined by SOP and recorded per first ans above.

  

Ques:   Annex C of IEC 62366-1 has a statement on user interface of unknown provenance, can you explain the intent of this? Is this referring to legacy that does not have usability documentation?

Ans:  As discussed in the webinar, UOUP, applies to user interfaces for which there is no real UE or similar documentation available from a company / vendor.  How it will be specifically applied I don't yet know -- we'll need some field history.  Pending that, I tend to recommend that it be applied for anything posing a serious UE issue for which evidence of any HF / UE activity hasn't been performed / documented.  The principle could apply for a company's own legacy products. Since the analogy to SOUP software was made, how that principle is implements re: software, might help in defining how to implement UOUP.

Ques:  What class devices does usability apply to?

Ans:  There's no specific reference to class of devices, either US or EU.  Obviously, Class II (and IIa, -b) and III would be more complicated, higher risk, and more likely to require  more HF / UE action -- with new major changes, or new devices, a written discussion / rationale appropriate to need / risk, would be appropriate. General controls / cgmps (includes 820.30 where risk mgmt / HF / UE are employed / documented) are also a requirement for Class I in the US, so address as appropriate, defined by your SOP.

John E. Lincoln  jel@jelincoln.com

USABILITY  ENGINEERING / HUMAN  FACTORS  ENGINEERING -- THE NEW IEC 62366-1:2015, and IEC/TR 62366-2:2016

Some answers to some questions raised on my webinar on the above subject:

Ques: Examples of FDA HF guidance documents, etc?:

https://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/HumanFactors/ucm119190.htm#guidancehf

As mentioned, use the search box in upper right on FDA web site, as well as links from the above link.

Ques:  HE75?

Ans:  The ANSI web site sells both HE75 and IEC 62366-1:2015 together, and states:

"The ANSI/AAMI HE75 and ANSI/AAMI/IEC 62366 Human Factor Set addresses a broad range of human factors engineering (HFE) topics in a structured format. The material emphasizes adoption of a user-centered focus throughout the product design and development process, with the goal of making medical devices easier to use and less prone to use error. By providing a structured approach to user interface design, this set documents can help manufacturers develop safe and usable medical devices.

• The ANSI/AAMI HE75 and ANSI/AAMI/IEC 62366 Human Factor Set includes
• ANSI/AAMI HE75:2009 (R2013) (ANSI/AAMI HE 75:2009 (R2013))
• ANSI/AAMI/IEC 62366-1:2015"

http://webstore.ansi.org/RecordDetail.aspx?sku=ANSI%2FAAMI+HE75+and+ANSI%2FAAMI%2FIEC+62366+Human+Factor+Set&source=google&adgroup=ansi-aami&gclid=CjwKEAiA0fnFBRC6g8rgmICvrw0SJADx1_zAwESZsbs9ED83LJflK_Qq0H773imIq4qMb-lGfKA12hoCrbnw_wcB

Not too helpful in my opinion.

The FDA allows you the manufacturer to determine how to address (and they allow plenty of leeway as long as the requirements are addressed by your procedures , and you follow your own procedures).  For CE-marking, work with the preferences of your Notified Body, following the general outline presented in the webinar / standard (the 9 stages, documented in a Usability Engineering File), and and  with consideration of the new EU MDR.

I still believe that the new IEC 62366-1:2015 is the documented process to follow with all current and further UE projects, following and documenting the 9 stages in the UE File as we discussed in the webinar.  The FDA and ANSI/AAMI HE 75 and others could be used in the actual HF/UE analysis where appropriate for ideas, but the documented process should follow IEE 62366-1:2015's 9 stages and the document deliverable in the UE File.

I suspect it will take awhile for some consensus in implementation to build (with the EU implementation).  But I believe that the "end" result will be similar to what I've outlined above. 

-- John E. Lincoln  --  jel@jelincoln.com 

"Catchup" 510(k)s

This issue was posed by an attendee at my recent webinar on Device Changes and the 510(k),
discussing FDA's K97-1 and their two new August 8, 2016 draft guidance documents.

We have several older devices that were cleared in the late 90's. These have gone through several changes over the years that were all justified as being not significant, so no new 510(k)'s were filed. We have heard about companies in our industry doing what they would consider "catchup" 510(k)'s where they are essentially submitting a 510(k) every so often since the creep effect tends to happen even if they do not see all of the changes as necessarily justifying a new 510(k) per the regs. We would like to do this with some of our products. My questions are 1. Have you ever seen/heard of this approach and would you recommend it and 2. If we did this, obviously it would be for all of the changes that have already taken place on the device so how would that affect the marketing of your device during the review? I know you mentioned that if you had a change and decided to submit a new 510(k) you would obviously have to wait on the FDA's approval before enacting that change, but all of our changes have already been enacted. Would you need to wait until your next change/come up with a small change to submit a new 510(k)?

My responses:

QUES 1. Have you ever seen/heard of this approach and would you recommend it?

ANS:  Tho I've heard of it occasionally I don't recommend it.  I view it as a waste of the company's money and the FDA's time.  

I've repeatedly seen FDA Investigators have no problems with routine changes to devices having the last 510(k) dated from the early 1990's, subject to the following caveats:  

IF the principles of design control are used for the changes (820.30)(remember design control came on in 1996-7, so while a DHF and the other 8 elements weren't a requirement before the mid-90's, they would  be for the changed device after); 

AND/OR CGMP document / production process change control is rigorous (820.40 and 820.70), well documented; 

AND fully supported by patient risk management (ISO 14971), usability (human factors) engineering (IEC 72366-1, -2); 

AND applicable verifications (tests) and validations (sum total of tests for a device/process/equipment), as appropriate, fully / scientifically support the decision; 

AND the points in the webinar per K97-1 (and the drafts) are followed / documented;

AND all the above lead to an honest decision that each change, And the cummulative changes, DO NOT raise new issues of safety / effectiveness (or the mandatory issues for a new 510(k) like Indications for Use, Performance ... aren't applicable...).  

"Catchup" is not an FDA term or requirement per se.  It is only applicable if one of the two points emphasized in the FDA memo / guidances and the webinar apply:  

1) the last change is major enough to raise new issues of safety /  effectiveness; or 

2) all the cummulative minor changes, with the last one being the "tipping point" considered together, now raise new issues of safety / effectiveness. 

The K97-1 and the two drafts emphasize that it is the company's responsibility to make the decision re: need for a new 510(k), and that most device changes handled under the QS Regulation (21 CFR 820) will not require a new 510(k).

To do "catchup" just to play it safe to me is a cop out (trying to push the company's responsibility to make that decision into the FDA's lap) against the above disciplines that have to be in force at a company anyway for it to be CGMP compliant.  My opinion. 

QUES 2. If we did this, obviously it would be for all of the changes that have already taken place on the device so how would that affect the marketing of your device during the review? I know you mentioned that if you had a change and decided to submit a new 510(k) you would obviously have to wait on the FDA’s approval before enacting that change, but all of our changes have already been enacted. Would you need to wait until your next change/come up with a small change to submit a new 510(k)?

ANS:  My answer to the above is the same as my discussion on the slide about a "wrong" decision, tho in this case it is the company, not the FDA, that thinks the company should have filed a new 510(k) for an earlier change. 

Again, the issue is if the last change(s) raised new issues of safety and efficacy that cannot be fully settled by test (V&V) data, or are expressesly called for in the K97-1 / guidances (as mentioned in 1 above), then a new 510(k) is required.

If a new 510(k) is required for safety / efficacy issues, or the other reasons mentioned in K97-1 / draft guidances (not just a general "catchup"), then the version(s) having the new safety / efficacy issues cannot be marketed until cleared (by a cleared 510(k)) by the FDA. 

The version of the device having those previous changes up to the point where those new issues are raised can continue to be marketed / sold.

If the decision to file a new 510(k) is based on new safety / efficacy issues (not just "catchup"), you would file at the point of that finding, e.g., due to an internal audit, FDA audit, etc., and not wait to have another change before filing.

Any time a new 510(k) is filled, all changes to the device, from  the last cleared company 510(k) to the time of filing the new would be included in the new submission.

Once the new 510(k) is cleared by the FDA, the cycle above (and as discussed by the webinar and in the guidances) starts over, with the new, cleared 510(k) as your new device "baseline".

John E. Lincoln    jel@jelincoln.com

V&V of Browser

Recently an attendee at my last V&V workshop asked about validating a browser used on their iPads at their company.  It was a new one on me. 

My initial reply:  
However, since you are using the browser only for viewing - that's your requirement - you'd just need to verify that the views you pull up are correct.  So I'd write a description of the scope / purpose, state your requirement(s), do a brief IQ (desired browser(s) there), OQ (browser(s) operate), and PQ(s) (browser information is correct), consider any applicable Part 11 issues under OQ, and this may be an exception to my class recommendation on number of PQs.  One PQ, having a small sample, say n=30 (use my citation for n=30 in your narrative, from Juran and Gryna's "Quality Planning and Analysis"), might be sufficient, basically comparing what you pull up on the browser(s) being used in your company, with another source(s) / alternative browsers.  The risk seems to be low to non-existent, so I'd analyze risk in the document to justify the minimal verification effort.  If you've already validated the use of the iPads in your use environment, perhaps some of your tests involved use of a browser, and you could add an addendum to that validation to specifically reference the browser usage, and that there were not problems noted.

Also, there is no guidance document on browsers.  Any browser use is probably not a serious issue for validation, in that basically what it does is find a URL, or a subject on some server based on what you type in.  Then you in a sense verify the results when you check what was found against your request.  Browsers are not responsible for accuracy of content, which the user is responsible for.  Depending upon how you use it, you may be able to show that there's nothing to verify in its operations.  If so, I'd write that as part of the rationale or justification for not doing a V&V and include it in the file, or make that the brief file.

COTS (Commercial Off The Shelf) Software Question / Answer

Ques: We are in the process of validating a capital equipment, it is a class II medical device. It includes both software and hardware.  The software component is not separable and is not accessible/modifiable but it is the major interface that user can configure/run the device and monitor some parameters. The software also will raise alarms when there is an undesirable situation/risk to the patient. The device was developed overseas and is not have FDA approved.  We were not involved in the validation practices during the design and development of the software, nor we have access to the vendor’s codes and majority of their documentation.  During the webinar you talked about the IQ/OQ/PQ approach in validation of the software. Under the circumstances, is it acceptable to use IQ/OQ/PQ approach as the only validation practice in a 510K FDA submission without any other type of software validation activity/documents?

Ans:  Yes.  The webinar addressed primarily this final stage, integrated / systems level software validation.  You would do a "black box" V&V (verifying software operation by means of the hardware's operation) of hardware and software using IQ, OQ, PQs as described*, but include any information you know about the development methods and in-process test methods used in the development of the custom software.  

Alarms are a particular issue with the FDA.

You would have to use the guidance document I cited, as its required for a submission to the FDA, and compile the 11 documents for your submission (while all are filled out and part of your DHF, not all will be submitted as determined by Level of Concern).  

Where you cannot obtain the necessary information from the vendor, that would be stated under the applicable document, e.g, Design Spec, and Development.  Whatever you can supply, should be included (from any tech manuals, Wikipaedia, web site info, verifiable, etc.).

This is a typical approach with COTS software where you don't have access to the code.

*  IQ -- Software requirements met by hardware; installed properly;

OQ -- Software initializes and shuts down properly. Required features exist and function. Any settings are optimized. 21 CFR 11 issues addressed, exist, operate, if applicable

PQs -- Repeatability and reproducibility of applicable requirements, e.g., screen outputs, alarms, etc, are challenged by worst case runs, multiple samples per run. 

-- John E. Lincoln

COTS (Commercial Off The Shelf) Software Question / Answer

Ques: We are in the process of validating a capital equipment, it is a class II medical device. It includes both software and hardware.  The software component is not separable and is not accessible/modifiable but it is the major interface that user can configure/run the device and monitor some parameters. The software also will raise alarms when there is an undesirable situation/risk to the patient. The device was developed overseas and is not have FDA approved.  We were not involved in the validation practices during the design and development of the software, nor we have access to the vendor’s codes and majority of their documentation.  During the webinar you talked about the IQ/OQ/PQ approach in validation of the software. Under the circumstances, is it acceptable to use IQ/OQ/PQ approach as the only validation practice in a 510K FDA submission without any other type of software validation activity/documents?

Ans:  Yes.  The webinar addressed primarily this final stage, integrated / systems level software validation.  You would do a "black box" V&V (verifying software operation by means of the hardware's operation) of hardware and software using IQ, OQ, PQs as described*, but include any information you know about the development methods and in-process test methods used in the development of the custom software.  

Alarms are a particular issue with the FDA.

You would have to use the guidance document I cited, as its required for a submission to the FDA, and compile the 11 documents for your submission (while all are filled out and part of your DHF, not all will be submitted as determined by Level of Concern).  

Where you cannot obtain the necessary information from the vendor, that would be stated under the applicable document, e.g, Design Spec, and Development.  Whatever you can supply, should be included (from any tech manuals, Wikipaedia, web site info, verifiable, etc.).

This is a typical approach with COTS software where you don't have access to the code.

*  IQ -- Software requirements met by hardware; installed properly;

OQ -- Software initializes and shuts down properly. Required features exist and function. Any settings are optimized. 21 CFR 11 issues addressed, exist, operate, if applicable

PQs -- Repeatability and reproducibility of applicable requirements, e.g., screen outputs, alarms, etc, are challenged by worst case runs, multiple samples per run. 

-- John E. Lincoln