Question on Yield and OEE in Phama Production Lots

• QUES:  I read in some FDA document about yield. I have seen some warning letters exactly on this topic as well. However, there are many instances where OEE is prepared instead of yield. And classified as Non GMP. Are these 2 different things ? And should OEE be classified as Non-GMP ?

• View John E.’s profile
  

  
  

  ANS:  OEE = Overall Equipment Effectiveness (OEE).  OEE is a measure of how effectively a manufacturing operation is being utilized.  I can't give you an FDA "approved" answer as I have no information / experience from them on that subject.  My personal opinion currently is that I would not recommend substituting one for the other.  Yield is a specific lab calculation(s) required under the CGMPs for various processes in / during and at the end of a lot and logged as part of the batch record. OEE is more a manufacturing process utilization measurement, usually more generalized, which could be part of the CGMPs P&PC "tools" but is not specifically mandated in the CGMPs.  And you have to meet the CGMPs. If you could show a 100% correlation between the two at the level of the requirements of yield and its intervals required in the process for each lot, and the OEE data is handled under the rigors / documentation of the CGMPs, you may have a rationale for such use, but that's highly ify and I can't comment on that vis-a-vis your operation.  

• -- jel@jelincoln.com

 Questions on FDA Registration and 510(k) by an Asian client on a facility move:

QUES 1) The unit which we are delisting, will this be subjected to any USFDA
audit for the production done so far ?

 

ANS:  All production is subject to review / audit by the Agency, usually if there are problems. Once the facility is shut down, such review is only done on the existing records, lot / batch records, logs, validation test reports, etc.  I recommend you do a shut down verification per one of my previous e-mails I just sent you, to verify that everything was operating in specification for the last lot / last item in the last lot produced.

   
QUES 2) The US FDA 510K was first given to the XXXX unit and now is also
available to the YYYY unit. The 510K is very important for us and I am assuming that even if we remove the old unit it will not affect the 510K given to our company.

 

ANS:  The 510(k) remains with the company, unless it's sold with the device. The company's address can change, but that doesn't change the 510(k)'s  ownership.  The FDA however currently has no way to update old 510(k)s on file with them, 'tho they recognize that's a problem.

 

However whenever you change production locations, equipment, processes tied to a 510(k), you are responsible to revalidate the things that changed or moved to prove that the product is still being made the same (and as documented in the 510(k) - see the two guidance documents on changes to devices and the 510(k)'s) and no quality issues have arisen as a result of the move / changes.

-- jel@jelincoln.com 

 Some European Companies Selling in the US Aren't Playing By the Rules

As a consultant working with clients all over the world, I sometimes get push-back to a recommended action using reasoning similar to the following comment from a client:

Most of the pharma facilities in this area are not following the cGMP rules, even though they sell to USA market.  This happens when if they bring in new equipment. The facilities here are almost 20-30 years old and they all seem to have the understanding that if they print the batch record on a paper then they are free from Part 11 requirements. They don't do cleaning after each run.  They claim they only use a paper system, and aren't under 21  CFR Part 11. Some companies are doing API manufacturing in a non-sterile environment, not good aseptic processing . Some are installing new equipment, but with no proper (or only haphazard)  verification / validation, and many other non-conformances. It doesn't appear that the FDA has been here for inspections. So, if my competitors are not obligated to follow the cGMPs, why should I?

I usually answer similarly to the following:

If the companies sell in the US they are required to follow the US FDA CGMPs for their
industry, e.g., 21 CFR 820 for devices, and 21 CFR 210, -211plus for drugs.  They don't need 

to follow 21 CFR11 if they used paper records and manual signatures, BUT all their people 

have to do that.  If some are using the e-records / e-signatures to do CGMP activities / 

records, then it doesn't matter what they say their policy is, they're viewed as under Part 11 

by the US FDA.  

The FDA has 11 resident posts located world-wide, outside the US, 3 in mainland China.  They

are still backlogged due to COVID 19 in inspections, and they base the frequency of

those inspections on the company's products' risk to patient.  But I have worked with some 

EU companies that have had US FDA inspections and gotten Warning Letters, ditto Asia, so 

the FDA is currently inspecting some companies in the EU and Asia.  Those companies had 

previously had great 

notified body audits, incidentally.  The CGMP disconnects I see are usually with the biggest 

problems in CAPA, Trending, andthe definition of risk (the FDA focuses on patient safety (per 1SO 14971), many EU and Asian 

companies also include      

financial, regulatory, schedules under risk as well).  I see a lot of problems with V&V all over the 

world, including in the US.Some validations not holding variables constant, e.g., with written P/Ns and specs, change control,

key variables not validated, no predetermined acceptance criteria set before running the V&V, et al.

Sad to say, I see the same things that you've mentioned.  As a consultant, I usually get called after 

an FDA Inspection, when the company gets a lot of 483 observations and/or a Warning Letter for

for things they should have been doing all along but weren't.

Some current Covid 19 vaccine start-up production issues, drug 

shortages, and baby formula shortage problems here in the US were due to failures in cross-

contamination controls, basic CGMPs. In some cases, they've been misinformed by another consultant.

I try to educate such companies and their personnel, but some feel I'm only requiring some document

to "pad my fee", so, sad to say they'll find out the hard way when they finally are inspected and get a 

Warning Letter, or are shut down after some of their customers in the field get sick or die.

-- jel@jelincoln.com

02/16/25  Added "comment" to first sentence above. Added "(ISO 14971)" to para. 5 above. - JEL

 Equipment Software / Firmware V&V, Part 11 Issues

QUES:  I read some of your articles on IVT network. Content of the articles are very clear and informative. I still have some questions though. Can I ask if you don't mind ? For medical device manufacturing industry, we follow 21 cfr 820, 21 cfr 11. Under 21 cfr 820 there is Device Master Record, Device History Record & Device History File.

 Let's say there are 3 machines used for manufacturing a FDA regulated medical device. 1. Inspection Machine, 2. Assembly Machine, 3. Blister Machine. Q:Will these machines at any stage create electronic Record that we must preserve under FDA regulations? E.g., Recipes/ configurations, Visual Inspection (Quality Related) pictures of Labels, Alarms

ANS:  The CGMPs for devices, 21 CFR 820 require a DMR or recipe for how the device is built (BOM template, references to applicable assembly and test SOPs, drawings, required labeling, traveler template, etc.). The DHR or lot record provides proof that the DMR was followed for that lot, e.g., the BOM is filled out with lot numbers and quantities of each part used, the traveler documents line clearances, machine settings, QC test results, etc.. These records may be kept manually / paper, or electronically. If kept electronically, or if E-SOPs, etc., are used, then 21 CFR 11 becomes operable, and any computer systems used to generate, use, save, change those e-records / e-signatures must be validated, to include Part 11 issues, e.g., audit trails, date / time stamping, et al. That would include the three pieces of equipment you mention; if they generate an e-record to be used in proving CGMP compliance (to 820) to be retained in the DHR, etc., then they have to include Part 11 issues as part of the required validation. Since all equipment must be validated, and all software / firmware must be validated. If the equipment uses software to run, but not to generate records (i.e., you record the information to set or run the equipment and capture any data manually (e.g., manually write info on the traveler), then the equipment and its software would be validated as normally (e.g., IQ, OQ, PQs), but Part 11 elements would not be part of the validation. Part 11 only becomes a requirement if you are using e-records and/or e-signatures in lieu of paper records and/or manual signatures, to prove or satisfy a CGMP requirement / provide a CGMP record.

QUES:  Thanks a lot John for the Answer 🙂 To give you a clearer picture, We have SAP which creates the batch number, expiry date, units to be produced etc and sends to SCADA. Machines receive this information from SCADA. So there is no communication from Machine Control system to SAP. What else SCADA communicates to Machines & vice versa? Recipe Name, good/bad items produced, Machine states(Running/stopped/aborted, etc.) but no Alarms are communicated. Also, there is a significant amount of Machine Parameters configurations (also a part of Machine recipe) needed to produce the Items, remain in machine. It's not recorded anywhere else, only in machine. But machines don't have Electronic signature at any stage. SCADA also maintains the EBR. Like line clearance, goods received etc Electronic signature is maintained in SCADA to validate the changes. We also have Electronic Document Management System where we have DMR, DHF. Change control is maintained in this EDMS.  

ANS:  Basically what you described fits what I sent you previously.  Not only does the computer systems need to be validated per 21 CFR 820, but those computer / electronic systems involved in facilitating and/or  documenting CGMP actions (if any) also need to be validated to 21 CFR 11 on top of the general CGMP validation (I usually add Pt 11 test cases to the OQ).

-- Jel@jelincoln.com

Also, where validated machines generate data that needs to be retained in the GMP record, instead of the normal data integrity requirement of two signature, one entry, one verification, only one signature is required, one verification. -- JEL 09/19/2023

 Human Factors / Usability Engineering Question:

QUES:  As you mentioned on the webinar, the usability engineering is done as needed based on the product. So how should we decide if we need the usability engineering on specific product? Take anesthesia needles as an example, we know this is a mature design on the market for a long time. Is there any way to prove that this similar design won’t cause any use error? By MAUDE or adverse events?"

 

ANS:  The question is -  "How similar is the new design to the old one, as to whether it raises new issues of safety and efficacy, especially around ease / safety of use?" Not -  that a current design precludes an occasional use error, as there will always be an infrequent use error with any device, even one with an optimized design for safety / intuitive use. If the new product's design is substantially unchanged, and has a long field use history, so that it's use / use interface issues would be the same as the predicate's design, then there's no need for a usability engineering study.  Your SOP on HF / UE should discuss and allow these approaches, then you have to follow your SOP. Document your rationale.

FDA's 2016 Guidance on HF / UE:  

"As part of their design controls, manufacturers conduct a risk analysis that includes the risks associated with device use and the measures implemented to reduce those risks. ANSI/AAMI/ISO 14971, Medical Devices – Application of risk management to medical devices, defines risk as the combination of the probability of occurrence of harm and the severity of the potential harm. However, because probability is very difficult to determine for use errors, and in fact many use errors cannot be anticipated until device use is simulated and observed, the severity of the potential harm is more meaningful for determining the need to eliminate (design out) or reduce resulting harm. If the results of risk analysis indicate that use errors could cause serious harm to the patient or the device user, then the manufacturer should apply appropriate human factors or usability engineering processes according to this guidance document. This is also the case if a manufacturer is modifying a marketed device to correct design deficiencies associated with use, particularly as a corrective and preventive action (CAPA)."  

--  https://www.fda.gov/media/80481/download , pg 2, para 1

 

Yes, MAUDE / adverse events would be one of several (see the UE File template example in my slides) possible sources to check.  If there could be a questions as the need to do a UE analysis, and you chose not to, you could do a one or two page plus "UE File" stating the limited analysis done to reach the conclusion that a full-blown study was not needed.

Patient / Device Risk Management (ISO 14971) is always done on new device and major change to existing devices.  Human factors / usability engineering is done as needed, per the above in red.  Each activity is described in a File, which is part of Design Control (or CGMP Change Control) depending on where or when in the product's life cycle the initial design or design change is initiated. 

 

-- jel@jelincoln.com 

To the first paragraph under "answer", added new last sentence on "document your answer." - JEL 05/01/24

 

Mil Std 105E  Sampling Plans Not Dead Yet

Mil Std 105 E was officially cancelled in 1995 and replaced by ANSI/ASQ Z 1.4 for sampling by attributes.  At that time I purchased a copy of Z 1.4 (for a fee ~$199.00, and copyrighted; whereas the Mil Std stated that its "distribution was unlimited" - no fees and no copyright worries), and started to switch my clients over to Z 1.4. But in checking I found that the Z 1.4  sampling plan was identical to the 105E's, so I went back to 105E with those of my clients who hadn't updated / wanted that type of plan, using the105E's plans copied in the SOP's I wrote for them, without having to purchase Z 1.4 / updates for every client and worrying about the associated copyright issues. 

However, one of my clients had an FDA CGMP compliance inspection, and was called out for using 105E, an "obsoleted standard" for acceptance sampling, and the FDA inspector was adamant in the need to change to an alternative. I couldn't convince the inspector that the plans were identical, so, rather than argue, we changed (in a couple of instances, to Nicholas Segura's C=0, also for a fee and copyrighted, and derived from Z 1.4).  

Recently, while updating an QC inspection SOP, I stumbled upon some older Quality Digest articles arguing for the advantages of the old Mil Std105E, and which referenced ASTM E2234, which "carries forward" that standard (e.g., see Wikipedia reference below).  So now I once again have Mil Std 105 E as an option for clients for its sampling plans for QC sampling in CGMP applications in medical device manufacturing, because they have always been and are still valid, but I now had a valid, easy to understand rationale for their continued use.       

"MIL-STD-105 was a United States defense standard that provided procedures and tables for sampling by attributes based on Walter A. Shewhart, Harry Romig, and Harold F. Dodge sampling inspection theories and mathematical formulas. Widely adopted outside of military procurement applications.

The last revision was MIL-STD-105E; it has been carried over in ASTM E2234 - "This practice establishes lot or batch sampling plans and procedures for inspection by attributes using MIL-STD-105E as a basis for sampling a steady stream of lots indexed by acceptance quality limit (AQL). It provides the sampling plans of MIL-STD-105E in ASTM format for use by ASTM committees and others and recognizes the continuing usage of MIL-STD-105E in industries supported by ASTM. This practice also establishes lot or batch sampling plans and procedures for inspection by attributes."

It was officially cancelled in February 1995 by a Notice of Cancellation. This Notice was updated in March 2001 and again in February 2008. The current Notice of Cancellation (Notice 3) recommends that future acquisitions refer to: MIL-STD-1916, "DoD Preferred Methods for Acceptance of Product", or ANSI/ASQ Z1.4, "Sampling Procedures and Tables for Inspection by Attributes"."

-- Wikipedia and ASTM E2234 description (Color added by JEL)

The above information is presented FYI only.

-- jel@jelincoln.com

12/25/2023 - Added additional to ASTM E2234. - JEL

02/14/25  See also   https://www.astm.org/e2234-09r18.html - JEL

 Recent questions on a webinar on Verification and Validation:

My responses:

 

Ques 1: 

In the handouts the term “pre-approval” is mentioned several times. Going through the recording did not fully make this term clear to me. Would be great to have some details on what´s behind this.

Ans 1:  I use the term when I add to a validation test report a pre-approval signature block.  The document is routed to the stakeholders for review and approval before the validation is run, to get changes and "buy-in" to the format and test cases beforehand.  It reduces the problems of getting post-approvals when the signatories didn't have a chance to agree to and sign off on the proposed validation prior to performing it. Your validation SOP would be written to address this feature if you chose to employ it. 

 

Ques 2: 

I would like your assistance with a question raised during a discussion regarding manufacturing process verification/validation:

In one of the implant’s identified critical processes, a visual inspection (under microscope by a technician) is one of the tools used to validate the process.

 The question we are facing is whether the visual inspection should be in a quantitative scale rather than qualitative.

Please bear in mind that the visual inspection is done by a technician who uses specific criteria for approval/disapproval of the implant.

 

Ans 2:  Quantitative is generally better that qualitative, as subjectivity / interpretation elements are greatly reduced.  But qualitative inspections are used frequently.  How you structure the visual inspection to reduce subjectivity / interpretation would be a key consideration.  One can also be used to complement the other for some inspection elements. And monitoring the results over time will provide the best indication of the effectiveness of a chosen approach, a CGMP requirement anyway.

- jel@jelincoln.com

Definition added 08/30/23:  Qualitative observation results cannot be measured while quantitative observation gives measurable data. Quantities like area, height, weight, temperature, weight, time, speed, etc., are examples of quantitative observation while smell, taste, texture, color, etc., are examples of qualitative observation.

Years ago, on specs on/in injection-molded plastic parts, we changed a qualitative "test" (a subjective "too many") to quantitative (mostly; an unacceptable volume to total black specs per set area per the chart) by implementing the use of a TAPPI Dirt Estimation Chart, with a pass/fail range.  See:

https://www.tappi.org/publications-standards/standards-methods/charts--datase/

- JEL 09/19/2023 

Additional justification for "pre-approval": 

 

FDA's Guidance:  "Analytical Procedures and Methods Validation for Drugs and Biologics", issued July 2015, beginning at line 264: 

     "Validation data must be generated under a protocol approved by the sponsor following current   

     good manufacturing practices with the description of methodology of each validation characteristic       and predetermined and justified acceptance criteria, using qualified instrumentation."

-- color added, similar statements can be found in other such documents.  E.g., Google "What is pre-execution approval of validation test scripts?"  -  JEL 10/13/2023