Food and Drug Administration Quality Metrics Reporting Program; Establishment of a Public Docket; Request for Comments, 13295-13299 [2022-04972]

Agencies

• Food and Drug Administration
• DEPARTMENT OF HEALTH AND HUMAN SERVICES

[Federal Register Volume 87, Number 46 (Wednesday, March 9, 2022)]
[Notices]
[Pages 13295-13299]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2022-04972]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Food and Drug Administration

[Docket No. FDA-2022-N-0075]


Food and Drug Administration Quality Metrics Reporting Program; 
Establishment of a Public Docket; Request for Comments

AGENCY: Food and Drug Administration, HHS.

ACTION: Notice; establishment of a public docket; request for comments.

-----------------------------------------------------------------------

SUMMARY: The Food and Drug Administration (FDA or Agency) is announcing 
the establishment of a docket to solicit comments on changes to FDA's 
previously proposed quality metrics reporting program (QM Reporting 
Program). This notice describes considerations for refining the QM 
Reporting Program based on lessons learned from two pilot programs with 
industry that were announced in the Federal Register in June 2018, a 
Site Visit Program and a Quality Metrics Feedback Program, as well as 
stakeholder feedback on FDA's 2016 revised draft guidance for industry 
entitled ``Submission of Quality Metrics Data.'' FDA is interested in 
responses to the questions listed in section III of this document, in 
addition to any general comments on the proposed direction for the 
program. This notice is not intended to communicate our regulatory 
expectations for reporting quality metrics data to FDA but is instead 
intended to seek input from industry to inform the future regulatory 
approach.

DATES: Submit either electronic or written comments by June 7, 2022.

ADDRESSES: You may submit comments as follows. Please note that late, 
untimely filed comments will not be considered. Electronic comments 
must be submitted on or before June 7, 2022. The https://www.regulations.gov electronic filing system will accept comments until 
11:59 p.m. Eastern Time at the end of June 7, 2022. Comments received 
by mail/hand delivery/courier (for written/paper submissions) will be 
considered timely if they are postmarked or the delivery service 
acceptance receipt is on or before that date.

Electronic Submissions

    Submit electronic comments in the following way:
     Federal eRulemaking Portal: https://www.regulations.gov. Follow the instructions for submitting comments. 
Comments submitted electronically, including attachments, to https://

[[Page 13296]]

www.regulations.gov will be posted to the docket unchanged. Because 
your comment will be made public, you are solely responsible for 
ensuring that your comment does not include any confidential 
information that you or a third party may not wish to be posted, such 
as medical information, your or anyone else's Social Security number, 
or confidential business information, such as a manufacturing process. 
Please note that if you include your name, contact information, or 
other information that identifies you in the body of your comments, 
that information will be posted on https://www.regulations.gov.
     If you want to submit a comment with confidential 
information that you do not wish to be made available to the public, 
submit the comment as a written/paper submission and in the manner 
detailed (see ``Written/Paper Submissions'' and ``Instructions'').

Written/Paper Submissions

    Submit written/paper submissions as follows:
     Mail/Hand delivery/Courier (for written/paper 
submissions): Dockets Management Staff (HFA-305), Food and Drug 
Administration, 5630 Fishers Lane, Rm. 1061, Rockville, MD 20852.
     For written/paper comments submitted to the Dockets 
Management Staff, FDA will post your comment, as well as any 
attachments, except for information submitted, marked and identified, 
as confidential, if submitted as detailed in ``Instructions.''
    Instructions: All submissions received must include the Docket No. 
FDA-2022-N-0075 for ``FDA Quality Metrics Reporting Program; 
Establishment of a Public Docket; Request for Comments.'' Received 
comments, those filed in a timely manner (see ADDRESSES), will be 
placed in the docket and, except for those submitted as ``Confidential 
Submissions,'' publicly viewable at https://www.regulations.gov or at 
the Dockets Management Staff between 9 a.m. and 4 p.m., Monday through 
Friday, 240-402-7500.
     Confidential Submissions--To submit a comment with 
confidential information that you do not wish to be made publicly 
available, submit your comments only as a written/paper submission. You 
should submit two copies total. One copy will include the information 
you claim to be confidential with a heading or cover note that states 
``THIS DOCUMENT CONTAINS CONFIDENTIAL INFORMATION.'' The Agency will 
review this copy, including the claimed confidential information, in 
its consideration of comments. The second copy, which will have the 
claimed confidential information redacted/blacked out, will be 
available for public viewing and posted on https://www.regulations.gov. 
Submit both copies to the Dockets Management Staff. If you do not wish 
your name and contact information to be made publicly available, you 
can provide this information on the cover sheet and not in the body of 
your comments and you must identify this information as 
``confidential.'' Any information marked as ``confidential'' will not 
be disclosed except in accordance with 21 CFR 10.20 and other 
applicable disclosure law. For more information about FDA's posting of 
comments to public dockets, see 80 FR 56469, September 18, 2015, or 
access the information at: https://www.govinfo.gov/content/pkg/FR-2015-09-18/pdf/2015-23389.pdf.
    Docket: For access to the docket to read background documents or 
the electronic and written/paper comments received, go to https://www.regulations.gov and insert the docket number, found in brackets in 
the heading of this document, into the ``Search'' box and follow the 
prompts and/or go to the Dockets Management Staff, 5630 Fishers Lane, 
Rm. 1061, Rockville, MD 20852, 240-402-7500.

FOR FURTHER INFORMATION CONTACT: Jean Chung, Center for Drug Evaluation 
and Research, Food and Drug Administration, 10903 New Hampshire Ave., 
Bldg. 75, Rm. 6655, Silver Spring, MD 20993, 301-796-1874, 
[email protected].

SUPPLEMENTARY INFORMATION:

I. Background

A. Quality Metrics

    For pharmaceutical manufacturing, quality metrics are objective 
means of measuring, evaluating, and monitoring the product and process 
life cycle to proactively identify and mitigate quality risks; thereby 
managing operations at higher levels of safety, efficacy, delivery, and 
performance. Quality metrics are used throughout the drug and 
biological product industry to monitor quality control systems and 
processes and drive continuous improvement efforts in manufacturing. 
Quality metrics are important because failure to update and innovate 
manufacturing practices and lack of operational reliability (i.e., 
state of control) can lead to quality problems that have a negative 
impact on public health.
    The minimum standard for ensuring that a manufacturer's products 
are safe and effective is compliance with current good manufacturing 
practice (CGMP) requirements as outlined in current regulations and as 
recommended in current policies (21 CFR parts 210 and 211 for drug 
products and the International Conference on Harmonisation guidance for 
industry entitled ``Q7 Good Manufacturing Practice Guidance for Active 
Pharmaceutical Ingredients'' (September 2016); available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/q7-good-manufacturing-practice-guidance-active-pharmaceutical-ingredients-guidance-industry). However, compliance with CGMP does not necessarily 
indicate whether a manufacturer is investing in improvements and 
striving for sustainable compliance, which is the state of having 
consistent control over manufacturing performance and quality. 
Sustainable CGMP compliance is difficult to achieve without a focus on 
continual improvement.
    An effective Pharmaceutical Quality System (PQS) ensures both 
sustainable CGMP compliance and supply chain robustness. Quality 
metrics data can contribute to a manufacturer's ability to develop an 
effective PQS because metrics provide insight into manufacturing 
performance and enable the identification of opportunities for updates 
and innovation to manufacturing practices. Quality metrics also play an 
important role in supplier oversight and can be used to inform the 
oversight of outsourced activities and material suppliers as well as 
appropriate monitoring activities to minimize supply chain disruptions.
    Quality metrics data provided by establishments can also be useful 
to FDA. These data can assist the Agency in developing compliance and 
inspection policies and practices to improve the Agency's ability to 
predict, and therefore possibly mitigate, future drug shortages, and to 
encourage the pharmaceutical industry to implement innovative quality 
management systems for pharmaceutical manufacturing. For example, 
quality metrics data can be applied to FDA's risk-based inspection 
scheduling, reducing the frequency and/or length of routine 
surveillance inspections for establishments with metrics data that 
suggest sustainable compliance. Additionally, the submission of quality 
metrics data can provide ongoing insight into an establishment's 
operations between inspections.
    As part of FDA's shift towards a risk-based approach to regulation, 
the Agency proposed to develop and implement a QM Reporting Program to 
support its quality surveillance activities, as described in section 
I.B of this notice. Under this program, FDA

[[Page 13297]]

intends to analyze the quality metrics data submitted by establishments 
to: (1) Obtain a more quantitative and objective measure of 
manufacturing quality and reliability at an establishment; (2) 
integrate the metrics and resulting analysis into FDA's comprehensive 
quality surveillance program; and (3) apply the results of the analysis 
to assist in identifying products at risk for quality problems (e.g., 
quality-related shortages and recalls).

B. FDA Guidance for Industry on the Submission of Quality Metrics Data

    In July 2015, FDA issued the draft guidance entitled ``Request for 
Quality Metrics'' (80 FR 44973), which described a potential mandatory 
program for product-based reporting of quality metrics. Under this 
proposed program, manufacturers would have submitted four primary 
metrics (lot acceptance rate (LAR), product quality complaint rate 
(PQCR), invalidated/overturned out-of-specification rate (IOOSR), and 
annual product review (APR) or product quality review on-time rate) and 
three optional metrics (senior management engagement, corrective and 
preventative action (CAPA) effectiveness, and process capability/
performance). Stakeholder comments on the guidance included concerns 
regarding the burden associated with collecting, formatting, and 
submitting data at a product level across multiple establishments; 
technical comments on the proposed metrics and definitions; and legal 
concerns regarding the proposed mandatory program. Stakeholder 
commenters also suggested a phased-in approach to allow learning by 
both industry and FDA.
    In response to this feedback, FDA published a revised draft 
guidance in November 2016 entitled ``Submission of Quality Metrics 
Data'' (81 FR 85226). The 2016 guidance described an initial voluntary 
phase of the QM Reporting Program, with participants reporting data 
either by product or establishment, through an FDA submission portal. 
FDA removed one of the four metrics from the 2015 draft guidance and 
requested submission of the remaining three key metrics: (1) LAR to 
measure manufacturing process performance; (2) IOOSR to measure 
laboratory robustness; and (3) PQCR to measure patient or customer 
feedback and proposed incentives for participation. This guidance also 
described how FDA intended to utilize the submitted data. Stakeholder 
comments on the guidance indicated that the FDA-standardized 
definitions remained a challenge and incentives to participate in a 
voluntary program needed to be strengthened (e.g., direct collaboration 
with FDA to develop the program was an example of a strong incentive). 
Commenters requested a better understanding of the value and utility of 
the data to be submitted to FDA and how FDA would measure success of 
the program. Commenters also expressed a preference for a pilot program 
to gather industry input before implementing a widespread QM Reporting 
Program.

C. Lessons Learned From FDA's Quality Metrics Pilot Programs

    In Federal Register notices issued on June 29, 2018, FDA announced 
the availability of two pilot programs, a Quality Metrics Site Visit 
Program (83 FR 30751) and a Quality Metrics Feedback Program (83 FR 
30748) for any establishment that has a quality metrics program 
developed and implemented by the quality unit and used to support 
product and process quality improvement.
    The Quality Metrics Site Visit Program offered experiential 
learning for FDA staff and provided participating establishments an 
opportunity to explain the advantages and challenges associated with 
implementing and managing a Quality Metrics program. For example, 
participants provided feedback in the form of case studies to 
demonstrate the differences between the metric definitions proposed in 
the FDA draft guidances and definitions commonly used by industry for 
the same metrics. They proposed changes to the definitions, justifying 
why those changes (if any) would be needed. FDA toured the operations 
of 14 establishments worldwide and engaged with establishments on 
topics such as: How quality metrics data are collected, analyzed, 
communicated (e.g., dashboards, business intelligence platforms), and 
reported throughout the organization in a structured and centralized 
manner; how management utilizes quality metrics data to monitor the 
performance of their supply network; how management leverages metrics 
to promote data-driven decisions; how an establishment implements and 
monitors continuous improvements based on metrics; how various quality 
metrics are defined; how actions were taken from observations resulting 
from quality metrics data reviews; and how efforts to proactively 
mitigate and prevent shortages are coordinated.
    In the Quality Metrics Feedback Program, participating 
establishments presented their quality metrics programs to FDA staff. 
The presentations were followed by discussions and knowledge sharing 
that focused on analytical strategies, exploratory data analyses, data 
preparation and structure, and visualizations for communication, as 
well as demonstrations on how FDA plans to analyze the data using 
advanced analytical techniques (e.g., data/text mining, interactive 
visualizations), sophisticated statistical methods (e.g., control 
charts, time series analysis), and machine learning (e.g., predictive 
analytics, natural language processing). In these discussions, FDA also 
obtained feedback on industry's anticipated challenges in applying the 
approach described in FDA's revised draft guidance. Participants had 
the opportunity to submit their quality metrics data through an FDA 
submission portal and provide feedback on their user experience. The 
industry participants represented different sectors of the 
pharmaceutical industry including innovator drug products, generic drug 
products, nonprescription (also known as over-the-counter (OTC)) drug 
products, and biological products.
    The dedicated meetings with industry during the two pilot programs 
that focused on data analytics resulted in the following key lessons 
learned for FDA, which will inform the direction of the QM Reporting 
Program:
    1. Different industry sectors prefer different metrics due to their 
individual operations and business dynamics needs. Therefore, it is 
necessary to implement a program with sufficient flexibility when 
choosing metrics. Identifying critical practice areas (e.g., 
manufacturing process performance) and allowing establishments to 
select appropriate metrics from several options is a more feasible 
approach.
    2. Any metric chosen to be reported should be meaningful to the 
practice area being measured, and the data collected on that metric 
should be able to influence decision making about process improvements 
and capital investments.
    3. In some instances, a combination of metrics rather than a single 
metric is preferred to assess a particular practice area.
    4. The majority of participants prefer to report data at an 
establishment level and have the capability to segment by product, but 
some participants prefer product-level reporting due to their business 
structure (e.g., a vertically integrated company).
    5. Calculating LAR and PQCR based on the definitions in the 2016 
revised draft guidance can result in mathematical discrepancies such as 
rates over 100 percent or invalid calculations (i.e., dividing by 
zero)). These discrepancies are caused by inherent variabilities from 
real-time

[[Page 13298]]

operations (e.g., lots may not be dispositioned in the same quarter in 
which they were started) or how denominators are defined for a 
specified period of time.
    6. While LAR and IOOSR are quality metrics that are routinely 
monitored by establishments, they are not discerning metrics due to 
limited variability over time or limited scope and can result in false 
positives by highlighting nonexistent performance issues. Other metrics 
should be identified as surrogates for manufacturing process 
performance and laboratory robustness. Examples include, but are not 
limited to, right-first-time rate, process capability, and adherence to 
lead time.
    7. The effectiveness of the quality system is a critical component 
of a QM Reporting Program as evidenced by numerous establishments 
collecting data around their PQS. Examples include metrics related to 
the effectiveness of CAPA programs, repeat deviations, maintenance 
programs, and timeliness.
    8. Metrics related to quality culture are important indicators of 
performance and reliability, but unlike other quality metrics, it is 
difficult to capture quality culture at an establishment based on 
numerical metrics alone. Both numerical key performance indicators 
(KPIs) (e.g., APR timeliness and near misses) and qualitative summaries 
(e.g., descriptions of management commitment or quality planning) can 
be used to further understand quality culture.
    9. FDA's analysis of the data submitted during the Quality Metrics 
Feedback Program indicates that the use of statistical quality control 
applications (e.g., statistical process control and process capability) 
and machine learning/natural language processing are appropriate and 
meaningful analytical strategies to assess quality metrics data 
submitted by establishments.

II. Proposed Direction for an FDA QM Reporting Program

    FDA has applied the lessons learned from the pilot programs and 
other stakeholder feedback toward refining the QM Reporting Program 
that was presented in the 2016 revised draft guidance. In this section, 
we summarize a potential direction for the program, and in section III 
we request input on specific aspects of this approach.
    FDA believes that a change in the entities responsible for 
collecting and submitting quality metrics data is not needed. ``Covered 
establishments,'' as defined in the 2016 revised draft guidance, are 
establishments engaged in the manufacture, preparation, propagation, 
compounding or processing of a ``covered drug product'' (products 
subject to an approved application under section 505 of the Federal 
Food, Drug, and Cosmetic Act (FD&C Act) (21 U.S.C. 355) or section 351 
of the Public Health Service Act; legally marketed pursuant to section 
505G of the FD&C Act (21 U.S.C. 355h) (nonprescription drugs marketed 
without an approved drug application); or marketed as unapproved 
finished drug products) or an active pharmaceutical ingredient used in 
the manufacture of a covered drug product. ``Covered establishments'' 
include contract laboratories, contract sterilizers, and contract 
packagers.
    FDA is considering changes to other aspects of the QM Reporting 
Program. Stakeholders have indicated that different industry sectors 
may prefer different quality metrics. To provide flexibility to 
manufacturers, FDA would focus less on standardization of quality 
metrics and definitions. Instead, FDA would identify practice areas 
that are critical to ensure sustainable product quality and 
availability and would permit manufacturers to select a metric(s) from 
each practice area that are meaningful and enable establishments to 
identify continual improvement opportunities. The metric definitions 
would not specify how establishments calculate particular metrics. 
Rather, the reporting establishment would select the most appropriate 
metric(s) from each practice area and inform FDA how it was calculated. 
Through the collective feedback gathered from pilot participants, FDA 
has identified the following four general practice areas as appropriate 
at this time for the QM Reporting Program: (1) Manufacturing Process 
Performance, (2) PQS Effectiveness, (3) Laboratory Performance, (4) 
Supply Chain Robustness. Examples of quality metrics associated with 
each practice include the following:

1. Manufacturing Process Performance

     Process Capability/Performance Indices (Cpk/Ppk): A 
measure that compares the output of a process to the specification 
limits and can be calculated as a proportion (e.g., total number of 
attributes with Ppk greater than 1.33 divided by total number of 
attributes where Ppk is used). It is important to consider standard 
deviation measurements using a reasonable sample size.
     LAR: A measure of the proportion of lots that were 
accepted in a given time period. Examples of inputs that can be used to 
calculate LAR include lots completed, lots dispositioned, lots 
attempted, lots rejected, lots released, lots approved, abandoned lots, 
and parallel/backup lots.
     Right-First-Time Rate: A measure of the proportion of lots 
manufactured without the occurrence of a non-conformance. Examples of 
inputs that can be used to calculate a right-first-time rate include 
number of deviations, lots dispositioned, lots attempted, number of 
nonconformances, and lots approved in the first pass.
     Lot Release Cycle Time: A measure of the amount of time it 
takes for the lot disposition process. Lot release cycle time can be 
calculated with an appropriate unit of measurement such as number of 
hours or days.

2. PQS Effectiveness

     CAPA Effectiveness: A measure of the proportion of CAPA 
plan implemented and deemed effective (i.e., effectiveness 
verifications closed as effective). Examples of inputs that can be used 
to calculate CAPA effectiveness include number of CAPAs initiated, 
CAPAs closed on time, CAPAs closed as ``effective,'' overdue CAPAs, and 
CAPAs resulting in retraining.
     Repeat Deviation Rate: A measure of the proportion of 
recurring deviation measures. Examples of inputs that can be used to 
calculate repeat deviation rate include total number of deviations and 
number of deviations with the same assignable root cause.
     Change Control Effectiveness: A measure of timeliness and 
effectiveness of implemented changes to GMP facilities, systems, 
equipment, or processes. Examples of inputs that can be used to 
calculate this metric include on-time closure of the change, total 
number of late effectiveness checks, total number of changes initiated, 
number of changes that are initiated reactively versus proactively, and 
total number of changes deemed effective.
     Overall Equipment Effectiveness: A measure of operating 
productivity, utilizing planned production time. Overall equipment 
effectiveness can be calculated using inputs related to availability 
(e.g., planned production time, operating time), performance (e.g., 
production capacity), and quality (e.g., production output that does 
not result in acceptable product).
     Unplanned Maintenance: A measure of the proportion of 
maintenance time that was not planned or scheduled. Examples of inputs 
that can be used to calculate this metric include total maintenance 
hours and planned maintenance hours.

[[Page 13299]]

3. Laboratory Performance

     Adherence to Lead Time: A measure of the proportion of 
tests in the laboratory that are completed on time according to 
schedule requirements. Adherence to lead time can be calculated, for 
example, by tracking initiation and testing turnover time in release 
and stability tests (i.e., the number of days between the start date 
and completion date for quality control (QC)); tracking data review and 
documentation; tracking final result reporting prior to batch 
disposition; or comparing QC testing completion date against the target 
date.
     Right-First-Time Rate: A measure of the proportion of 
tests conducted without the occurrence of a deviation. Right-first-time 
rate as a metric for laboratory performance can be calculated, for 
example, by tracking the invalid assay rate, the number of assays 
invalidated due to human errors, or CGMP documentation errors during 
review.
     IOOSR: A measure that indicates a laboratory's ability to 
accurately perform tests. Examples of inputs that can be used to 
calculate this metric include total number of tests conducted and total 
number of out-of-specification results invalidated due to an aberration 
of the measurement process.
     Calibration Timeliness: A measure of a laboratory's 
adherence to inspecting, calibrating, and testing equipment for its 
intended purposes as planned. This metric can be measured by tracking 
calibration criteria and schedules.

4. Supply Chain Robustness

     On-Time In-Full (OTIF): A measure of the extent to which 
shipments are delivered to their destination containing the correct 
quantity and according to the schedule specified in the order. This 
metric can be calculated using inputs such as the number of orders 
shipped, number of past due orders, or number of orders shipped within 
tolerance.
     Fill Rate: A measure that quantifies orders shipped as a 
percentage of the total demand for a given period. Examples of inputs 
that can be used to calculate this metric include total number of 
orders shipped, the number of orders placed, and the number of orders 
received.
     Disposition On-Time: A measure of the proportion of lots 
in which the disposition was carried out on time. Examples of inputs 
that can be used to calculate this metric include the total number of 
lots dispositioned and the total number of lots dispositioned on time.
     Days of Inventory On-Hand: A measure of how a company 
utilizes the average inventory available. It is the number of days that 
inventory remains in stock.
    Given that the majority of participants in the pilot programs 
prefer to report data at an establishment level, FDA is considering an 
approach for aggregating and reporting quality metrics data at the 
establishment level, with the option to segment by manufacturing train, 
product type, or product level (e.g., application number or product 
family).
    Once the data are submitted, FDA intends to analyze the information 
with statistical and machine learning methods to provide useful 
insights for inspection resource allocation. Examples include 
examination of product trends and clusters; exploratory and time-series 
analyses for signal identification, thereby monitoring the health of 
the establishment over time; and utilizing quality metrics data as an 
input into machine learning models to assist in determining an 
establishment's overall PQS effectiveness.

III. Request for Comments

    We are seeking comment on the following aspects of FDA's proposed 
direction for its QM Reporting Program. We note that the questions 
posed in this section are not meant to be exhaustive. We are also 
interested in any other pertinent information that stakeholders and any 
other interested parties would like to provide on FDA's QM Reporting 
Program. FDA encourages stakeholders to provide the rationale for their 
comments, including available examples and supporting information.

A. Reporting Levels

    1. Do you agree that reporting should be aggregated at an 
establishment level?
    2. Would reporting at an establishment level facilitate submission 
of quality metrics data by contract manufacturing organizations?
    3. If you normally assess metrics by product family at an 
establishment, what are useful definitions of ``product family'' from 
your industry sector?

B. Practice Areas and Quality Metrics

    1. If you think the general practice areas listed in section II of 
this notice would not meet the objectives of FDA QM Reporting Program, 
what other practice areas should FDA consider?
    2. If FDA were to consider Quality Culture as one of the general 
practice areas, what are the critical components of a robust quality 
culture and can any of these components be measured quantitatively? If 
so, how do you recommend quality culture information be captured as a 
quantitative metric (e.g., near misses, APR on-time, binary response to 
Quality Culture survey, or other numerical metrics/KPIs)?
    3. Do you think that any of the examples of quality metrics 
proposed by FDA would not be an appropriate measure for the designated 
practice area?
    4. What other metrics should FDA consider for a designated practice 
area?
    5. FDA is interested in an establishment's experience with 
implementing process capability and performance metrics. For example, 
how would you report Cpk and/or Ppk to FDA as part of the QM Reporting 
Program (e.g., reporting Cpk and/or Ppk for certain products, 
aggregated at the establishment level)?
    6. A metric may need to be changed or adjusted by an establishment 
to better monitor PQS effectiveness, inform appropriate business 
strategy, or capture insightful trends, thereby driving continual 
improvement behaviors. What criteria should be applied to justify 
changing or modifying a quality metric (by either the establishment or 
by FDA)? How frequently would you expect changes or modifications to be 
needed?
    7. When would you rely on multiple metrics versus a single metric 
as an indicator when assessing a particular practice area (e.g., two 
metrics are considered in combination because one metric influences the 
other)? What combination of metrics have been meaningful and useful?

C. Other Considerations

    1. Are there considerations unique to specific product categories 
(e.g., generic drug products, OTC drug products, or biological 
products) that should be addressed in the QM Reporting Program?
    2. What would be the optimal reporting frequency for quality 
metrics data submissions (e.g., monthly, quarterly, or yearly, and 
segmented by quarter or month)?
    3. In instances where a manufacturer is not able to extract 
domestic data and its submission to FDA contains both U.S. and foreign 
data, how can these data be submitted to FDA in a manner that would 
still be informative?
    4. Are there any other aspects of FDA's proposed direction for the 
program that FDA should address in future policy documents?

    Dated: February 28, 2022.
Lauren K. Roth,
Associate Commissioner for Policy.
[FR Doc. 2022-04972 Filed 3-8-22; 8:45 am]
BILLING CODE 4164-01-P