Patent Quality Metrics for Fiscal Year 2017 and Request for Comments on Improving Patent Quality Measurement, 16142-16145 [2016-06851]

Download as PDF asabaliauskas on DSK3SPTVN1PROD with NOTICES 16142 Federal Register / Vol. 81, No. 58 / Friday, March 25, 2016 / Notices The Southeast Permits Office proposes to collect additional information on five applications for economic analysis and for purposes of notifying respondents. These data include race, sex, and business type and ownership information, as well as email addresses and the option to provide cellular contact information for digital notifications. The revision will also include a small business certification section, so NMFS can determine if the respondent is a small or large business according to standards established by the Small Business Administration. These proposed revisions will not change the current cost burden but will increase the annual time burden for respondents. Currently, NMFS requires fishermen (respondents) to display one adhesive decal on their vessel indicating that they have a Federal fishing permit in at least one of two Gulf fisheries; the applicable permits are the Charter Vessel/Headboat Permit for Gulf Reef Fish, the Charter Vessel/Headboat Permit for Gulf Coastal Migratory Pelagic fish, and their respective Historical Captain endorsements. NMFS proposes to revise OMB Control Number 0648–0205 to split the single decal covering both fisheries into two decals, with one decal administered with each specific fishery permit or endorsement. In addition, this revision also addresses a new fee of $10 per decal to cover administrative costs, as required by NOAA Finance Handbook, Exhibit 9–1. The Federal Permit Application for Vessels Fishing in the Exclusive Economic Zone would also be revised to reflect the new fee. The decal is currently issued at no cost to permit applicants. These decals allow individuals and law enforcement officials to easily identify vessels that have Federal permits. Affected Public: Business or other forprofit organizations. Frequency: Annually and on occasion. Respondent’s Obligation: Mandatory. This information collection request may be viewed at reginfo.gov. Follow the instructions to view Department of Commerce collections currently under review by OMB. Written comments and recommendations for the proposed information collection should be sent within 30 days of publication of this notice to OIRA_Submission@ omb.eop.gov or fax to (202) 395–5806. Dated: March 22, 2016. Sarah Brabson, NOAA PRA Clearance Officer. [FR Doc. 2016–06803 Filed 3–24–16; 8:45 am] BILLING CODE 3510–22–P VerDate Sep<11>2014 18:30 Mar 24, 2016 Jkt 238001 DEPARTMENT OF COMMERCE United States Patent and Trademark Office [Docket No.: PTO–P–2016–0006] Patent Quality Metrics for Fiscal Year 2017 and Request for Comments on Improving Patent Quality Measurement United States Patent and Trademark Office, Commerce. ACTION: Request for comments. AGENCY: The United States Patent and Trademark Office (USPTO) is revising its patent quality metrics to better identify quality-related issues and more clearly communicate its quality measurements to the public. The new patent quality metrics are part of the USPTO’s Enhanced Patent Quality Initiative (EPQI), which was launched in 2015 to engage patent stakeholders in enhancing patent quality. As part of the Enhanced Patent Quality Initiative, the prior patent quality metrics have been reassessed, and new patent quality metrics are now being designed for adoption for fiscal year 2017. The new patent quality metrics for use in fiscal year 2017 are planned to focus on the correctness and clarity of Office actions and will be applied through a newly unified review process using a standardized review form that will permit data from a significantly larger number of finished product quality reviews conducted at the agency to be aggregated and mined for information. The USPTO will also mine data on transactions during patent prosecution (e.g., the types of actions taken by the applicant and the USPTO) to assess examination processes and identify potential quality issues requiring further study. The review process will apply the new quality metrics and standardized form to increase the accuracy, consistency, transparency, clarity, and simplicity of USPTO quality review procedures. The USPTO is seeking comment from its stakeholders on further improvements to the changes proposed herein. DATES: Comment Deadline Date: To be ensured of consideration in the development of the next iteration of metrics, written comments must be received on or before May 24, 2016. ADDRESSES: Comments should be sent by electronic mail message over the Internet addressed to: QualityMetrics2017@uspto.gov. Comments may also be submitted by postal mail addressed to: Mail Stop Comments—Patents, Commissioner for Patents, P.O. Box 1450, Alexandria, VA, SUMMARY: PO 00000 Frm 00016 Fmt 4703 Sfmt 4703 22313–1450, marked to the attention of Michael Cygan, Senior Legal Advisor, Office of Patent Legal Administration, Office of the Deputy Commissioner for Patent Examination Policy. Although comments may be submitted by postal mail, the Office prefers to receive comments by electronic mail message over the Internet because sharing comments with the public is more easily accomplished. Electronic comments are preferred to be submitted in plain text, but also may be submitted in ADOBE® portable document format or MICROSOFT WORD® format. Comments not submitted electronically should be submitted on paper in a format that facilitates convenient digital scanning into ADOBE® portable document format. Timely filed comments will be available for public inspection at the Office of the Commissioner for Patents, currently located in Madison East, Tenth Floor, 600 Dulany Street, Alexandria, Virginia 22314. Comments also will be available for viewing via the Office’s Internet Web site (http://www. uspto.gov/patent/laws-and-regulations/ comments-public/comments-improvingpatent-quality-measurement). Because comments will be made available for public inspection, information that the submitter does not desire to make public, such as an address or phone number, should not be included in the comments. It would be helpful to the USPTO if comments included information about: (1) The name and affiliation of the individual responding; and (2) an indication of whether the comments represent views of the respondent’s organization or are the respondent’s personal views. FOR FURTHER INFORMATION CONTACT: Michael T. Cygan, Senior Legal Advisor, at (571) 272–7700. Inquiries regarding this notice may be directed to the Office of Patent Legal Administration, by telephone at (571) 272–7701, or by electronic mail at PatentPractice@ uspto.gov. SUPPLEMENTARY INFORMATION: I. Background Prior to fiscal year 2005, the USPTO quality metric was solely directed to the correctness of the final output of the examination process that would result in a patent: An allowed application. During fiscal years 2005 through 2009, the USPTO expanded its review efforts, employing two official metrics of examination quality: (1) The correctness of the examiner’s determination of allowance of an application; and (2) the quality of the actions taken during the E:\FR\FM\25MRN1.SGM 25MRN1 asabaliauskas on DSK3SPTVN1PROD with NOTICES Federal Register / Vol. 81, No. 58 / Friday, March 25, 2016 / Notices course of examination. In fiscal year 2010, the first metric was modified to include final Office actions, and the second metric was modified to focus on the quality of non-final Office actions during prosecution. All quality analysis was performed by random selection of actions for review by a dedicated Office of Patent Quality Assurance (OPQA) team of reviewers, which reviewed each selected action to determine whether all required claim rejections were properly made in compliance with the patent statutes. In 2011, based on stakeholder input, the USPTO adopted a new ‘‘Composite Quality Metric’’ for fiscal years 2011– 2015 to track performance of those aspects that affect quality and provide a single comprehensive metric representing the overall state of patent examination quality. The Composite Quality Metric was composed of seven total factors: (1) The final disposition review, (2) the in-process review, (3) the first action on the merits (FAOM) search review, (4) the complete FAOM review, (5) the external quality survey, (6) the internal quality survey, and (7) an aggregation of five factors from the USPTO’s Quality Index Report (QIR). The first four factors continued the USPTO’s focus on the statutory compliance of work product; i.e., the correctness of the Office actions. The first four factors were derived from the results of reviews of randomly selected Office actions that were conducted by OPQA. These reviews continued the USPTO’s focus on the statutory compliance of work product; i.e., the correctness of the Office actions, with only a basic assessment of whether the examiner had sufficiently set forth his or her position for any claim rejections. The next two factors were derived from surveys that assessed both internal and external stakeholder views on USPTO quality. The final factor was based on the USPTO’s QIR, which measures the degree to which actions in the prosecution of patent applications reveal trends indicative of quality concerns and uses a statistical analysis of occurrences of certain types of events (e.g., reopening after final Office actions, consecutive non-final Office actions, consecutive restriction requirements) based on data available through the USPTO’s Patent Application Locating and Monitoring (PALM) system. Performance in the overall Composite Quality Metric and in each of the component metric factors has been published on the USPTO dashboard Web site on a quarterly basis. The information from the Composite Quality Metric has been used to identify trends VerDate Sep<11>2014 18:30 Mar 24, 2016 Jkt 238001 and areas of concern and to target those areas in need of increased training and/ or resources. On February 5, 2015, the USPTO launched the Enhanced Patent Quality Initiative to improve the quality of patents issued by the USPTO. This initiative began with a request for public comments on a set of six proposals outlined in a Federal Register Notice. See Request for Comments on Enhancing Patent Quality, 80 FR 6475 (Feb. 5, 2015). The USPTO also held a two-day ‘‘Quality Summit’’ on March 25 and 26, 2015, at the USPTO headquarters in Alexandria, Virginia, to discuss the quality concerns of patent stakeholders and to receive feedback on the USPTO’s proposals. Following the Quality Summit, the USPTO has continued its engagement with the public through numerous roadshows, events, and stakeholder meetings to further refine the steps that may be taken to improve quality. The Enhanced Patent Quality Initiative targets three pillars of patent quality: (1) Excellence in work products; (2) excellence in measuring patent quality; and (3) excellence in customer service. In furtherance of the second pillar of patent quality, the USPTO is focusing on improving the internal metrics used to evaluate patent examination quality and on improving the communication of its patent examination quality measurements to the public. Through this initiative, the USPTO has received numerous comments on establishing appropriate quality metrics. The USPTO has considered all of the comments received through the Summit, the Federal Register Notice, and numerous quality outreach events. Based on the information received to date, the USPTO has identified key aspects of quality measurement essential to developing more effective quality metrics. First, the clarity of the examiner’s determinations and the rationale underlying the decisions made in Office actions is an important part of overall patent examination quality and should be emphasized in reviews of USPTO work product. Second, individual metrics that clearly reflect individual aspects of USPTO work product would better communicate patent quality than a single quality composite number that combines scores from unrelated sources such as surveys, procedural efficiency statistics, and substantive patentability compliance reviews. Third, improving the granularity of work product quality measurement to monitor compliance with each statutory provision and enable meaningful data at the work PO 00000 Frm 00017 Fmt 4703 Sfmt 4703 16143 group and art unit level is highly desirable for providing targeted training resulting in greater consistency. Fourth, monitoring the process of examination, i.e., the type and number of actions taken during prosecution as reflected in the QIR, remains a high priority that is best used to spot unusual trends or occurrences that deserve further attention. Lastly, capturing a larger number of finished product quality reviews conducted at the agency and using a standardized review form will lead to a significantly greater number of data points, which will allow for greater consistency in the review of application quality within the Patents Organization. More information on the public comments received on the metrics, and how those are being used to identify improvements to the metrics, is available at http://www.uspto.gov/ patent/initiatives/quality-metrics. In view of these guideposts, a new set of metrics is now being proposed to incorporate these and other improvements to the collection of data and reporting of metrics. II. Improving Measurement of Patent Examination Quality As the next step in advancing the second pillar of the Enhanced Patent Quality Initiative, the proposed fiscal year 2017 patent quality metrics refocus the USPTO’s measurement of the quality of the work products produced from first Office action through final disposition. The proposed metrics continue to assess the correctness of an examiner’s determinations in a given Office action with increased attention on assessing whether the examiner clearly set forth his or her reasoning in a given Office action. In addition, the Office will continue to review the transactions taken during patent prosecution through the QIR, but this information will be used to identify the need for further investigation rather than being measured against a goal. Additionally, the USPTO is changing its reporting of the quality metrics to provide simpler and clearer communication of results to the public. A. Measurement of Statutory Compliance and Clarity in Work Products The patent quality metrics of work product proposed here for fiscal year 2017 provide a tighter focus on measuring two foundational characteristics of patent examination: Statutory compliance and clarity of decision making in Office actions. These proposed patent quality metrics continue to measure correctness of actions in terms of their compliance E:\FR\FM\25MRN1.SGM 25MRN1 asabaliauskas on DSK3SPTVN1PROD with NOTICES 16144 Federal Register / Vol. 81, No. 58 / Friday, March 25, 2016 / Notices with each of the statutory requirements for issuance of a patent. To this end, a sampling of Office actions will continue to be reviewed both for improperly made rejections and for failure to make rejections where required by statute. The substantive review items will also include other items, for example, the propriety of the examiner’s search, any interpretation of claim language under 35 U.S.C 112(f), any determination that an action is made final, any restriction or election of species requirement. Furthermore, the new metrics greatly enhance the review of the clarity of the components of Office actions by including new clarity review items specifically designed for each of the substantive patentability determinations made in Office actions. For example, when reviewing an Office action containing an obviousness rejection under 35 U.S.C. 103, the review items consider not only whether the obviousness rejection was proper, but also whether the statement of the rejection mapped the elements identified in the prior art to the claim limitations, and whether the statement of the rejection explained the reasons for the rejection in a clear manner. The new clarity review items will also include, for example, items directed to the sufficiency of the recordation of any interview and the propriety of any reasons for allowance of an application. For fiscal year 2017, the USPTO is proposing to capture the correctness and clarity review items with a single standardized review form as a repository for all of the review items, replacing the review-specific forms used in the 2011–2015 Composite Quality Metric. The review questions on such a standardized form, colloquially referred to as the ‘‘Master Review Form,’’ is planned to be used by all USPTO reviewers for finished product quality reviews of actions at every stage of prosecution. This Master Review Form will contain the above-described criteria for recording correctness for each of the substantive patentability requirements and for recording the clarity of each of those decisions and the supporting rationales set forth in the Office action under review. The full list of correctness and clarity items in the draft proposed version of the Master Review Form is available for viewing at http://www. uspto.gov/patentquality. The USPTO welcomes and appreciates feedback on the elements of this form through this notice, and will use the input to help finalize the Master Review Form that will be deployed throughout the USPTO in fiscal year 2017. This draft proposed ‘‘Master Review Form’’ was developed as part of the VerDate Sep<11>2014 18:30 Mar 24, 2016 Jkt 238001 Clarity and Correctness Data Capture program, which is part of the USPTO’s Enhanced Patent Quality Initiative. The Clarity and Correctness Data Capture Program has been instituted to better capture the data produced through the different types of reviews within the Patents Organization. Historically, reviews have been performed not only by the quality assurance team, but also by other Technology Center personnel, with each reviewing area setting its own reviewing criteria. Moreover, the only work product reviews recorded for identification of trends were those undertaken by the Office of Patent Quality Assurance. The Master Review Form is designed to provide standardized reviewing criteria for quality reviews of finished work product. Through application of standardized reviewing criteria, the USPTO can better leverage the results from the many levels of review conducted at the agency. The improvements to the data capture process will enable meaningful data analysis at a more granular level than previously possible, permitting valid inferences to be drawn at the workgroup and art unit levels. Through this process, the USPTO and the stakeholders in the patent system will be able to gain a greater understanding of the state of patent prosecution and to work better together towards its improvement. B. Measurement of Transactions During Patent Prosecution A further aspect of the new patent quality metrics will be the leveraging of the data representing the thousands of transactions made by the USPTO during prosecution to reveal information on the quality of the patent prosecution process itself. Transactions during prosecution, such as restrictions, first Office actions, and allowances, are monitored through the USPTO’s PALM system. The USPTO monitors many of these transactions through its QIR. Since 2011, the USPTO has included some of these transactions, such as the number of occurrences of consecutive non-final rejections, as part of its reported quality data. For the proposed 2017 quality metrics, transactional data from the QIR will be used to identify information that can be used to prevent reopening of prosecution, reduce rework, and improve the consistency of decision making throughout the USPTO. Key indicators of the efficiency of prosecution will be instances of reopening of prosecution and repeated non-final Office actions, as well as other instances of rework (e.g., consecutive final Office actions, consecutive PO 00000 Frm 00018 Fmt 4703 Sfmt 4703 restrictions). These indicators do not, by themselves, provide a numerical measure of quality. Rather, these indicators will reveal trends and outlier behavior that will draw attention to potential quality concerns. C. Clearer Reporting of the Metrics In presenting the results of the quality data, the USPTO will seek to further improve the usefulness and transparency of our quality reporting and to communicate the results in a clear and simple manner. The 2011– 2015 Composite Quality Metric, which combined seven different quality variables into a single composite number, will be discontinued. The Quality Index Report will be used to identify potential areas of concern, rather than as providing a single, reportable number. While internal and external surveys will still be performed, the results will not be part of the quality metric, but instead will serve as independent checks on the quality metrics. D. Refinement of Proposed Quality Metrics in FY 2016 Fiscal year 2016 will represent a transitional period for the quality metrics, emphasizing the fine-tuning of the fiscal year 2017 patent quality metrics. The USPTO will test and refine its proposed Master Review Form. This Master Review Form will contain new items, such as additional clarity review items, that will require a period of data collection to create numerical baselines for these items. The Master Review Form will initially be used in targeted reviews to determine the effectiveness of each individual clarity and correctness review item. The transactional data from the QIR will also be reviewed during 2016 to optimize the data analysis therein. Stakeholder comments on the Master Review Form in response to this notice will also form an important part of the process of optimizing the components of the patent quality metrics. During this transitional period, the information gleaned during fiscal year 2016 will be used to produce a finalized set of quality metrics for fiscal year 2017 that will represent the next phase of quality measurement, analysis, tracking, and reporting at the USPTO. III. Feedback Sought on Improving Metrics of Patent Examination Quality The USPTO seeks input and comments from the public through this notice and through public outreach on the following: (1) Is the USPTO moving in the right direction by choosing to focus on two E:\FR\FM\25MRN1.SGM 25MRN1 Federal Register / Vol. 81, No. 58 / Friday, March 25, 2016 / Notices core metrics: A work product metric representing correctness of actions, and a clarity metric that more thoroughly explores the sufficiency of the examiner’s reasoning in an Office action, thus moving away from the prior goal-based quality ‘‘score’’ that reflected not only quality of work product but also results of surveys, used to discover both internal and external stakeholder opinions, and QIR process indicators? Which of the proposed clarity and correctness review items in the proposed standardized ‘‘Master Review Form,’’ available at http://www.uspto. gov/patentquality, should be used as the key drivers of patent examination quality metrics? (2) How can patent metrics best provide objective, rather than subjective, measurements of qualityrelated features in clarity and correctness reviews? In addition to the three questions posed above, the USPTO welcomes comments on any and all areas of quality measurement. Suggestions for rephrased or additional quality metrics review items, especially clarity indicators, are welcomed. The USPTO will consider all submitted comments as it develops the next iteration of quality metrics. For the most current information on this and other patent quality initiatives, please visit the Enhanced Patent Quality Initiative micro site at http://www. uspto.gov/patentquality. Dated: March 22, 2016. Michelle K. Lee, Under Secretary of Commerce for Intellectual Property and Director of the United States Patent and Trademark Office. [FR Doc. 2016–06851 Filed 3–24–16; 8:45 am] FOR FURTHER INFORMATION CONTACT: Barry S. Lineback, Telephone: (703) 603–7740, Fax: (703) 603–0655, or email CMTEFedReg@AbilityOne.gov. Procurement List Proposed Additions and Deletions Deletions AGENCY: On 2/19/2016 (81 FR 8486), the Committee for Purchase From People Who Are Blind or Severely Disabled published notice of proposed deletions from the Procurement List. After consideration of the relevant matter presented, the Committee has determined that the products listed below are no longer suitable for procurement by the Federal Government under 41 U.S.C. 8501–8506 and 41 CFR 51–2.4. Regulatory Flexibility Act Certification I certify that the following action will not have a significant impact on a substantial number of small entities. The major factors considered for this certification were: 1. The action will not result in additional reporting, recordkeeping or other compliance requirements for small entities. 2. The action may result in authorizing small entities to furnish the products to the Government. 3. There are no known regulatory alternatives which would accomplish the objectives of the Javits-WagnerO’Day Act (41 U.S.C. 8501–8506) in connection with the products deleted from the Procurement List. End of Certification Accordingly, the following products are deleted from the Procurement List: Products Procurement List; Deletions Committee for Purchase From People Who Are Blind or Severely Disabled. ACTION: Deletions from the Procurement List. AGENCY: This action deletes products from the Procurement List that were previously furnished by a nonprofit agency employing persons who are blind or have other severe disabilities. DATES: Effective: April 24, 2016. ADDRESSES: Committee for Purchase From People Who Are Blind or Severely Disabled, 1401 S. Clark Street, Suite 715, Arlington, Virginia 22202–4149. asabaliauskas on DSK3SPTVN1PROD with NOTICES SUMMARY: VerDate Sep<11>2014 18:30 Mar 24, 2016 Jkt 238001 COMMITTEE FOR PURCHASE FROM PEOPLE WHO ARE BLIND OR SEVERELY DISABLED SUPPLEMENTARY INFORMATION: BILLING CODE 3510–16–P COMMITTEE FOR PURCHASE FROM PEOPLE WHO ARE BLIND OR SEVERELY DISABLED 16145 NSN(s)—Product Name(s): 7530–00–160–8475—Index Sheet Set, Alphabetical, 8 1⁄2″ x 11″, Buff 7530–00–160–8477—Index Sheet Set, Alphabetical, 11″ x 8 1⁄2″, Buff Mandatory Source of Supply: Life’sWork of Western PA, Pittsburgh, PA Contracting Activity: General Services Administration, New York, NY Barry S. Lineback, Director, Business Operations. [FR Doc. 2016–06827 Filed 3–24–16; 8:45 am] BILLING CODE 6353–01–P PO 00000 Frm 00019 Fmt 4703 Sfmt 4703 Committee for Purchase From People Who Are Blind or Severely Disabled. ACTION: Proposed Additions to and Deletions from the Procurement List. The Committee is proposing to add products to the Procurement List that will be furnished by a nonprofit agency employing persons who are blind or have other severe disabilities and, deletes services previously furnished by such agencies. Comments Must be Received on Or Before: 4/24/2016. ADDRESSES: Committee for Purchase From People Who Are Blind or Severely Disabled, 1401 S. Clark Street, Suite 715, Arlington, Virginia 22202–4149. SUMMARY: FOR FURTHER INFORMATION OR TO SUBMIT COMMENTS CONTACT: Barry S. Lineback, Telephone: (703) 603–7740, Fax: (703) 603–0655, or email CMTEFedReg@ AbilityOne.gov. SUPPLEMENTARY INFORMATION: This notice is published pursuant to 41 U.S.C. 8503 (a)(2) and 41 CFR 51–2.3. Its purpose is to provide interested persons an opportunity to submit comments on the proposed actions. Additions If the Committee approves the proposed additions, the entities of the Federal Government identified in this notice will be required to procure the products listed below from the nonprofit agency employing persons who are blind or have other severe disabilities. The following products are proposed for addition to the Procurement List for production by the nonprofit agency listed: NSN(s)—Product Name(s) 7220–00–NSH–0022—Mat, Floor, Chair, 45″ x 53″ x 0.110″, w/20″ x 12″ Lip 7220–00–NSH–0023—Mat, Floor, Chair, 45″ x 53″ x 0.110″, w/25″ x 12″ Lip 7220–00–NSH–0024—Mat, Floor, Chair, 46″ x 60″ x 0.110″, w/25″ x 12″ Lip 7220–00–NSH–0025—Mat, Floor, Chair, 46″ x 60″ x 0.110″, Without Lip 7220–00–NSH–0026—Mat, Floor, Chair, 60″ x 60″ x 0.110″, Without Lip 7220–00–NSH–0030—Mat, Floor, Chair, 36″ x 48″ x 0.150″, w/20″ x 12″ Lip 7220–00–NSH–0031—Mat, Floor, Chair, 45″ x 53″ x 0.150″, w/25″ x 12″ Lip 7220–00–NSH–0032—Mat, Floor, Chair, 45″ x 53″ x 0.150″, w/20″ x 12″ Lip 7220–00–NSH–0033—Mat, Floor, Chair, 45″ x 53″ x .220″, w/20″ x 12″ Lip E:\FR\FM\25MRN1.SGM 25MRN1

Agencies

[Federal Register Volume 81, Number 58 (Friday, March 25, 2016)]
[Notices]
[Pages 16142-16145]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2016-06851]


-----------------------------------------------------------------------

DEPARTMENT OF COMMERCE

United States Patent and Trademark Office

[Docket No.: PTO-P-2016-0006]


Patent Quality Metrics for Fiscal Year 2017 and Request for 
Comments on Improving Patent Quality Measurement

AGENCY: United States Patent and Trademark Office, Commerce.

ACTION: Request for comments.

-----------------------------------------------------------------------

SUMMARY: The United States Patent and Trademark Office (USPTO) is 
revising its patent quality metrics to better identify quality-related 
issues and more clearly communicate its quality measurements to the 
public. The new patent quality metrics are part of the USPTO's Enhanced 
Patent Quality Initiative (EPQI), which was launched in 2015 to engage 
patent stakeholders in enhancing patent quality. As part of the 
Enhanced Patent Quality Initiative, the prior patent quality metrics 
have been reassessed, and new patent quality metrics are now being 
designed for adoption for fiscal year 2017. The new patent quality 
metrics for use in fiscal year 2017 are planned to focus on the 
correctness and clarity of Office actions and will be applied through a 
newly unified review process using a standardized review form that will 
permit data from a significantly larger number of finished product 
quality reviews conducted at the agency to be aggregated and mined for 
information. The USPTO will also mine data on transactions during 
patent prosecution (e.g., the types of actions taken by the applicant 
and the USPTO) to assess examination processes and identify potential 
quality issues requiring further study. The review process will apply 
the new quality metrics and standardized form to increase the accuracy, 
consistency, transparency, clarity, and simplicity of USPTO quality 
review procedures. The USPTO is seeking comment from its stakeholders 
on further improvements to the changes proposed herein.

DATES: Comment Deadline Date: To be ensured of consideration in the 
development of the next iteration of metrics, written comments must be 
received on or before May 24, 2016.

ADDRESSES: Comments should be sent by electronic mail message over the 
Internet addressed to: QualityMetrics2017@uspto.gov. Comments may also 
be submitted by postal mail addressed to: Mail Stop Comments--Patents, 
Commissioner for Patents, P.O. Box 1450, Alexandria, VA, 22313-1450, 
marked to the attention of Michael Cygan, Senior Legal Advisor, Office 
of Patent Legal Administration, Office of the Deputy Commissioner for 
Patent Examination Policy.
    Although comments may be submitted by postal mail, the Office 
prefers to receive comments by electronic mail message over the 
Internet because sharing comments with the public is more easily 
accomplished. Electronic comments are preferred to be submitted in 
plain text, but also may be submitted in ADOBE[supreg] portable 
document format or MICROSOFT WORD[supreg] format. Comments not 
submitted electronically should be submitted on paper in a format that 
facilitates convenient digital scanning into ADOBE[supreg] portable 
document format.
    Timely filed comments will be available for public inspection at 
the Office of the Commissioner for Patents, currently located in 
Madison East, Tenth Floor, 600 Dulany Street, Alexandria, Virginia 
22314. Comments also will be available for viewing via the Office's 
Internet Web site (http://www.uspto.gov/patent/laws-and-regulations/comments-public/comments-improving-patent-quality-measurement). Because 
comments will be made available for public inspection, information that 
the submitter does not desire to make public, such as an address or 
phone number, should not be included in the comments. It would be 
helpful to the USPTO if comments included information about: (1) The 
name and affiliation of the individual responding; and (2) an 
indication of whether the comments represent views of the respondent's 
organization or are the respondent's personal views.

FOR FURTHER INFORMATION CONTACT: Michael T. Cygan, Senior Legal 
Advisor, at (571) 272-7700. Inquiries regarding this notice may be 
directed to the Office of Patent Legal Administration, by telephone at 
(571) 272-7701, or by electronic mail at PatentPractice@uspto.gov.

SUPPLEMENTARY INFORMATION:

I. Background

    Prior to fiscal year 2005, the USPTO quality metric was solely 
directed to the correctness of the final output of the examination 
process that would result in a patent: An allowed application. During 
fiscal years 2005 through 2009, the USPTO expanded its review efforts, 
employing two official metrics of examination quality: (1) The 
correctness of the examiner's determination of allowance of an 
application; and (2) the quality of the actions taken during the

[[Page 16143]]

course of examination. In fiscal year 2010, the first metric was 
modified to include final Office actions, and the second metric was 
modified to focus on the quality of non-final Office actions during 
prosecution. All quality analysis was performed by random selection of 
actions for review by a dedicated Office of Patent Quality Assurance 
(OPQA) team of reviewers, which reviewed each selected action to 
determine whether all required claim rejections were properly made in 
compliance with the patent statutes.
    In 2011, based on stakeholder input, the USPTO adopted a new 
``Composite Quality Metric'' for fiscal years 2011-2015 to track 
performance of those aspects that affect quality and provide a single 
comprehensive metric representing the overall state of patent 
examination quality. The Composite Quality Metric was composed of seven 
total factors: (1) The final disposition review, (2) the in-process 
review, (3) the first action on the merits (FAOM) search review, (4) 
the complete FAOM review, (5) the external quality survey, (6) the 
internal quality survey, and (7) an aggregation of five factors from 
the USPTO's Quality Index Report (QIR). The first four factors 
continued the USPTO's focus on the statutory compliance of work 
product; i.e., the correctness of the Office actions. The first four 
factors were derived from the results of reviews of randomly selected 
Office actions that were conducted by OPQA. These reviews continued the 
USPTO's focus on the statutory compliance of work product; i.e., the 
correctness of the Office actions, with only a basic assessment of 
whether the examiner had sufficiently set forth his or her position for 
any claim rejections. The next two factors were derived from surveys 
that assessed both internal and external stakeholder views on USPTO 
quality. The final factor was based on the USPTO's QIR, which measures 
the degree to which actions in the prosecution of patent applications 
reveal trends indicative of quality concerns and uses a statistical 
analysis of occurrences of certain types of events (e.g., reopening 
after final Office actions, consecutive non-final Office actions, 
consecutive restriction requirements) based on data available through 
the USPTO's Patent Application Locating and Monitoring (PALM) system. 
Performance in the overall Composite Quality Metric and in each of the 
component metric factors has been published on the USPTO dashboard Web 
site on a quarterly basis. The information from the Composite Quality 
Metric has been used to identify trends and areas of concern and to 
target those areas in need of increased training and/or resources.
    On February 5, 2015, the USPTO launched the Enhanced Patent Quality 
Initiative to improve the quality of patents issued by the USPTO. This 
initiative began with a request for public comments on a set of six 
proposals outlined in a Federal Register Notice. See Request for 
Comments on Enhancing Patent Quality, 80 FR 6475 (Feb. 5, 2015). The 
USPTO also held a two-day ``Quality Summit'' on March 25 and 26, 2015, 
at the USPTO headquarters in Alexandria, Virginia, to discuss the 
quality concerns of patent stakeholders and to receive feedback on the 
USPTO's proposals. Following the Quality Summit, the USPTO has 
continued its engagement with the public through numerous roadshows, 
events, and stakeholder meetings to further refine the steps that may 
be taken to improve quality.
    The Enhanced Patent Quality Initiative targets three pillars of 
patent quality: (1) Excellence in work products; (2) excellence in 
measuring patent quality; and (3) excellence in customer service. In 
furtherance of the second pillar of patent quality, the USPTO is 
focusing on improving the internal metrics used to evaluate patent 
examination quality and on improving the communication of its patent 
examination quality measurements to the public. Through this 
initiative, the USPTO has received numerous comments on establishing 
appropriate quality metrics. The USPTO has considered all of the 
comments received through the Summit, the Federal Register Notice, and 
numerous quality outreach events. Based on the information received to 
date, the USPTO has identified key aspects of quality measurement 
essential to developing more effective quality metrics.
    First, the clarity of the examiner's determinations and the 
rationale underlying the decisions made in Office actions is an 
important part of overall patent examination quality and should be 
emphasized in reviews of USPTO work product. Second, individual metrics 
that clearly reflect individual aspects of USPTO work product would 
better communicate patent quality than a single quality composite 
number that combines scores from unrelated sources such as surveys, 
procedural efficiency statistics, and substantive patentability 
compliance reviews. Third, improving the granularity of work product 
quality measurement to monitor compliance with each statutory provision 
and enable meaningful data at the work group and art unit level is 
highly desirable for providing targeted training resulting in greater 
consistency. Fourth, monitoring the process of examination, i.e., the 
type and number of actions taken during prosecution as reflected in the 
QIR, remains a high priority that is best used to spot unusual trends 
or occurrences that deserve further attention. Lastly, capturing a 
larger number of finished product quality reviews conducted at the 
agency and using a standardized review form will lead to a 
significantly greater number of data points, which will allow for 
greater consistency in the review of application quality within the 
Patents Organization. More information on the public comments received 
on the metrics, and how those are being used to identify improvements 
to the metrics, is available at http://www.uspto.gov/patent/initiatives/quality-metrics. In view of these guideposts, a new set of 
metrics is now being proposed to incorporate these and other 
improvements to the collection of data and reporting of metrics.

II. Improving Measurement of Patent Examination Quality

    As the next step in advancing the second pillar of the Enhanced 
Patent Quality Initiative, the proposed fiscal year 2017 patent quality 
metrics refocus the USPTO's measurement of the quality of the work 
products produced from first Office action through final disposition. 
The proposed metrics continue to assess the correctness of an 
examiner's determinations in a given Office action with increased 
attention on assessing whether the examiner clearly set forth his or 
her reasoning in a given Office action. In addition, the Office will 
continue to review the transactions taken during patent prosecution 
through the QIR, but this information will be used to identify the need 
for further investigation rather than being measured against a goal. 
Additionally, the USPTO is changing its reporting of the quality 
metrics to provide simpler and clearer communication of results to the 
public.

A. Measurement of Statutory Compliance and Clarity in Work Products

    The patent quality metrics of work product proposed here for fiscal 
year 2017 provide a tighter focus on measuring two foundational 
characteristics of patent examination: Statutory compliance and clarity 
of decision making in Office actions. These proposed patent quality 
metrics continue to measure correctness of actions in terms of their 
compliance

[[Page 16144]]

with each of the statutory requirements for issuance of a patent. To 
this end, a sampling of Office actions will continue to be reviewed 
both for improperly made rejections and for failure to make rejections 
where required by statute. The substantive review items will also 
include other items, for example, the propriety of the examiner's 
search, any interpretation of claim language under 35 U.S.C 112(f), any 
determination that an action is made final, any restriction or election 
of species requirement.
    Furthermore, the new metrics greatly enhance the review of the 
clarity of the components of Office actions by including new clarity 
review items specifically designed for each of the substantive 
patentability determinations made in Office actions. For example, when 
reviewing an Office action containing an obviousness rejection under 35 
U.S.C. 103, the review items consider not only whether the obviousness 
rejection was proper, but also whether the statement of the rejection 
mapped the elements identified in the prior art to the claim 
limitations, and whether the statement of the rejection explained the 
reasons for the rejection in a clear manner. The new clarity review 
items will also include, for example, items directed to the sufficiency 
of the recordation of any interview and the propriety of any reasons 
for allowance of an application.
    For fiscal year 2017, the USPTO is proposing to capture the 
correctness and clarity review items with a single standardized review 
form as a repository for all of the review items, replacing the review-
specific forms used in the 2011-2015 Composite Quality Metric. The 
review questions on such a standardized form, colloquially referred to 
as the ``Master Review Form,'' is planned to be used by all USPTO 
reviewers for finished product quality reviews of actions at every 
stage of prosecution. This Master Review Form will contain the above-
described criteria for recording correctness for each of the 
substantive patentability requirements and for recording the clarity of 
each of those decisions and the supporting rationales set forth in the 
Office action under review. The full list of correctness and clarity 
items in the draft proposed version of the Master Review Form is 
available for viewing at http://www.uspto.gov/patentquality. The USPTO 
welcomes and appreciates feedback on the elements of this form through 
this notice, and will use the input to help finalize the Master Review 
Form that will be deployed throughout the USPTO in fiscal year 2017.
    This draft proposed ``Master Review Form'' was developed as part of 
the Clarity and Correctness Data Capture program, which is part of the 
USPTO's Enhanced Patent Quality Initiative. The Clarity and Correctness 
Data Capture Program has been instituted to better capture the data 
produced through the different types of reviews within the Patents 
Organization. Historically, reviews have been performed not only by the 
quality assurance team, but also by other Technology Center personnel, 
with each reviewing area setting its own reviewing criteria. Moreover, 
the only work product reviews recorded for identification of trends 
were those undertaken by the Office of Patent Quality Assurance. The 
Master Review Form is designed to provide standardized reviewing 
criteria for quality reviews of finished work product. Through 
application of standardized reviewing criteria, the USPTO can better 
leverage the results from the many levels of review conducted at the 
agency. The improvements to the data capture process will enable 
meaningful data analysis at a more granular level than previously 
possible, permitting valid inferences to be drawn at the workgroup and 
art unit levels. Through this process, the USPTO and the stakeholders 
in the patent system will be able to gain a greater understanding of 
the state of patent prosecution and to work better together towards its 
improvement.

B. Measurement of Transactions During Patent Prosecution

    A further aspect of the new patent quality metrics will be the 
leveraging of the data representing the thousands of transactions made 
by the USPTO during prosecution to reveal information on the quality of 
the patent prosecution process itself. Transactions during prosecution, 
such as restrictions, first Office actions, and allowances, are 
monitored through the USPTO's PALM system. The USPTO monitors many of 
these transactions through its QIR. Since 2011, the USPTO has included 
some of these transactions, such as the number of occurrences of 
consecutive non-final rejections, as part of its reported quality data. 
For the proposed 2017 quality metrics, transactional data from the QIR 
will be used to identify information that can be used to prevent 
reopening of prosecution, reduce rework, and improve the consistency of 
decision making throughout the USPTO. Key indicators of the efficiency 
of prosecution will be instances of reopening of prosecution and 
repeated non-final Office actions, as well as other instances of rework 
(e.g., consecutive final Office actions, consecutive restrictions). 
These indicators do not, by themselves, provide a numerical measure of 
quality. Rather, these indicators will reveal trends and outlier 
behavior that will draw attention to potential quality concerns.

C. Clearer Reporting of the Metrics

    In presenting the results of the quality data, the USPTO will seek 
to further improve the usefulness and transparency of our quality 
reporting and to communicate the results in a clear and simple manner. 
The 2011-2015 Composite Quality Metric, which combined seven different 
quality variables into a single composite number, will be discontinued. 
The Quality Index Report will be used to identify potential areas of 
concern, rather than as providing a single, reportable number. While 
internal and external surveys will still be performed, the results will 
not be part of the quality metric, but instead will serve as 
independent checks on the quality metrics.

D. Refinement of Proposed Quality Metrics in FY 2016

    Fiscal year 2016 will represent a transitional period for the 
quality metrics, emphasizing the fine-tuning of the fiscal year 2017 
patent quality metrics. The USPTO will test and refine its proposed 
Master Review Form. This Master Review Form will contain new items, 
such as additional clarity review items, that will require a period of 
data collection to create numerical baselines for these items. The 
Master Review Form will initially be used in targeted reviews to 
determine the effectiveness of each individual clarity and correctness 
review item. The transactional data from the QIR will also be reviewed 
during 2016 to optimize the data analysis therein. Stakeholder comments 
on the Master Review Form in response to this notice will also form an 
important part of the process of optimizing the components of the 
patent quality metrics. During this transitional period, the 
information gleaned during fiscal year 2016 will be used to produce a 
finalized set of quality metrics for fiscal year 2017 that will 
represent the next phase of quality measurement, analysis, tracking, 
and reporting at the USPTO.

III. Feedback Sought on Improving Metrics of Patent Examination Quality

    The USPTO seeks input and comments from the public through this 
notice and through public outreach on the following:
    (1) Is the USPTO moving in the right direction by choosing to focus 
on two

[[Page 16145]]

core metrics: A work product metric representing correctness of 
actions, and a clarity metric that more thoroughly explores the 
sufficiency of the examiner's reasoning in an Office action, thus 
moving away from the prior goal-based quality ``score'' that reflected 
not only quality of work product but also results of surveys, used to 
discover both internal and external stakeholder opinions, and QIR 
process indicators? Which of the proposed clarity and correctness 
review items in the proposed standardized ``Master Review Form,'' 
available at http://www.uspto.gov/patentquality, should be used as the 
key drivers of patent examination quality metrics?
    (2) How can patent metrics best provide objective, rather than 
subjective, measurements of quality-related features in clarity and 
correctness reviews?
    In addition to the three questions posed above, the USPTO welcomes 
comments on any and all areas of quality measurement. Suggestions for 
rephrased or additional quality metrics review items, especially 
clarity indicators, are welcomed. The USPTO will consider all submitted 
comments as it develops the next iteration of quality metrics.
    For the most current information on this and other patent quality 
initiatives, please visit the Enhanced Patent Quality Initiative micro 
site at http://www.uspto.gov/patentquality.

    Dated: March 22, 2016.
Michelle K. Lee,
Under Secretary of Commerce for Intellectual Property and Director of 
the United States Patent and Trademark Office.
[FR Doc. 2016-06851 Filed 3-24-16; 8:45 am]
 BILLING CODE 3510-16-P