Secretarial Review and Publication of the Annual Report to Congress Submitted by the Contracted Consensus-Based Entity Regarding Performance Measurement, 46695-46731 [2013-18478]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES

[Federal Register Volume 78, Number 148 (Thursday, August 1, 2013)]
[Notices]
[Pages 46695-46731]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2013-18478]



[[Page 46695]]

Vol. 78

Thursday,

No. 148

August 1, 2013

Part II





Department of Health and Human Services





-----------------------------------------------------------------------





Secretarial Review and Publication of the Annual Report to Congress 
Submitted by the Contracted Consensus-Based Entity Regarding 
Performance Measurement; Notice

Federal Register / Vol. 78, No. 148 / Thursday, August 1, 2013 / 
Notices

[[Page 46696]]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES


Secretarial Review and Publication of the Annual Report to 
Congress Submitted by the Contracted Consensus-Based Entity Regarding 
Performance Measurement

    AGENCY: Office of the Secretary of Health and Human Services, HHS.

ACTION: Notice.

-----------------------------------------------------------------------

    SUMMARY: This notice acknowledges the Secretary of the Department 
of Health and Human Services' (HHS) receipt and review of the Annual 
Report submitted to the Secretary and Congress by the contracted 
consensus-based entity (CBE) as mandated by section 1890(b)(5) of the 
Social Security Act, as created by section 183 of the Medicare 
Improvements for Patients and Providers Act of 2008 (MIPPA) and amended 
by section 3014 of the Affordable Care Act of 2010. The statute 
requires the Secretary to review and publish the report in the Federal 
Register together with any comments of the Secretary on the report not 
later than six months after receiving the report. This notice fulfills 
those requirements.

FOR FURTHER INFORMATION CONTACT: Ann Page (202) 260-6473.

I. Background

    Rising health care costs coupled with the growing concern over the 
level of and variation in quality and efficiency in the provision of 
health care raise important challenges for the United States. Section 
183 of MIPPA created Section 1890 of the Social Security Act, which 
requires the Secretary of the Department of Health and Human Services 
(HHS) to contract with a consensus-based entity to perform multiple 
duties pertaining to health care performance measurement. These 
activities support HHS's efforts to promote high-quality, patient-
centered, and financially sustainable health care. The statute mandates 
that the contract be competitively awarded for a period of four years 
and may be renewed under a subsequent bidding process.
    In January, 2009, a competitive contract was awarded by HHS to the 
National Quality Forum (NQF) for a four-year period. The contract 
specified that the CBE should conduct its business in an open and 
transparent manner, provide the opportunity for public comment and 
ensure that membership fees do not pose a barrier to participation in 
the scope of HHS's contract activities, if applicable.
    The HHS four-year contract includes the following major tasks:
    Priority Setting Process: Formulation of a National Strategy and 
Priorities for Health Care Performance--The CBE shall synthesize 
evidence and convene key stakeholders to make recommendations on an 
integrated national strategy and priorities for health care performance 
measurement in all applicable settings. The CBE shall give priority to 
measures that: Address the health care provided to patients with 
prevalent, high-cost chronic diseases; provide the greatest potential 
for improving quality, efficiency and patient-centered health care; and 
may be implemented rapidly due to existing evidence, standards of care 
or other reasons. Additionally, the CBE shall take into account 
measures that: May assist consumers and patients in making informed 
health care decisions; address health disparities across groups and 
areas; and address the continuum of care across multiple providers, 
practitioners and settings.
    Endorsement of Measures: Implementation of a Consensus Process for 
Endorsement of Health Care Quality Measures--The CBE shall provide for 
the endorsement of standardized health care performance measures. This 
process shall consider whether measures are evidence-based, reliable, 
valid, verifiable, relevant to enhanced health outcomes, actionable at 
the caregiver level, feasible to collect and report, and responsive to 
variations in patient characteristics such as health status, language 
capabilities, race or ethnicity, and income level and is consistent 
across types of health care providers including hospitals and 
physicians.
    Maintenance of Consensus Endorsed Measures--The CBE shall establish 
and implement a process to ensure that endorsed measures are updated 
(or retired if obsolete) as new evidence is developed.
    Promotion of the Development of Electronic Health Records--The CBE 
shall promote the development and use of electronic health records that 
contain the functionality for automated collection, aggregation, and 
transmission of performance measurement information. However, in 
January of 2013, this task was repealed and, as a result, removed from 
the CBE's statutory duties by the American Taxpayer Relief Act (Pub. L. 
112-240, Title VI, Sec.  609(a)(2)).
    Convening Multi-Stakeholder Groups--The CBE shall convene multi-
stakeholder groups to provide input into the selection of certain 
categories of quality and efficiency measures, including measures for 
use in certain specific Medicare programs, for use in programs that 
report performance information to the public, and for use in health 
care programs that are not included under the Social Security Act. The 
multi-stakeholder groups consider measures to be implemented through 
the federal rulemaking process for various federal health care quality 
reporting and quality improvement programs including those that address 
certain Medicare services provided through hospices, hospital inpatient 
and outpatient facilities, physician offices, cancer hospitals, end 
stage renal disease (ESRD) facilities, inpatient rehabilitation 
facilities, long-term care hospitals, and psychiatric hospitals and 
home health care programs.
    Annual Report to Congress and the Secretary--Under section 
1890(b)(5)(A) of the Act, by not later than March 1 of each year 
(beginning with 2009) the CBE shall submit to Congress and the 
Secretary of HHS an annual report. The report shall contain a 
description of:
    (i) The implementation of quality and efficiency measurement 
initiatives and the coordination of such initiatives with quality and 
efficiency initiatives implemented by other payers;
    (ii) recommendations on an integrated national strategy and 
priorities for health care performance measurement;
    (iii) performance of its duties required under its contract with 
HHS;
    (iv) gaps in endorsed quality and efficiency measures, which shall 
include measures that are within priority areas identified by the 
Secretary under the National Quality Strategy established under section 
399HH of the Public Health Service Act (National Quality Strategy), and 
where quality and efficiency measures are unavailable or inadequate to 
identify or address such gaps;
    (v) areas in which evidence is insufficient to support endorsement 
of quality and efficiency measures in priority areas identified by the 
Secretary under the National Quality Strategy, and where targeted 
research may address such gaps; and
    (vi) the convening of multi-stakeholder groups to provide input on: 
(1) The selection of quality and efficiency measures from among such 
measures that have been endorsed by the CBE and such measures that have 
not been considered for endorsement by the CBE but are used or proposed 
to be used by the Secretary for the collection or reporting of quality 
and efficiency measures; and (2) national priorities for improvement in 
population health and the delivery of health care services for 
consideration under the National Quality Strategy.

[[Page 46697]]

    Section 1890(b)(5)(B) of the Social Security Act requires 
Secretarial review and publication of this report in the Federal 
Register, together with any comments of the Secretary on the report not 
later than 6 months after receiving the report.
    The first annual report covered the performance period of January 
14, 2009 to February 28, 2009 or the first six weeks post contract 
award. In March 2009, NQF submitted the first annual report to Congress 
and the Secretary of HHS. Given the short timeframe between award and 
the statutory requirement for the submission of the first annual 
report, this first report provided a brief summary of future plans. The 
Secretary published a notice in the Federal Register in compliance with 
the statutory mandate for review and publication of the annual report 
on September 10, 2009 (74 FR 46594).
    In March 2010, NQF submitted to Congress and the Secretary the 
second annual report covering the period of performance of March 1, 
2009 through February 28, 2010. The second annual report was published 
in the Federal Register on October 22, 2010 (75 FR 65340) to comply 
with the statutorily required Secretarial review and publication.
    In March 2011, NQF submitted the third annual report to Congress 
and Secretary of HHS. The third annual report, which covers March 1, 
2010 through February 28, 2011, was published in the Federal Register 
on September 7, 2011 (76 FR 55474).
    In March 2012, NQF submitted its fourth annual report to Congress 
and the Secretary. The report covers the period of performance of 
January 14, 2011 through January 13, 2012. The fourth annual report was 
published in the Federal Register on September 14, 2012 (77 FR 56920).
    In March 2013, NQF submitted its fifth annual report to Congress 
and the Secretary. The report covers the period of performance of 
January 14, 2012 through December 31, 2012. Because the first annual 
report covered only six weeks, there have been five annual reports 
under this four-year contract. This notice complies with the statutory 
requirement for Secretarial review and publication of the fifth NQF 
annual report.

II. March 2013--Consensus-Based Entity Report to Congress and the HHS 
Secretary

    Submitted in March 2013, the fifth annual report to Congress and 
the Secretary spans the period of January 14, 2012 through December 31, 
2012.
    A copy of NQF's submission of the March 2013 annual report to 
Congress and the Secretary of HHS can be found at: https://www.qualityforum.org/Publications/2013/03/2013_NQF_Report_to_Congress.aspx. The fifth NQF annual report is reproduced in section III 
of this notice.

III. NQF Report of 2012 Activities to Congress and the Secretary of the 
Department of Health and Human Services

    This report was funded by the U.S. Department of Health and Human 
Services under contract number: HHSM-500-2009-00010C.

1. Executive Summary

    In the last six years, Congress passed statutes that call upon HHS 
to work with a consensus-based entity (the entity) to facilitate multi-
stakeholder input into (1) setting national priorities for improvement 
in quality and (2) recommending use of performance measures in federal 
programs to achieve these priorities. The statutes also call upon a 
consensus-based entity to review and endorse a portfolio of 
standardized performance measures to be used by stakeholders in public 
and private quality improvement and accountability programs. Note: The 
relevant statutory language appears in italicized text throughout this 
report. The first of these statutes is the 2008 Medicare Improvements 
for Patients and Providers Act (MIPPA) (PL 110-275), which established 
the responsibilities of the consensus-based entity by creating section 
1890 of the Social Security Act and was passed under President Bush. 
The second statute is the 2010 Patient Protection and Affordable Care 
Act (ACA) (Pub. L. 111-148), which modified and added to the consensus-
based entity's responsibilities, and was passed under President Obama. 
The 2013 American Taxpayer Relief Act (Pub. L. 112-240) extended 
funding under the MIPPA statute to the consensus-based entity through 
fiscal year 2013. HHS awarded contracts related to the consensus-based 
entity identified in the statute to the National Quality Forum (NQF). 
As amended by the above laws, the Social Security Act (the Act)--
specifically section 1890(b)(5(A))--also mandates that the entity 
report to Congress and the Secretary of the Department of Health and 
Human Services (HHS) no later than March 1st of each year. The report 
must include descriptions of: (1) How NQF has implemented quality and 
efficiency measurement initiatives under the Act and coordinated these 
initiatives with those implemented by other payers; (2) NQF's 
recommendations with respect to activities conducted under the Act on 
an integrated national strategy and priorities for healthcare 
performance measurement in all applicable settings; (3) NQF's 
performance of the duties required under its contract with HHS; (4) 
gaps in endorsed measures that NQF has identified, including measures 
that are within priority areas identified by the Secretary under HHS' 
national strategy; (5) areas NQF has identified in which evidence is 
insufficient to support endorsement of measures in priority areas 
identified by the National Quality Strategy, and where targeted 
research may address such gaps, and (6) the matters described in 
clauses (i) and (ii) of paragraph (7)(A) of section 1890(b). To address 
the last item, the report will cover the new multi-stakeholder group 
input duties for the consensus-based entity as outlined in section 
3014(a), which created section 1890(b)(7) and (8) of the Act. The first 
of these duties includes providing multi-stakeholder input on the 
selection of quality and efficiency measures both endorsed and those 
not endorsed by the entity, that are used or proposed to be used by the 
Secretary for collection or reporting of quality and efficiency 
measures. The second duty requires that the consensus-based entity 
provide multi-stakeholder group input on national priorities for 
improvement in population health and in the delivery of healthcare 
services for consideration under the National Quality Strategy.
    This fourth Annual Report highlights NQF's work conducted between 
January 14, 2012 and December 31, 2012 related to these statutes and 
conducted under a federal contract with the U.S. Department of Health 
and Human Services.The deliverables produced under contract in 2012 are 
referenced throughout this report, and a full list is included in 
Appendix A.
Facilitating Coordinated Action To Achieve the National Quality 
Strategy
    Section 1890(b)(1) of the Social Security Act mandates that the 
entity shall synthesize evidence and convene key stakeholders to make 
recommendations on an integrated national strategy and priorities for 
healthcare performance measurement in all applicable settings. In 
making such recommendations, the entity shall ensure that priority is 
given to measures: that address the health care provided to patients 
with prevalent, high-cost, chronic diseases; that focus on the greatest 
potential for improving the quality, efficiency, and patient-
centeredness of healthcare; and that

[[Page 46698]]

may be implemented rapidly due to existing evidence and standards of 
care. In addition, the entity will take into account measures: that may 
assist consumers and patients in making informed healthcare decisions; 
address health disparities across groups and areas; and address the 
continuum of care a patient receives, including services furnished by 
multiple healthcare providers or practitioners and across multiple 
settings.
    Under section 1890(b)(5)(A)(ii) of the Social Security Act, the 
entity is mandated to include in the annual report a description of the 
recommendations it has made, with respect to activities conducted under 
the Social Security Act, on an integrated national strategy, and 
priorities for healthcare performance measurement in all applicable 
settings.
    Since 2009, the NQF-convened National Priorities Partnership (NPP) 
has helped to provide multi-stakeholder input into the selection of 
high-impact goals, related priorities, and subsequent strategies that 
constitute the first-ever National Strategy for Quality Improvement in 
Healthcare (NQS). Released in 2011, the NQS outlines three specific 
aims for the U.S. healthcare system--better care, healthy people and 
communities, and affordable care. To achieve these aims, the NQS 
established six priorities to help the healthcare community focus their 
efforts, including:
     Making care safer by reducing harm caused in the delivery 
of care;
     Ensuring that each person and family are engaged as 
partners in their care;
     Promoting effective communication and coordination of 
care;
     Promoting the most effective prevention and treatment 
practices for the leading causes of mortality, starting with 
cardiovascular disease;
     Working with communities to promote wide use of best 
practices to enable healthy living; and
     Making quality care more affordable for individuals, 
families, employers, and governments by developing and spreading new 
healthcare delivery models.
    The NPP is a collaborative public-private partnership of more than 
50 organizations that have a shared stake in how healthcare is 
delivered, received, and paid for. NPP continues to advise HHS on how 
to evolve the NQS' three aims, and its counsel was well reflected in 
HHS's 2012 National Strategy for Quality Improvement in Healthcare, an 
annual NQS progress report required by Congress.
    Beyond forging agreement at the strategic goal level, it is 
challenging to get leaders to implement agreed-upon strategies at the 
care delivery and community level, given limited time and resources. In 
2012, NPP focused on how to advance patient safety by aligning its work 
with HHS' ``Partnership for Patients'' effort. Through a series of web-
based and in-person meetings that NPP hosted throughout 2012, nearly 
2,700 participants from multiple sectors were able to learn about and 
share new improvement approaches, information, tools, and professional 
connections to accelerate their individual contributions to achieving 
safety related improvements. At a more detailed level, NPP developed 
action plans to focus a range of national and local organizations in 
diverse sectors on how to align efforts to reduce preventable 
readmissions and improve maternity care, relying on proven 
interventions. NPP also created a web-based system or ``action 
registry'' to track related commitments to improvement activities 
focused on readmissions and maternity care to enable learning across 
participants. Launched in the fourth quarter of 2012, the registry now 
houses over 50 actions by 30 different organizations.
Endorsing and Maintaining Measures, Related Tools, and Information
    Under section 1890(b)(2) of the Social Security Act, the entity 
must provide for the endorsement of standardized healthcare performance 
measures. As part of the endorsement process, NQF is required to 
consider whether measures are evidence-based, reliable, valid, 
verifiable, relevant to enhanced health outcomes, actionable at the 
caregiver level, feasible for collecting and reporting data, responsive 
to variations in patient characteristics, and consistent across 
healthcare providers. In addition, under section 1890(b)(3), the NQF 
must maintain endorsed measures, by establishing and implementing a 
process to ensure that endorsed measures are retired if obsolete or 
brought up to date as new evidence is developed.
    NQF strategically manages its portfolio of 700-plus endorsed 
measures to increase impact and decrease burden, growing the portfolio 
in some areas and shrinking it in others. More specifically, it 
replaces existing measures with those that are better, reflect new 
medical evidence, or are more relevant; removes measures that are no 
longer effective or where the evidence base has evolved; and expands 
the portfolio to address well-recognized measurement gaps.
    The NQS priorities guide the management of the measure portfolio by 
NQF expert committees. In addition to concentrating on endorsing 
measures suitable for public reporting, performance-based payment, and 
other accountability purposes, NQF evolves its portfolio so that the 
measures are also clinically relevant and actionable for providers. 
Payers and patients are interested in measures that they can use to 
compare and select providers; clinicians and hospitals seek clinically 
relevant measures to benchmark themselves against so they have the 
information they need to focus their improvement efforts for the 
benefit of their patients. A mix of measures is essential to creating 
and continuously evolving a portfolio that meets the needs of diverse 
stakeholders.
    In 2012, NQF completed 16 endorsement projects--reviewing 430 
submitted measures and endorsing 301 measures, or 70 percent. This set 
included 81 new measures and 220 measures that maintained their 
endorsement after being considered in light of new evidence and/or 
against new competing measures submitted to NQF for consideration. The 
newly endorsed measures align with needs identified in the NQS and 
address several critical areas, including patient outcomes, underserved 
populations, healthcare disparities, and hospital readmissions.
    In comparison, NQF completed 11 projects and endorsed 170 measures 
in 2011. This increased productivity can be attributed to efforts to 
make the review process more efficient--the average measure review time 
decreased from 12 months to 7 months during 2012--as well as to other 
enhancements to the endorsement process. Specifically, as part of the 
Consensus Development Process pilot program, NQF provided earlier, more 
detailed feedback to measure developers about a first-order criterion 
(i.e., importance to measure) to further the goal that development 
dollars are spent on measures that are viewed as consequential by the 
field. Furthermore, when a measure is re-evaluated for continued 
endorsement, NQF now requires committees to consider the measure's use 
and whether such use has resulted in improvement or has led to 
unintended consequences, ensuring that committee members are informed 
about the measure's impact.
    Under section 1890(b)(4) of the Social Security Act, the entity has 
been responsible for promoting the development and use of electronic 
health records (EHRs) that contain the functionality for automated 
collection,

[[Page 46699]]

aggregation, and transmission of performance measurement information.
    In an effort to move beyond measures that rely on administrative 
data or that are collected from paper-based medical records, NQF 
continued its work in 2012 to facilitate the development and reporting 
of electronic measures, or eMeasures, that can help accelerate the 
adoption of electronic health records (EHRs). Such efforts include work 
at the granular level (e.g., standardizing data elements so they can be 
collected from varied EHRs to build eMeasures) and at the more 
conceptual level (e.g., the NQF-convened eMeasure Learning 
Collaborative). Created by NQF at the behest of measure developers, EHR 
vendors, HHS, and clinicians, the eMeasure Learning Collaborative is a 
forum for sharing best practices and tackling issues that are barriers 
to developing and implementing eMeasures, such as figuring out how to 
enhance ``upstream'' communication between measure developers and other 
stakeholders so that affected parties have the opportunity to 
collaborate on data requested and its representation in eMeasure logic 
during the measure development process. In 2012, NQF also launched the 
Health IT Knowledge Base and glossary to facilitate a unified 
understanding of terms and measurement approaches used in EHRs and more 
broadly, health IT, and to disseminate best practices, among other 
projects.
Aligning Accountability Measures To Enhance Value
    Under section 1890(b)(1) of the Social Security Act, the entity 
shall synthesize evidence and convene key stakeholders to make 
recommendations and priorities for healthcare performance measurement 
in all applicable settings.
    Under section 1890(b)(5)(A)(i) of the Social Security Act, the 
entity must report on the implementation of quality and efficiency 
measurement initiatives under the Social Security Act and the 
coordination of these initiatives with quality and efficiency 
initiatives implemented by other payers.
    Under section 1890(b)(7) of the Social Security Act, NQF is 
specifically responsible for convening multi-stakeholder groups to 
provide input to the Secretary of HHS on the selection of certain 
categories of NQF-endorsed and non-endorsed quality and efficiency 
measures (measures NQF has not considered for endorsement but the 
Secretary uses or is proposing to use for the collection or reporting 
of quality and efficiency measures). Beginning in 2012, NQF has been 
required to transmit the input of the multi-stakeholder groups to the 
Secretary not later than February 1st of each year. Under section 
1890(a)(5), the Secretary must consider multi-stakeholder input as part 
of a pre-rulemaking process the Secretary must complete prior to the 
adoption of measures during the Federal rulemaking process. NQF 
provides this multi-stakeholder input through its Measure Applications 
Partnership (MAP).
    Agreement about how to define quality, safety, and costs in a 
portfolio of endorsed measures is an important first step toward 
measure alignment, which then needs to be followed by consensus across 
stakeholder groups about the use of endorsed measures.
    The NQF-convened MAP--which comprises stakeholders from a wide 
array of healthcare sectors and 10 federal agencies, as well as 110 
subject matter experts--focuses on recommending measures for federal 
public reporting, payment, and other programs to enhance healthcare 
value. As part of its mission, MAP also strives for alignment with the 
private sector on the use of such measures. In February 2012, MAP 
provided multi-stakeholder input to HHS about the considered use of 
measures in over 17 different federal Medicare benefit programs and the 
Electronic Health Record (EHR) Incentive Program as a part of its first 
annual pre-rulemaking report required by statute. This input was well-
heeded, as evidenced by a degree of concordance--or agreement between 
MAP's recommendations and the Centers for Medicare & Medicaid Services 
(CMS) final rules for quality reporting, public reporting, and value-
based purchasing programs issued in 2012--which averaged 70 percent 
concordance across programs.\1\ Where discordance exists, it appears to 
be due to timing. For example, in some cases, such as the Physician 
Quality Reporting System (PQRS), CMS is moving measures rapidly into a 
program to encourage clinician participation and concurrently 
encouraging that these measures be reviewed by NQF for possible 
endorsement.
    To help guide future measure development related to the NQS and to 
inform use of measures in value-based programs going forward (including 
future annual pre-rulemaking reports to HHS), MAP released a Strategic 
Plan for Measurement in October 2012. A key part of the plan focuses on 
defining the concept of ``families of measures'' in high-impact areas, 
some of which cross conditions and settings. The objective of these 
families, or sets of measures, is to knit together related measures 
currently found in different programs, care settings, levels of 
analysis, and populations to drive improvement and reduce measurement 
burden. In addition, the plan calls for further engagement of 
stakeholders to glean additional feedback about measure use and 
usefulness.
    At the same time, MAP released its Families of Measures report, 
which defines measure families in four key areas--safety, care 
coordination, cardiovascular, and diabetes care--with the goal of 
promoting more cohesion and integration of care regardless of setting, 
provider, level of intensity, or timing. An additional and equally 
important goal is reducing measurement and reporting burden through 
alignment for hospitals, physicians, and other providers as it relates 
to these four areas.
    A 2012 NQF analysis (conducted outside of the federal contract) of 
NQF-endorsed measures in use shows that about 29 percent of measures 
are being used by two or more key stakeholders simultaneously, 
including the federal government, private payers, states, communities, 
and other users. Given its size and reach, the federal government is an 
important driver, using more than half of NQF's measure portfolio in 
its various pay-for-reporting and pay-for-performance programs, 
followed by private payers and states using 41 percent and 28 percent, 
respectively. Further, NQF's analysis shows that alignment in use of 
the same measures increased across these key sectors between 2011 and 
2012.2 3 A 2011 RAND study of 75 organizations revealed a 
strong preference for NQF-endorsed measures where they exist because 
they are vetted, evidence-based, and known to be more credible with 
providers.\4\
Filling Measurement Gaps
    Under section 1890(b)(5)(A)(iv) of the Social Security Act, the 
entity is required to report on gaps in endorsed quality and efficiency 
measures including measures within priority areas identified by HHS 
under the agency's National Quality Strategy, and where quality and 
efficiency measures are unavailable or inadequate to identify or 
address such gaps. Under section 1890(b)(5)(v) of the Social Security 
Act, NQF is also required to report on areas in which evidence is 
insufficient to support endorsement of quality and efficiency measures 
in priority areas identified by the Secretary under the National 
Quality Strategy and where targeted research may address such gaps.
    The science of performance measurement continues to evolve in 
response to the needs and preferences of

[[Page 46700]]

various stakeholders, new and updated data platforms, the capacity of 
providers to collect and report measures, and other factors. In 2012, 
NQF conducted an extensive analysis of its current measures portfolio 
against both the National Quality Strategy priority areas and high-
impact conditions to meet requirements under section 1890(b)(5)(A)(iv) 
of the Social Security Act. This analysis provides a more in-depth 
understanding of what NQF-endorsed measures exist against key strategic 
frameworks, which of these measures are being used in the field, and 
where gaps persist--either because the measures have not yet been 
developed or they are in existence but are not being used.
    The extent to which each NQS priority at the goal level has NQF-
endorsed measures available to drive change is varied but generally 
promising. For example, a large part (40%) of the NQF portfolio 
addresses the important area of patient safety which includes 
healthcare acquired conditions and hospital readmissions. Fewer 
measures (7 percent) address patient and family engagement. Overall, 
measures for specific goals--including shared decision-making, patient 
navigation and self-management, shared accountability, healthy 
lifestyle behaviors, community interventions to improve health, and 
access, cost, and resource use--are less prevalent.
    Looking across both the NQS priority areas and high-impact Medicare 
and child health conditions, the analysis found gaps in measures of 
preventive care, patient-reported outcomes (particularly quality of 
life and functional status), appropriateness (particularly for 
specialty care), access to timely palliative care, and health and 
healthcare disparities. Additionally, the analysis revealed the need 
for better population-level measures to assess improvements in health 
and healthcare. An assessment of the NQF portfolio of endorsed measures 
revealed that while certain high-impact conditions have an abundance of 
measures--e.g., cardiovascular disease, end-stage renal disease, and 
diabetes--many of the high-impact childhood conditions have few or no 
NQF-endorsed measures. Finally, all but one of the 92 NQF-endorsed 
measures in use in federal and at least two other non-federal programs 
address a specific NQS goal or a high-impact condition.
    While certainly there is room for improvement, the analysis 
suggests that the existing portfolio generally addresses agreed upon 
frameworks and that there is alignment in use of such measures across 
various sectors. Going forward, resources should be dedicated to 
delving more deeply into the identified gap areas to prioritize measure 
development and endorsement efforts so that the most needed measurement 
gaps are addressed first.
    Furthermore, NQF's efforts are focused on furthering alignment as 
it relates to measurement strategies to enhance healthcare value 
through its public-private partnerships and its evidence-based, 
consensus-driven method for reviewing and endorsing measures. 
Ultimately, however, for the U.S. healthcare system to be transformed, 
measurement-driven efforts will need to be mutually reinforced with 
changes to current payment and delivery systems that drive the system 
toward greater integration and accountability. Only then will we be 
able to put the U.S. healthcare system on the path to achieving the 
NQS' three, interconnected, and ambitious aims.

2. Facilitating Coordinated Action To Achieve the National Quality 
Strategy

    Section 1890(b)(1) of the Social Security Act mandates that the 
entity shall synthesize evidence and convene key stakeholders to make 
recommendations on an integrated national strategy and priorities for 
healthcare performance measurement in all applicable settings. In 
making such recommendations, the entity shall ensure that priority is 
given to measures: That address the healthcare provided to patients 
with prevalent, high-cost chronic diseases; that have the greatest 
potential for improving the quality, efficiency, and patient-
centeredness of healthcare; and that may be implemented rapidly due to 
existing evidence and standards of care. In addition, the entity will 
take into account measures that may assist consumers and patients in 
making informed healthcare decisions, address health disparities across 
groups and areas, and address the continuum of care a patient receives, 
including services furnished by multiple healthcare providers or 
practitioners and across multiple settings.
    The National Quality Strategy (NQS), released in March 2011, set 
forth a cohesive roadmap for achieving patient-centered, affordable 
care that promotes healthy people and communities (see pages 3-4 for a 
more detailed explanation). Upon its release, its authors emphasized 
that the national quality strategy requires the active engagement and 
support of healthcare stakeholders across the country for quality 
improvements and success.
    For the increasing number of stakeholders that have committed to 
making the NQS a reality, the path and methods to achieve its aims are 
not always apparent. Additionally, as the hard work of achieving care 
of the highest value accelerates, stakeholders are increasingly 
recognizing that performance measurement and quality improvement are 
only achievable by working across sectors and organizations, and they 
seek effective and efficient ways to connect across the healthcare 
delivery system.
    The NPP focused its 2012 efforts on bringing diverse people and 
organizations together in their pursuit of the NQS, and in conducting 
analyses and activities that helped to refine the next critical 
priorities of the healthcare community.
Advising on the National Quality Strategy
    NPP members called for the creation of the NQS and in 2012 
continued to shape its direction by offering input to the HHS 
Secretary. In September 2011, HHS asked the NPP to recommend measures 
for evaluating progress in achieving the NQS. This input was integrated 
into the 2012 National Strategy for Quality Improvement in Healthcare, 
an annual NQS progress report required by Congress. The progress report 
reflected near-universal agreement with NPP recommendations. Multi-
stakeholder input into the NQS and follow-on work to achieve its goals 
embody the spirit of alignment encouraged by the NQS authors, ensuring 
that the strategy is informed, embraced, and viewed as achievable by 
both public and private sectors. Without this shared vision, progress 
is likely to be marred by competing, unfocused, or discordant efforts.
Identifying and Spreading Solutions To Achieve the National Quality 
Strategy
    Under section 1890(b)(5)(A)(i) of the Social Security Act, the 
entity is to provide a description of its implementation of quality and 
efficiency measurement initiatives under the Social Security Act and 
the coordination of those initiatives with those implemented by other 
payers.
    In addition to offering multi-stakeholder input on the NQS, the NPP 
focused on helping to disseminate proven and scalable solutions for its 
implementation; making connections across sectors and between 
organizations; and inspiring people to take highly focused, 
coordinated, and targeted action. Much of this work happened as part of 
the HHS Partnership for Patients patient safety effort, which has two 
ambitious and important goals: reducing hospital-

[[Page 46701]]

acquired conditions by 40 percent and preventable hospital readmissions 
by 20 percent by the end of 2013.
    Establishing the ``who, what, how, and when'' of action is the 
first step in solving large-scale challenges that cut across 
organizations and sectors. To that end, NPP partners and an extended 
network of contributors (more than 750 in total) spent part of 2012 
developing these problem-solving pathways--with an initial focus on 
fashioning shared solutions to improving maternity care and reducing 
preventable readmissions. The NPP selected these two areas for specific 
reasons. Current trends in maternity care and readmissions demonstrate 
an opportunity for improvement that can simultaneously reduce 
unnecessary patient harm and healthcare costs. Both areas also 
represent aspects of healthcare ripe for pooling and focusing the 
efforts of many--patients and families, providers, payers, and 
policymakers, to name a few.
    For example, since 1979, the American Congress of Obstetricians and 
Gynecologists (ACOG) has advocated for the avoidance of elective 
deliveries before 39 completed weeks gestation, yet early elective 
inductions are common in the United States despite the known potential 
harms for mothers and babies.\5\ Similarly, rates of cesarean section 
have risen in recent decades to nearly 32 percent despite potential 
harms, including greater likelihood of asthma for the child. In fact, 
the cesarean rate is rising fastest among women who are least likely to 
benefit--healthy women at low risk of labor and birth complications.\6\ 
Studies reveal that higher cesarean rates do not lead to improved 
outcomes, and rates above 15 percent may do more harm than good.\7\ 
Furthermore, there is strong evidence to support the need to address 
avoidable admissions and readmissions. Almost one in five Medicare 
patients discharged from the hospital is readmitted within 30 days, 
putting patients at increased risk of complications or infections and 
accounting for approximately $15 billion of excess Medicare spending 
each year.8 9 10 While some admissions and readmissions are 
planned and appropriate, approximately 40 percent of hospital 
admissions among nursing home residents may be avoidable.\11\
    In addition to these two specific areas of focus, NPP hosted 
several larger scale forums on behalf of the Partnership for Patients 
in 2012. NPP-hosted forums were designed to identify innovative ways to 
help multiple organizations meet Partnership for Patients' safety goals 
and to help spread proven patient safety interventions. Without these 
exchanges, organizations often find themselves trying to improve in a 
vacuum, working with a limited number of ideas and/or interventions, or 
struggling to innovate given their human and financial resources. The 
structure of these forums, oriented around idea exchanges and sharing 
of case studies and examples, fostered efficient information sharing, 
so that those on the frontlines of improving patient safety were 
supported in their efforts and therefore could more readily effect 
change. More than 400 organizations that support the Partnership for 
Patients attended these events. The first three meetings were focused 
on education regarding the National Quality Strategy and the importance 
of alignment between sectors; catalyzing action; and sharing success 
stories in achieving patient safety. The November 2012 NPP-Partnership 
for Patients event focused exclusively on how to achieve meaningful 
patient and family engagement, which is essential for solving all 
patient safety issues and achieving a patient-centered healthcare 
system. After the first meeting in January 2012, 100 percent of 
attendees felt the meeting enhanced their ability to contribute to 
public-private sector collaboration. NPP augmented the four in-person 
forums with online educational `webinars.' In total, over the course of 
2012, nearly 2,700 people from multiple sectors participated in NQF-
hosted webinars and in-person events in support of the Partnership for 
Patients.
    In 2012, NQF designed a web-based, interactive ``registry'' where 
organizations can share information about their own actions to advance 
the NQS; search data about the actions of others; find partners to work 
with; and learn from others. The registry, available on the NQF Web 
site, allowed for broader engagement, participation, and content that 
facilitates alignment around a focused set of patient safety activities 
and that clarifies who is doing what, when, with whom, and to what end. 
Launched in the fourth quarter of 2012, the registry now houses over 50 
actions by 30 different organizations.
Deliverables Associated With These Activities

----------------------------------------------------------------------------------------------------------------
                                                                  Status (as of 1/7/   Notes/scheduled or actual
            Description                       Output                    2013)                completion date
----------------------------------------------------------------------------------------------------------------
NPP support for Partnership for     4 quarterly convenings for  Completed............  Content of meetings and
 Patients' HHS initiative focused    100+ people each, and 3                            webinars were captured
 on patient safety.                  webinars reaching 550+.                            in individual summaries.
NPP support for Partnership for     2 public web meetings       Completed............  Content of meetings and
 Patients' HHS initiative focused    reaching 500+ and 2                                calls were captured in
 on patient safety.                  public conference calls,                           individual summaries.
                                     reaching 100+.
NPP support for Partnership for     Formed two Action teams     Completed............  .........................
 Patients' HHS initiative focused    around Readmissions and
 on patient safety.                  Maternal Health. Early
                                     development of additional
                                     action teams around
                                     Million Hearts/
                                     Cardiovascular Health and
                                     Patient & Family
                                     Engagement.
NPP support for Partnership for     Created the Action          Completed............  .........................
 Patients' HHS initiative focused    Registry, a virtual space
 on patient safety.                  for organizations to
                                     share their quality
                                     improvement activities--
                                     or ``actions''--around
                                     the six priority areas of
                                     the National Quality
                                     Strategy and make
                                     connections with each
                                     other.
NPP support for Partnership for     Quarterly reports for HHS.  Completed............  .........................
 Patients' HHS initiative focused
 on patient safety.
----------------------------------------------------------------------------------------------------------------


[[Page 46702]]

3. Supporting National Healthcare Measurement Needs

    Under section 1890(b)(2) of the Social Security Act, the entity 
must provide for the endorsement of standardized healthcare performance 
measures. The endorsement process shall consider whether measures are 
evidence-based, reliable, valid, verifiable, relevant to enhanced 
health outcomes, actionable at the caregiver level, feasible for 
collecting and reporting data, responsive to variations in patient 
characteristics, and consistent across healthcare providers. In 
addition, under section 1890(b)(3) of the Social Security Act, the NQF 
must maintain endorsed measures, including retiring obsolete measures 
and bringing other measures up to date.
    Standardized healthcare performance measures help clinicians 
understand whether the care they offered their patients was optimal and 
appropriate, and if not, where to focus their efforts to improve the 
care they deliver. Measures are also used by all types of public and 
private payers for a variety of accountability purposes, including 
feedback and benchmarking, public reporting, and incentive-based 
payment. Lastly, measures are an essential part of making healthcare 
more transparent to all, important for those who receive care or help 
make care decisions for loved ones.
    Working with a variety of stakeholders to build consensus, NQF 
reviews and endorses healthcare performance measures that underpin 
federal and private-sector initiatives focused on enhancing the value 
of healthcare services.
    Ten years ago, NQF endorsed its first voluntary, national consensus 
performance measures to answer the call for standardized measurement of 
healthcare services. These first measures were a stepping-stone for 
creating a consensus-driven effort that bridged nearly every interested 
party in healthcare. The 10-year result of this national experiment is 
a portfolio of more than 700 NQF-endorsed measures, most of which are 
in use; a more information-rich healthcare system; and a substantial 
emerging body of knowledge about measure development, use, and quality 
improvement.
    In the past five years, NQF, working in partnership with HHS and 
others, has focused more intensely on measures that add value and 
reduce burden for those who provide, pay for, and receive care. This 
movement has been facilitated through more stringent evaluation 
criteria that place greater emphasis on evidence and a clear link to 
outcomes, demonstrable impact and gaps in care, and testing that 
demonstrates measures' reliability and validity. NQF also has laid the 
foundation for the next generation of measures, including guidance on 
composite measurement, patient-reported outcome measures, disparities-
sensitive measures, electronic or eMeasures, and measures that evaluate 
complex but important areas such as resource use and population health. 
These activities are intended to inform the path toward targeted, 
prioritized measure development.
    There is increasing evidence that NQF's stringent criteria, 
portfolio management strategies, and collaboration with developers are 
having the desired effect on the portfolio. For example, in 2012 we 
observed the following:
     Guidance that expressed NQF's strong preference for 
outcome measures and that required process measures to demonstrate a 
clear link to outcomes led to more endorsed outcome measures. At the 
end of 2012, 27 percent of the measures in NQF's portfolio were outcome 
measures, compared to 24 and 18 percent in 2011 and 2010, respectively.
     A focus on harmonization resulted in fewer duplicative 
measures, and steering committees selecting the best-in-class measure 
whenever possible.
     Developers submitted more tested measures--which are more 
reliable, valid, and likely to meet NQF endorsement criteria--given 
NQF's increased emphasis on requirements for measure testing. With 
fewer untested measures to evaluate, steering committees were able to 
focus more on evaluating ``better'' measures.
    To apply the concept of constant improvement to its own work, NQF 
conducted in 2012 Lean improvement activities and other initiatives 
and/or projects intended to make the consensus development process more 
predictable, efficient, and navigable for those who develop and 
evaluate measures, while still maintaining the rigor of its multi-
stakeholder process. Measure developers primarily seek an earlier 
window to get broad-based committee input on a measure concept they are 
considering investing in; those who use measures are interested in 
process changes that may further shrink review cycle time while 
maintaining rigor. All parties are focused on ways to make sure finite 
measure development resources are used to meet the greatest measurement 
needs.
    To address these issues, NQF took steps to explore restructuring of 
its Consensus Development Process (CDP) in order to provide early 
guidance to measure developers on whether a measure concept meets NQF's 
criterion for ``importance to measure and report'' before they invest 
time and resources to fully develop and test a measure. The results of 
the pilot project, often referred to as the ``two-stage CDP,'' will be 
available in 2013; results will be used to drive additional 
enhancements that meet the critical needs of measure developers.
    NQF worked to enhance its approach to harmonization, specifically 
helping those who review measures to more consistently and adeptly 
recognize an opportunity for aligning measures. In 2012, NQF also 
conducted work to help committees evaluate measures for usability, a 
criterion for NQF endorsement with which steering committee members 
often struggle during deliberations.
    Lastly, outside of the HHS process improvement activities around 
measure development, NQF created a new multi-stakeholder task force on 
consensus, which, working with NQF staff, led a series of focus groups 
and research exercises to determine a definition of consensus and how 
to establish consensus in rare instances when the NQF membership vote 
is split.
    Results of NQF's Lean improvement work included reducing the 
average measure endorsement cycle time from 12 to 7 months, which is an 
important milestone to ensuring that the measures that matter most to 
our changing healthcare system are available for use as quickly as 
possible all without sacrificing the rigor of the endorsement process. 
Other results included the development of standard work for staff, 
developers, and committee members. This task force on consensus is 
slated to produce findings in early 2013.
Current State of NQF Measures Portfolio: Constricting and Expanding To 
Meet Evolving Needs
    NQF's measure portfolio includes more than 700 performance 
measures, covering a variety of different conditions and care settings. 
The portfolio is carefully managed in a variety of ways. First, working 
with various expert committees, NQF removes or puts into ``reserve 
status'' measures that consistently perform at the highest levels or 
``top out.'' This step signals an improvement success and helps to 
ensure that time is spent instead measuring areas in need of 
improvement. Second, NQF works with those who create measures to 
``harmonize'' related or near-identical measures to eliminate nuanced 
differences. Harmonization is critical to

[[Page 46703]]

reducing measurement burden for providers, who have been inundated with 
various misaligned measurement requests. Successful harmonization may 
result in fewer endorsed measures for providers to report and for 
payers and consumers to interpret. Lastly, where appropriate, NQF works 
with measure developers to replace multiple process measures with more 
meaningful outcome metrics. In 2012, NQF removed 103 measures from its 
portfolio for a variety of reasons: Measures no longer met endorsement 
criteria; measures were harmonized with other similar, competing 
measures; or measure developers chose to retire measures they no longer 
wished to maintain.
    While NQF pursues these proven trimming strategies to make its 
measure portfolio appropriately lean, it also aggressively seeks 
measures from the field that will help to fill known measure gaps and 
to align with the NQS goals. Several important factors motivate NQF to 
expand its portfolio, including: (1) The need for eMeasures; (2) 
pressure for measures that are applicable to multiple clinical 
specialties and settings of care; (3) national pursuit of new payment 
models such as bundled payment; and (4) the need for more advanced 
measures that help close cross-cutting gaps, such as care coordination 
and patient-reported outcomes. The measure portfolio reflects the 
combined ``dynamic yet static'' effect of these strategies: Although 
the portfolio is constantly changing due to new measures cycling in and 
others cycling out, the relative number of endorsed measures remained 
steady in 2012. Specifically, 93 measures were added and 103 measures 
were removed from the portfolio.
    The table below provides a snapshot of how the current NQF-endorsed 
measure portfolio aligns with the NQS, with the percentages reflecting 
the proportion of NQF-endorsed measures that support each of the six 
priorities. Some measures are counted in multiple priority areas. The 
table shows gaps in emerging measurement areas, including 
affordability, patient- and family-centered care, and community health 
and individual well-being. Work conducted in 2012 helped to close these 
known measure gaps and to pave the way for innovative measure 
development by the healthcare field.

                 Measures Compared to NQS Priority Areas
------------------------------------------------------------------------
                                                         Percentage of
                  NQS Priority area                     measures in the
                                                         NQF portfolio
------------------------------------------------------------------------
Safety...............................................                 27
Person- and Family-Centered Care.....................                  5
Prevention and Treatment Practices for Cardiovascular                 15
 Diseases............................................
Communication and Care Coordination..................                 30
Health and Well-Being................................                 15
Affordability........................................                  8
NQF Portfolio........................................                100
------------------------------------------------------------------------

    Furthermore, seven measure developers account for 64 percent of 
NQF's portfolio:

------------------------------------------------------------------------
                                                            Percent of
        Measure seward/developer             Number of         total
                                             measures        portfolio
------------------------------------------------------------------------
1. Centers for Medicare & Medicaid                   123              17
 Services...............................
2. National Committee for Quality                    116              16
 Assurance (NCQA).......................
3. Physician Consortium for Performance              102              14
 Improvement (PCPI).....................
4. Agency for Healthcare Research and                 56               8
 Quality (AHRQ).........................
5. Resolution Health, Inc...............              24               3
6. The Joint Commission.................              24               3
7. ActiveHealth Management..............              23               3
------------------------------------------------------------------------

Specific Measure Endorsement Accomplishments
    In 2012, NQF completed 16 measure endorsement projects--reviewing 
430 submitted measures and endorsing 301. These endorsed measures 
include 81 new measures and 220 measures that NQF expert committees 
concluded could maintain their previous endorsement after being 
reviewed against NQF's criteria and compared to new evidence or 
competing measures. Overall, measures undergoing maintenance were 
endorsed at a rate of 55 percent, and new measures submitted for 
endorsement were endorsed at a rate of 89 percent.
    Case in point: In the last year clinical projects with a large 
number of process measures had markedly lower endorsement rates for 
maintenance measures (e.g., perinatal care, 44 percent; pulmonary, 44 
percent; and renal disease, 36 percent). Newer measurement areas that 
are highly valued by clinicians and patients had higher endorsement 
rates, including disparities measures at 75 percent and palliative care 
at 64 percent. The disparities measures were primarily outcome 
measures, while the palliative measures were primarily process 
measures.
    The measures endorsed by NQF in 2012 align with needs called out in 
the NQS and address several critical areas including patient outcomes, 
hospital readmissions, underserved populations, and healthcare 
disparities. A complete listing on measures and measurement frameworks 
endorsed by NQF in 2012 under contract with HHS is available in 
Appendix A. Highlights include the following:
    Patient-reported experience measures. The healthcare community is 
working toward a more patient-driven system, in which individual needs 
and preferences are incorporated into care decisions. Measures that 
address patient experience, coupled with clinical measures, allow for a 
more comprehensive view of patient care. For example, coupling a 
measure that assesses whether post-surgical instructions for care were 
clear to the patient and his or her caregiver with measures that assess 
hip surgery complication rates creates a more complete picture of a 
patient's experience.
    In 2012, NQF endorsed several measures addressing patient 
experience in various care settings. For example, a measure from the 
American College of Surgeons evaluates patient satisfaction during 
hospitalization for surgical procedures. A measure from the Agency for 
Healthcare Research and Quality focuses on effective provider 
communication with patients regarding disease management, medication 
adherence, and test results. The American Medical Association developed 
seven measures that were endorsed; these measures address concerns such 
as individual health literacy, availability of language services, and 
patient engagement with providers in clinician offices and acute care 
facilities. Finally, measures from the Center for Gerontology and 
Health Care Research and the PROMISE Center evaluate how bereaved 
family members

[[Page 46704]]

perceive the quality of care provided to loved ones in hospices, 
nursing home facilities, and hospitals.
    NQF also convened two expert workshops to explore how patient-
reported outcomes (PROs) can be effectively used in performance 
measurement. Defined as a patient's health status as reported by the 
patient, PROs are seen as the next step forward in building a patient-
centered healthcare system. In the surgical example, a PRO might be 
information gleaned from a patient about when she could resume basic 
activities of daily living, start exercising, or return to work. The 
NQF portfolio already contains some patient-reported outcome measures. 
For example, patient reports are the basis of an NQF-endorsed measure 
of depression remission six months after treatment developed by 
Minnesota Community Measurement. Experiences by community coalitions, 
physician practices, and others implementing PROs helped inform NQF 
expert committees over the past year as they figured out how to 
overcome data, reporting, and methodological barriers to developing and 
using PRO-based performance measures.
    Readmissions measures. About one in five Medicare beneficiaries who 
leaves a hospital is readmitted within 30 days. Such unplanned 
readmissions--many of which are potentially preventable--take a 
significant toll on patients and their families, often resulting in 
prolonged illness or pain, emotional distress, and days of lost work. 
These readmissions also cost Medicare about $15 billion annually.\12\ 
Although Medicare beneficiaries are more likely to be rehospitalized, 
the private sector also spends billions of dollars each year on 
patients who have an unplanned readmission to the hospital within a 
month of an initial stay.
    NQF endorsed two hospital-wide, all-cause readmission measures and 
three condition-specific readmission measures that can help the 
healthcare community better understand and appropriately reduce 
hospital readmission rates. These measures align with major safety and 
affordability issues. However, as performance measures are increasingly 
used in pay-for-performance programs, concerns about the potential for 
unintended consequences, such as a negative impact on providers that 
care for vulnerable populations, have increased. These issues were 
prominent considerations during the 2012 endorsement deliberations over 
the hospital-wide, all-cause readmission measure (NQF measure 
1789), which was ultimately endorsed. To address multiple 
stakeholders' needs and concerns about the newly endorsed readmissions 
measures, the NQF Board of Directors issued guidance regarding the use 
of hospital-wide measures as it ratified the measure:
    Multiple factors affect readmission rates and other measures 
including the complexity of the medical condition and associated 
therapies; effectiveness of inpatient treatment and care transitions; 
patient understanding of and adherence to treatment plans; patient 
health literacy and language barriers; and the availability and quality 
of post-acute and community-based services, particularly for patients 
with low incomes. Readmission measurement should reinforce national 
efforts to focus all stakeholders' attention and collaboration on this 
important issue.
    In response to continued concerns about the use of the new 
hospital-wide, all-cause readmission measure (1789), NQF 
proposed a series of steps to take place after endorsement of that 
particular measure, including monitoring implementation; employing an 
expert multi-stakeholder group to review ``dry run'' data provided by 
CMS regarding measure 1789; evaluating new readmission 
measures for new conditions; and establishing ongoing monitoring 
approaches that ensure that more systematic feedback from measure users 
is integrated into endorsement deliberations. NQF also reviewed updates 
to the readmission measures to remove planned readmissions from the 
condition-specific measures that are generally not considered signals 
of quality, and is continuing efforts to harmonize hospital and health 
plan all-cause readmission measures.
    Patient safety measures. Americans are exposed to more preventable 
medical errors than patients in other industrialized nations, costing 
the United States close to $29 billion per year in additional 
healthcare expenses, lost worker productivity, and disability.\13\ 
These costs are passed on in a number of ways, including higher 
insurance premiums and taxes and lost wages. Proactively addressing 
medical errors and unsafe care will help to protect patients from harm, 
lead to more effective and equitable care, and appropriately reduce 
costs.
    NQF endorsed 32 patient safety measures in 2012, focusing on 
complications such as healthcare-associated infections, falls, 
medication safety, and pressure ulcers. These measures closely align 
with goals of the Partnership for Patients to make care safer.
    Resource use measures. Healthcare expenditures in the United States 
are unmatched by any other country. This spending, however, has not 
resulted in better health for Americans. In general, the United States 
lags behind other countries in terms of mortality, patient 
satisfaction, access to care, or quality of care within the healthcare 
system.14 15 16 Patients, insurers, state and regional 
leaders, federal policymakers, employers, and providers are all attuned 
to affordability and increasingly focused on how we can measure and 
reduce healthcare expenditures without harming patients.
    NQF endorsed its first set of resource use measures--designed to 
understand how healthcare resources are being used--in January 2012, 
and it endorsed an additional set in April 2012. These measures will 
offer a more complete picture of what drives healthcare costs from 
several perspectives. For example, one endorsed measure evaluates a 
primary care provider's risk-adjusted frequency and intensity of all 
services used to manage patients--including inpatient/outpatient, 
pharmacy, laboratory, radiology, and behavioral health services--using 
standardized prices. Another measure evaluates a primary care 
provider's risk-adjusted cost effectiveness at managing his patient 
population using actual prices paid by health plans. Similar measures 
also evaluate total resources used by individual patients with specific 
conditions, such as asthma and chronic obstructive pulmonary disease, 
over the course of a measurement year. And other measures evaluate 
total costs over an episode of care, such as costs associated with hip/
knee replacement, from diagnosis to treatment to rehabilitation. Used 
in concert with quality measures, these resource use measures will 
enable stakeholders to identify opportunities for creating a higher 
value healthcare system.
    Harmonized behavioral health measures. In 2012, NQF endorsed 10 
measures related to mental health and substance abuse, including 
measures of treatment for individuals experiencing alcohol or drug 
dependent episodes; diabetes and cardiovascular health screening for 
people with schizophrenia or bipolar disorder; and post-care follow-up 
rates for hospitalized individuals with mental illness. As a part of 
this process, NQF also brought together CMS and NCQA to harmonize two 
related measures into one measure addressing antipsychotic medication 
adherence in patients with schizophrenia.
    A multiple chronic conditions measurement framework. People with

[[Page 46705]]

multiple chronic conditions (MCCs) now comprise more than 25 percent of 
the U.S. population17 18 and this number is expected to 
grow. This population is more likely to see multiple clinicians, take 
five or more medications, and receive care that is fragmented, 
incomplete, inefficient, and ineffective.19 20 21 22 23 They 
are at significantly higher risk of adverse outcomes and complications.
    Despite the growing prevalence of people with MCCs, existing 
quality measures typically do not address issues associated with the 
care for individuals with MCCs, largely because of data sharing 
challenges and because measures are typically limited to addressing a 
singular disease and/or specific setting. As a result, NQF endorsed a 
measurement framework that establishes a shared vision for effectively 
measuring the quality of care for individuals with MCCs. Measure 
developers can use this framework to more quickly create measures for 
this population, filling a current measurement gap.
    Healthcare disparities measures. Research from the Institute of 
Medicine shows that racial and ethnic minorities often receive lower 
quality care than their white counterparts, even after controlling for 
factors such as insurance coverage, socioeconomic status, and 
comorbidities.\24\ Such disparities are exacerbated by additional 
factors, including that racial and ethnic minorities have poorer health 
status in general, face more barriers to care, and are more likely to 
have poor health literacy.
    With funding from the Robert Wood Johnson Foundation, NQF 
established a more detailed picture of how to approach measurement of 
healthcare disparities across settings and populations, beginning with 
a commissioned paper outlining methodological concerns. To ensure that 
disparities in care can be addressed most effectively, NQF developed an 
approach to identify measures that are more sensitive to disparities 
and, as such, should be stratified. From there, NQF endorsed 12 
performance measures that focused on patient-provider communication, 
cultural competence, and language services, among other issues. Now 
that these measures are endorsed, HHS has more opportunity to include 
these kinds of measures, which address a key NQS measurement priority, 
in federal programs.
Streamlining Measure Information
    Various healthcare entities gather, store, and need to access 
information about performance measures. Over the years, different 
measure information systems have been built, each with differing 
purposes, structure, and content. This diversity of places and 
approaches to storing such information confounds the ability to find 
and coordinate pieces of information about a given measure, such as a 
specific version, unique identifying number or name, specifications, 
purpose and context, and benchmarking results.
    HHS asked NQF to use its role as a neutral convener to work with a 
variety of public- and private-sector organizations to conduct a 
``Registry Needs Assessment.'' The assessment was geared toward 
understanding how various stakeholders currently approach gathering and 
storing performance measure information; assessing the desirability of 
a different approach including but not limited to a single ``measure 
registry'' system; and identifying the barriers to achieving more 
aligned and definitive ways to store and access consistent and 
comprehensive information about measures. The findings included 
recommendations for first steps such as developing shared definitions 
of measure ``metadata'' and versioning standards to enable alignment of 
measure information.
The Global to the Granular: NQF's Role in Accelerating the Adoption of 
eMeasures
    Under section 1890(b)(4) of the Social Security Act, the entity was 
tasked with promoting the development and use of electronic health 
records that contain the functionality for automated collection, 
aggregation, and transmission of performance measurement information.
    Currently, healthcare data largely live within system silos and on 
paper rather than in electronic form, which makes it nearly impossible 
for data to follow patients through various settings in which they 
receive care. Healthcare is safer and better coordinated when 
electronic health records (EHRs) and other clinical information 
technology systems reliably capture and share data across providers and 
patients to facilitate care--and as a byproduct of the clinical 
process--generate performance measurement information. Wide adoption of 
this kind of electronic infrastructure will spur implementation of the 
NQS, but has been hampered by a variety of issues.
    NQF's health IT work in 2012 focused on pulling together disparate 
organizations that play a role in moving quality from a paper-based 
world to one facilitated by technology. The faster we reach consensus 
on approaches to this new world, the faster we may achieve the goal of 
a fully empowered and connected electronic information system designed 
with the patient in mind.
    At the global level, NQF launched a series of activities designed 
to promote shared understanding among those involved in advancing 
electronic measurement and data infrastructure. It convened the 
eMeasure Learning Collaborative, a new environment for promoting best 
practices related to development and implementation of measures applied 
to electronic data sources (i.e., eMeasures). eMeasures are an 
innovation in advancing quality measurement, but significant barriers 
hamper their wider scale creation, adoption, and use. Through two in-
person meetings and other virtual convenings, NQF brought together 
hundreds of stakeholders including government representatives, EHR 
vendors, measure developers, clinicians, and hospitals--creating a 
unique forum for these parties to work together on new eMeasurement 
approaches.
    Specific eMeasure best practices emerged from this Learning 
Collaborative, particularly in three areas: Organizational leadership, 
data representation and clinical workflow, and learning health systems. 
For example, regarding data representation, all participants identified 
the need for measure developers and other stakeholders to communicate 
earlier in the eMeasurement process, particularly when measure 
developers are selecting data and representing data in eMeasure logic. 
For this best practice to become a reality, a national structure and 
process must exist to enable this level of dialogue. With respect to 
organizational leadership, participants suggested that provider 
organizations create inter-professional, physician-led teams focused on 
an integrated approach to eMeasure adoption, including data capture, 
reporting, workflow, clinical decision support, and evidence-based 
practice.
    Several of NQF's 2012 projects sought to facilitate a unified 
understanding of terms and measurement approaches used in the health IT 
field, so that measure developers and implementers, health IT vendors, 
standards organizations, and other users of eMeasures and tools work 
with a similar lexicon. For example, NQF launched the Health IT 
Knowledge Base, providing answers to some of the most common technical 
questions about NQF's related initiatives. Since August 2012, NQF added 
more than 70 new entries to the frequently asked questions section, 
stemming from its interactions with

[[Page 46706]]

eMeasure users and developers. NQF also added a glossary with more than 
150 terms and definitions. As a complement to the Knowledge Base, NQF 
provided opportunities for stakeholders to learn about best practices 
in eMeasurement through a series of NQF-hosted health IT webinars that 
reached more than 1,400 people during the past 12 months.
    As quality measurement shifts to an electronic platform, additional 
clarity is needed regarding the testing that assures that eMeasures can 
be used for a range of accountability applications, which require both 
precision and reliable and valid results. NQF worked with CMS and the 
Office of the National Coordinator for Health Information Technology 
(ONC) to ensure that the data capture for eMeasures is feasible without 
impeding clinical workflow. NQF's health IT initiatives in 2012 scaled 
down to the granular level as well, to help standardize the efforts of 
the creators and users of eMeasures. Developed by NQF, the Quality Data 
Model (QDM) is an ``information model'' that defines concepts used in 
quality measures and clinical care in a way that allows the information 
to be collected automatically from data already stored in an EHR.
    An example illustrates how the QDM can simplify and standardize the 
electronic collection and reporting of quality measures. If a 
physician's office wants to use its EHR to report on a measure that 
assesses the percentage of patients with a diagnosis of coronary artery 
disease (CAD) who were prescribed a lipid-lowering therapy, the EHR 
must first identify the patients with CAD within the physician's 
practice and then determine whether the patients had the therapy. If 
the physician's performance is going to be compared to her peers, then 
her EHR must define these elements in exactly the same way as every 
other EHR. The QDM supports this type of query regardless of the type 
of EHR by defining the necessary standard data elements (e.g., active 
diagnosis, active medication administered/ordered/dispensed) and the 
type of coding that the EHR may use to express the result (e.g., ICD-9 
code for diagnosis; RxNorm for medication, etc.). When all measure 
specifications are written in a common way, EHR vendors can more easily 
ensure that their EHRs can support quality measurement, and the 
validity of electronic-based reporting programs will likely increase. 
NQF released an updated version of the QDM in December 2012, which 
focused on simplifying and standardizing QDM measure logic to support 
implementation of the federal Meaningful Use regulations. NQF also 
regularly receives ongoing feedback and insights into best practices 
from a User Group of measure developers, physicians, hospitals, and EHR 
vendors who are currently actively involved in eMeasure use.
    NQF's work in standardizing eMeasurement extends to measure 
development. NQF partnered with a software developer to develop the 
Measure Authoring Tool (MAT), which is a publicly available, free, web-
based tool designed to allow measure developers to create eMeasures 
using the aforementioned QDM, without needing to write programming 
code. At the end of 2012, NQF prepared to transition the day-to-day 
operation of the MAT to HHS, giving HHS the opportunity to better 
position the MAT and eMeasures in federal programs using EHR-based 
performance measurement, and to support the MAT's evolution.
    Also in 2012, NQF completed the Critical Paths for Creating Data 
Platforms project. This effort helped assess the readiness of 
electronic data to support innovative measurement concepts and 
recommended steps to address data and infrastructure gaps and barriers 
in two high-priority domains: care coordination and patient safety. The 
care coordination report focused on transitions of care and 
communication of the patient plan of care. The patient safety report 
focused on effective use of infusion devices (e.g., giving medication 
through an IV) in acute care settings. The ability to capture data 
across settings is fundamental to gauging, for example, the degree of 
care coordination in a healthcare system. The final reports from these 
projects delineated specific steps that the government and private 
sector can take to enable electronic measurement in these areas.

                                  Deliverables Associated With These Activities
----------------------------------------------------------------------------------------------------------------
                                                                  Status (as of 1/7/   Notes/Scheduled or actual
            Description                       Output                    2013)               completion date
----------------------------------------------------------------------------------------------------------------
Surgery measures and maintenance    Two-phase project to        Completed............  Phase 1: 18 measures
 review.                             endorse new surgery                                endorsed in December
                                     measures and conduct                               2011.
                                     maintenance on existing                           NQF Board endorsed 24
                                     NQF-endorsed measures.                             measures in Phase 2 in
                                                                                        January 2012.
                                                                                       Phase 2 addendum endorsed
                                                                                        9 measures in May 2012.
                                                                                       51 endorsed measures
                                                                                        total, 42 maintenance.
Efficiency and resource-use         Endorsed measures of        Completed............  Imaging Efficiency
 measures.                           imaging efficiency; white                          (Complete)
                                     paper drafted; endorsed                           --6 imaging efficiency
                                     measures of healthcare                             measures endorsed in
                                     efficiency.                                        February 2011.
                                                                                         --1 imaging efficiency
                                                                                       measure was recommended
                                                                                       to be combined with an
                                                                                       existing NQF measure and
                                                                                       was endorsed in April
                                                                                       2011.
                                                                                        Efficiency--Resource Use
                                                                                             (In Progress).
                                                                                         Cycle 1: 4 measures
                                                                                       endorsed in January 2012.
                                                                                         Cycle 2: 4 measures
                                                                                       endorsed in April 2012.
                                                                                         --8 total measures
                                                                                       endorsed, zero
                                                                                       maintenance.
Cancer measures and maintenance     Project to endorse new      Completed............  Phase 1: 22 measures
 review.                             cancer measures and                                endorsed October 2012,
                                     conduct maintenance on                             18 maintenance.
                                     existing NQF-endorsed                             Phase 2: 16 measures
                                     measures.                                          endorsed in October
                                                                                        2012, 10 maintenance

[[Page 46707]]

 
Perinatal measures and maintenance  Project to endorse new      Completed............  14 perinatal measures
 review.                             perinatal measures and                             endorsed April 2012, 12
                                     conduct maintenance on                             maintenance.
                                     existing NQF-endorsed
                                     measures.
Renal measures and maintenance      Project to endorse new      Completed............  12 renal measures
 review.                             renal measures and                                 endorsed April 2012,
                                     conduct maintenance on                             nine maintenance.
                                     existing NQF-endorsed
                                     measures.
Pulmonary/critical-care measures    Project to endorse new      In progress..........  19 pulmonary/critical-
 and maintenance review.             pulmonary/critical-care                            care measures endorsed
                                     measures, and conduct                              July 2012, 16
                                     maintenance on existing                            maintenance. One
                                     NQF-endorsed measures.                             additional measure
                                                                                        endorsed in January
                                                                                        2013, with two final
                                                                                        measures still under
                                                                                        review.
Palliative and end-of-life care...  Project to endorse new      Completed............  14 palliative and end-of-
                                     palliative and end-of-                             life care measures
                                     life care measures and                             endorsed February 2012,
                                     conduct maintenance on                             2 maintenance.
                                     existing NQF-endorsed
                                     measures.
Care-coordination measures and      Set of endorsed care-       Completed............  12 care coordination
 maintenance review.                 coordination measures.                             measures endorsed August
                                                                                        2012, 12 maintenance.
Population Health Phase 1:          Set of endorsed measures    Completed............  19 population health
 Prevention measures and             for preventative services.                         measures endorsed May
 maintenance measures review.                                                           2012, 17 maintenance.
Population health Phase 2:          Commissioned paper          Completed............  Five measures also
 Population health measures.         addressing population                              endorsed in October
                                     health measurement issues                          2012, 3 maintenance.
                                     and set of endorsed
                                     population health
                                     measures, plus set of
                                     endorsed measures.
Behavioral health measures and      Set of endorsed measures    Phase 1 completed,     Phase 1 endorsed 10
 maintenance review.                 for behavioral health.      phase 2 slated for     measures in October
                                                                 2013.                  2012, 4 maintenance.
All-cause readmissions (expedited   Set of endorsed all-cause   Completed............  2 all-cause readmissions
 Consensus Development Process       readmission measures.                              measures endorsed June
 [CDP] review).                                                                         2012, zero maintenance.
Multiple Chronic Conditions         Work plan completed;        Completed............  May 2012.
 Measurement Framework report        interim report available
 analyzing measures being used to    for public comment.
 gauge quality of care for people
 with multiple chronic conditions.
Patient-reported outcomes (PROs)    Two workshops discussing    Completed............  Final report completed
 workshops addressing                commissioned papers                                December 2012.
 prerequisites for endorsed PRO      addressing methodological
 measures.                           prerequisites for NQF
                                     consideration of PRO
                                     measures for endorsement.
Oral health.......................  Report that catalogs oral   Completed............  July 2012.
                                     health measures, measure
                                     concepts, priorities and
                                     gaps in measurement.
Rapid-cycle CDP improvement         Summary of process          Completed............  May 2012.
 (measure-endorsement process).      improvement approach,
                                     events, and metrics used
                                     to enhance the quality
                                     and efficiency of CDP
                                     process.
GI/GU Two-Stage CDP...............  Proposed two-stage pilot    Stage 1 completed....  12 measure concepts
                                     project designed to                                approved in December
                                     provide early guidance to                          2012.
                                     measure developers on
                                     whether a measure concept
                                     meets NQF's criterion for
                                     importance to measure and
                                     report before they invest
                                     time and resources in
                                     specifying and testing a
                                     measure.
Patient-safety-complications        Set of endorsed measures    Completed............  14 measures endorsed June
 measures and maintenance review     on complications-related                           2012, 14 maintenance.
 (Phase 1).                          areas.                                            2 additional measures
                                                                                        endorsed August 2012, 2
                                                                                        maintenance.
                                                                                       16 measures total, 16
                                                                                        maintenance.
Infectious disease measures and     Set of endorsed infectious  In progress..........  14 measures endorsed
 maintenance review.                 disease measures.                                  January 2013, 10
                                                                                        maintenance. Two
                                                                                        measures still under
                                                                                        review.

[[Page 46708]]

 
Regionalized Emergency Medical      Provide guidance for        Completed............  .........................
 Care Services measure topic         measure development to
 prioritization.                     ASPR's prioritized areas
                                     of (1) ED crowding,
                                     including a specific
                                     focus on boarding and
                                     diversion, (2) emergency
                                     preparedness, and (3)
                                     surge capacity.
Registry Needs Assessment.........  Hosted a public workshop    Completed............  .........................
                                     that discussed measure
                                     information needs,
                                     requirements, and
                                     potential approaches to
                                     measure information
                                     management, as well as 2
                                     webinars--focused on
                                     measure information
                                     management systems and a
                                     discussion on major
                                     findings of the workshop,
                                     respectively. Final
                                     report summarized major
                                     findings and included
                                     public feedback.
Common formats for patient safety   Responsible--on behalf of   Completed............  .........................
 data.                               AHRQ--for coordinating a
                                     process to obtain
                                     comments from
                                     stakeholders about the
                                     Common Formats authorized
                                     by the Patient Safety and
                                     Quality Improvement Act
                                     of 2005.
QDM maintenance...................  Updated the QDM to          Updates to QDM are     Each new version of the
                                     incorporate additional      ongoing with input     QDM will be published as
                                     types of measurement data   from NQF members,      needed. NQF will post a
                                     needed to support           the QDM User Group     draft of modifications
                                     emerging measures. The      and other interested   for each version.
                                     QDM June 2012 Update was    stakeholders..
                                     released in summer for
                                     public comment.
                                    The QDM December 2012 was
                                     released in December
                                     based on feedback from
                                     the 2014 Clinical Quality
                                     Measure (CQM) development
                                     cycle for Meaningful Use
                                     Stage 2.
MAT...............................  Non-proprietary, web-based  Completed............  CMS assumed day-to-day
                                     tool that allows                                   responsibilities of the
                                     performance-measure                                MAT as of January 2013.
                                     developers to specify,
                                     submit, and maintain
                                     electronic measures in a
                                     more streamlined,
                                     efficient, and highly
                                     structured way.
Refinement of the eMeasure Process  Provided education and      Ongoing..............  Launched and maintained
 and Technical Assistance.           outreach to both HHS and                           the Health IT Knowledge
                                     its contractors, and to                            Base which includes
                                     the users of QDM,                                  frequently asked
                                     eMeasures, and the                                 questions (FAQs) from
                                     Measure Authoring Tool:                            webinars, technical
                                     measure developers, EHR                            assistance log, user
                                     vendors, and providers                             feedback, etc., a
                                     implementing measures.                             glossary of terms and
                                     This education and                                 links to Health IT
                                     outreach included both                             reports.
                                     interactive teaching                              Updated and maintained
                                     through webinars and live                          the Measure Authoring
                                     presentations, as well as                          Tool (MAT) User Guide.
                                     development of technical                          Provided technical
                                     information posted on                              assistance to HHS/ONC/
                                     NQF's Web site. Technical                          CMS eMeasure contractors
                                     support was also provided                          focusing on topics such
                                     to HHS/CMS/ONC as needed.                          as QDM and eMeasure
                                                                                        logic in preparation for
                                                                                        the release of MU2.
                                                                                        Participated in eMeasure
                                                                                        support calls and
                                                                                        meeting as requested by
                                                                                        ONC and CMS.
                                                                                       Completed 6 public
                                                                                        webinars with over 1850
                                                                                        total attendees,
                                                                                        focusing on the Measure
                                                                                        Authoring Tool (MAT),
                                                                                        Quality Data Model (QDM)
                                                                                        and eMeasures.
Commissioned paper on data sources  Final report and            Completed............  April 2012.
 and readiness of HIT systems to     commissioned paper.
 support care coordination.
Critical Paths....................  Examine new measurement     Completed............  Patient Safety and Care
                                     areas (e.g. care plans)                            Coordination final
                                     to understand the                                  reports completed in
                                     feasibility of measuring                           October and November
                                     such areas in an                                   2012.
                                     electronic environment.
eMeasure Learning Collaborative...  Examining issues related    Completed............  Final report completed in
                                     to implementation of                               December 2012.
                                     eMeasures with a multi-
                                     stakeholder group in
                                     order to define best
                                     practices and
                                     recommendations to the
                                     Office of the National
                                     Coordinator's Federal
                                     Advisory Committees.
eMeasure feasibility testing......  Review the current state    In progress..........  Draft guidance report
                                     of feasibility assessment                          will be finalized and
                                     for eMeasures and                                  released for public
                                     identify a set of                                  comment. Slated for
                                     principles,                                        completed by 4/5/13.
                                     recommendations, and
                                     criteria for adequate
                                     feasibility assessment.

[[Page 46709]]

 
Composite evaluation guidance.....  Reassess NQF's existing     In progress..........  Final report slated for
                                     guidance for evaluating                            completed by 4/5/13.
                                     composites, with
                                     particular consideration
                                     of recent changes in
                                     composite measure
                                     development and related
                                     methodology.
----------------------------------------------------------------------------------------------------------------

4. Aligning Measure Use To Enhance Value

    Under section 1890(b)(5)(A)(i) of the Social Security Act, the 
entity is required to provide a description of its implementation of 
quality and efficiency measurement initiatives under the Social 
Security Act and the coordination of those initiatives with those 
implemented by other payers.
    Under section 1890A of the Social Security Act, HHS is required to 
establish a pre-rulemaking process under which a consensus-based entity 
(currently NQF) would convene multi-stakeholder groups to provide input 
to the Secretary on the selection of quality and efficiency measures 
for use in federal programs as specified under section 1890(b)(7)(B) of 
the Social Security Act. The list of quality and efficiency measures 
HHS is considering for selection will be publicly published no later 
than December 1 of each year. No later than February 1 of each year, 
NQF will report the input of the multi-stakeholder groups which will be 
considered by HHS in the selection of quality and efficiency measures 
for use in federal programs as specified under section 1890(b)(7)(B) of 
the Social Security Act.
    Alignment with respect to use of the same performance measures is a 
critical strategy for accelerating improvement, reducing wasteful 
reporting burden, and enhancing transparency in healthcare. The NQF-
convened Measure Applications Partnership (MAP), launched in the spring 
of 2011 as mandated by the Patient Protection and Affordable Care Act 
(Pub. L. 111-148, section 3014), is a key facilitator of measure 
alignment across federal programs and between the public and private 
sectors. The input that the MAP provides to HHS for purposes of the 
pre-rulemaking process and national priorities under the National 
Quality Strategy results from multiple stakeholders composed of 
representatives from more than 60 major private-sector stakeholder 
organizations, 10 federal agencies, and 40 individual technical experts 
MAP's input enhances HHS's ability to coordinate its quality and 
efficiency measurement initiatives with those initiatives implemented 
by other payers.
    More specifically, MAP provides a forum for annual multi-
stakeholder input into which performance measures are used in federal 
public reporting and pay-for-performance programs in advance of related 
regulations being issued. This approach augments traditional 
rulemaking, allowing the opportunity for substantive dialogue with HHS 
before rules are issued, a chance for alignment across programs with 
respect to use of measures, and consideration of longer term 
implications. MAP also provides a unique forum for public- and private-
sector leaders to develop and then broadly vet a future-focused 
performance measurement strategy (outlined in the MAP strategic plan 
below), as well as the shorter term recommendations for that strategy 
on an annual basis in pre-rulemaking reports. MAP strives to offer 
recommendations that are cross-cutting and coordinated across: settings 
of care; federal, state, and private programs; levels of measurement 
analysis; payer type; and points in time.
    Published on February 1, 2012, MAP's first pre-rulemaking report 
offered recommendations related to 17 federal programs.\25\ This 
report:
     Recommended that 40 percent of the measures that CMS 
proposed at the end of 2011 move into federal programs targeting 
clinicians, hospitals, and post-acute care/long-term care (PAC/LTC) 
settings via rules issued in 2012, with another 15 percent targeted for 
future consideration after further development, testing, and 
feasibility issues are worked out. MAP did not support inclusion of the 
remaining 45 percent primarily because many of the measures did not 
have enough information, specificity, testing, or proof of 
implementation feasibility to guide MAP measure evaluation and 
selection. See Appendix C for the criteria MAP used to guide measure 
selection.
     Expressed clear preference for both using NQF-endorsed 
measures and for developing more robust feedback loops. Over 90 percent 
of the measures that MAP supported for inclusion in the first round of 
pre-rulemaking input were currently NQF-endorsed, with the remainder 
likely eligible for expedited review. In addition to these criteria, 
NQF is establishing more robust feedback loops that can help HHS, MAP, 
and the broader field to discern which of the endorsed measures are 
best suited for inclusion in future reporting and value-based 
purchasing programs. More specifically, in 2012 MAP analyzed what 
internal and external sources exist to obtain feedback from end users 
and informally engaged MAP members to understand how they would 
prioritize varying types of feedback information.\26\
     Considered how to further align measures across public 
programs and with the private sector with the goal of more targeted, 
inter-related sets of measures that are reported by different kinds of 
providers, in different settings, and across time.
     Laid out guiding principles for a three- to five-year 
measurement strategy where priority is placed on: (1) Measures that 
drive the system toward meeting the NQS; (2) measures that are person- 
rather than clinician-focused; and (3) measures that span settings, 
time, and types of clinicians. Person-centered measurement provides 
information about what matters to patients (e.g., ``Will I be able to 
run after I recover from knee surgery?'') and that is specific to 
patient populations or care over time, (e.g., ``Did I get the care and 
support needed to manage my diabetes so that I did not lose my vision 
or my mobility?''). This kind of measurement is predicated on a 
redesigned delivery and payment system and an HIT-enabled environment 
that facilitates both coordination and integration of care for a range 
of patients across the continuum.
    Federal Medicare and Meaningful Use rules issued over the course of 
2012 largely followed the MAP pre-rulemaking recommendations for 
inclusion or exclusion of measures in over 20 different payment and 
reporting programs that MAP was asked to consider. However, concordance 
between the HHS final rules issued in 2012 with the MAP 2012 
recommendations varied depending on the program (see table below for 
key

[[Page 46710]]

programs). Over 70% concordance was observed for the majority of 
relevant programs. Of the two programs that had lower concordance with 
MAP Recommendations, there were only five measures in one program (ESRD 
QIP) relevant to the analysis, and there was a relatively short time 
period available for HHS to consider MAP's input for the other program 
(Meaningful Use). There were various reasons for the individual 
instances of discordance. Where CMS did not finalize measures that MAP 
supported, the most common issue was difficulty of data collection or 
other burden imposed by those measures. Excluded from the concordance 
analysis were many measures that had not yet been reviewed or endorsed 
by NQF at the time of MAP's evaluation, leaving MAP with insufficient 
information to provide a definitive ``Support'' or ``Do Not Support'' 
recommendation. For example, in the Medicare Physician Fee Schedule 
rule, CMS included a number of non-endorsed measures that address the 
broad array of medical specialties to engage more physicians in federal 
physician-level programs. Going forward NQF is poised to quickly move 
these measures through review for potential endorsement.

  Concordance of MAP ``Support'' and ``Do Not Support'' Recommendations
  With Measures Included in Selected HHS Programs From HHS Final Rules
                             Issued in 2012
------------------------------------------------------------------------
                                                         Concordance of
                                                              MAP
                                                        Recommendations
                   HHS Final Rules                       With HHS Rules
                                                         Issued in 2012
                                                           (percent)
------------------------------------------------------------------------
Hospital IQR.........................................                 73
Hospital VBP.........................................                 71
Inpatient Psych Facility.............................                100
Meaningful Use.......................................                 50
Physician Quality Reporting System (PQRS)............                 79
End-Stage Renal Disease Quality Improvement Program                   40
 (ESRD QIP)..........................................
------------------------------------------------------------------------

    MAP Strategic Plan for Measurement. To spur progress toward a 
defined set of goals and priorities related to the NQS--which include 
improved quality and safety, more transparency, and enhanced value--MAP 
developed a three-year strategic plan for measurement (2012-2015). This 
plan was released on October 1, 2012, and is intended to inform HHS's 
future measure development planning, as well as shape annual rulemaking 
advice in the years ahead. The plan has the following three major 
components:
     Define sets of measures as families of measures with the 
objective of knitting together related measures currently found in 
different programs, care settings, levels of analysis, and populations. 
This approach complements the program-specific recommendations that MAP 
made in its pre-rulemaking report. Individual measures are carefully 
selected to work together as a ``family'' to drive the overall system 
toward better performance in a given area, promote more patient-
centeredness, and decrease reporting burden for providers. Families of 
measures are linked to a high-impact condition (e.g., diabetes) or an 
NQS priority (e.g., safety) and are intended to promote further measure 
alignment by specifying within the families more discrete core measure 
sets focused on hospitals, clinicians, or post-acute/long-term care. 
See MAP's Families of Measures report or for a summary of the report, 
see page 28.
     Engage stakeholders that develop, report, and use measures 
to glean feedback about the use and usefulness of measures. The idea is 
to create more effective two-way communication so that the experiences 
of end users directly inform MAP's recommendations to HHS, contribute 
to the thinking of the diverse stakeholders that participate directly 
and indirectly in MAP's activities, as well as inform the work of 
measure developers as they address identified measurement gaps in a 
more coordinated fashion.
     Develop analytic support for MAP decision making. The goal 
is to further enrich MAP's thinking and decision-making by integrating 
important data and information that are developed across NQF as a 
strategic byproduct of its different activities. These include input to 
priority setting and strategies, measurement review and endorsement, 
and advice on measure selection. This function would also draw upon the 
various outside efforts under way to glean information about measure 
use and impact. The analysis and integration of internal and external 
data will inform and likely refine MAP's overall selection criteria, as 
well as its recommendations to HHS in future pre-rulemaking reports. In 
addition, an independent third-party evaluation is planned to determine 
whether MAP is meeting its overall objectives.
    The MAP pre-rulemaking recommendations and strategic plan largely 
reflect the current reality of our siloed healthcare payment and 
delivery systems, but anticipate a future system with shared 
accountability for patient welfare, community health, and stewardship 
of scarce resources.
Families of Measures
    MAP selected safety, care coordination, cardiovascular conditions, 
and diabetes as its first focus areas for identification of families of 
measures-- all areas called out in the NQS and/or leading causes of 
mortality. MAP's first families of measures report was published on 
October 1, 2012.
    MAP reviewed 676 measures across these 4 topics, using criteria 
laid out in the report as a guide to inform selection. Of these 
measures, MAP recommended 55 safety, 60 care coordination, 37 
cardiovascular, and 13 diabetes measures for inclusion in 4 distinct 
families of measures. MAP further defined more discrete core measures, 
which include available measures, and gaps specific to a care setting 
(e.g. hospitals, post-acute care/long-term care), level of analysis 
(e.g. individual clinicians), or population drawn from each family of 
measures and made program-specific recommendations in its 2013 pre-
rulemaking report. MAP anticipates identifying families of measures for 
patient and family engagement, population health, affordability/cost, 
and mental health in 2013, pending funding decisions.
    MAP defined families of measures with the intent that their 
implementation would lead to performance improvement and further 
cohesion and synergy of care in a targeted area. Measures in a given 
family bridge healthcare settings, types of providers, and time and are 
interconnected in the way patients would ideally like to experience 
care. Families of measures also include identifying measure gaps, which 
strongly signal to developers where new measures are needed, and can 
help facilitate prioritization of funding for measure development.
    For example, the safety family of measures contains 9 topic areas 
and 22 subtopic areas. The topic areas include but are not limited to 
reducing healthcare-acquired infections and obstetrical adverse events 
and increasing procedural safety. Examples of specific gaps in the 
safety family of measures include post-discharge follow-up of 
infections in ambulatory settings, ventilator-associated events with 
special considerations for the pediatric population, and infection 
measures reported as rates rather than ratios, which would be more 
meaningful to consumers. The 55 measures selected for the safety family 
of measures follow themes such as creating a culture of safety, patient 
and caregiver

[[Page 46711]]

engagement, reporting meaningful safety information, and cost of care 
implications. These measures were selected for their ability to cross 
settings to simultaneously affect patients, caregivers, and purchasers 
and to ultimately increase safety for all patients.
Measure Use and Alignment
    Although the advantages of measure alignment are many, few studies 
have systematically examined this phenomenon. A 2011 RAND study of 75 
diverse organizations found that nearly all used NQF-endorsed measures, 
although there was considerable variability in which measures were used 
and for what purposes. Most used NQF-endorsed measures in quality 
improvement programs, followed closely by use in public reporting and 
then payment programs. The 2011 study also found that the organizations 
surveyed indicated a strong preference for NQF-endorsed measures where 
they exist because they are vetted, evidence-based, and known to be 
more credible with providers.\27\
    In 2011 and 2012, NQF conducted initial research outside of the HHS 
contract to better understand which organizations are using NQF-
endorsed measures and where there is alignment across sectors with 
respect to that use.28 29 In addition, NQF is developing 
more systematic approaches to capturing detailed feedback from end 
users about the usefulness of NQF measures in driving improvements in 
health and healthcare.
    The 2012 analysis showed that 86 percent of the 706 NQF-endorsed 
measures were in use, with the balance of the portfolio not in use 
largely consisting of measures recently endorsed (last 1-3 years) and 
expected to be used in the near future. Federal use of the NQF 
portfolio was stable at about 50 percent. Private payer use of the NQF 
portfolio grew from 21 percent to 35 percent during this period; state 
use grew from 21 percent to 23 percent. Much of the increase in private 
payer use is likely attributable to better data collection by NQF, 
rather than increased use of NQF-endorsed measures by private payers.
    The federal government, private plans, and states appear to be 
increasingly using the same NQF-endorsed measures. In 2012, the federal 
government and private payers used the same 76 measures in 
accountability programs, or 13 percent of the 606 NQF-endorsed measures 
in use. During the same period, federal and state alignment was 48 
measures, or 8 percent, and private payer and state alignment was 51 
measures, or 8 percent. In 2012, 25 measures were simultaneously used 
by the federal government, private payers, and states. When all users 
are taken into account (including local communities, registries and 
others users), about 29 percent of the NQF-endorsed portfolio was used 
by two or more stakeholders in 2012.

NQF Facilitates National, State, and Local Measure Alignment

 Improvement Targets: Inform the National Quality Strategy 
(National Priorities Partnership)
 Measures: Endorse and harmonize measures
 Incentives: Advise HHS on reporting/payment programs (Measure 
Applications Partnership)
 National-Local Actions: Develop tools to align use of measures 
(Quality Positioning System or QPS) and efforts of national/local 
organizations implementing strategies at the delivery system level 
(National Priorities Partnership)
Alignment at the Community Level
    Given the number and diversity of community-based efforts, it is 
challenging to get a comprehensive sense of how standardized measures 
are being used at the local, state, or regional levels. That said, the 
number of regional multi-stakeholder collaboratives or alliances that 
are collecting, reporting, and in some cases paying on the basis of 
performance measures appears to have grown over the past number of 
years. As of October 2012, the Robert Wood Johnson Foundation has 
cataloged on its Web site a compendium of nearly 260 state, local, or 
regional efforts to publicly report on healthcare performance across 
the United States.\30\
    To better understand the public-reporting activities in a subset of 
these community-based groups, NQF analyzed the measure use of 16 
alliances that receive funding from the Robert Wood Johnson Foundation 
through the Aligning Forces for Quality (AF4Q) program. This analysis 
showed that these alliances are using 171 NQF-endorsed measures in 
their reports to the public, and it provided insight to NQF as to the 
kinds of tools and capabilities communities are seeking as they evolve 
measurement efforts on the local level.
    Supported by the Robert Wood Johnson Foundation, NQF has developed 
tools outside of the HHS contracts to support local, state, and 
regional leaders interested in using NQF-endorsed measures, 
particularly those measures also used in federal programs. For example, 
NQF's publicly available Quality Positioning System (QPS) enables users 
to search a database of NQF-endorsed measures and to build a portfolio 
or custom list of NQF-endorsed measures that they use or in which they 
are interested. A QPS user can then compare that portfolio against 
measures used in federal and other national programs, aligning 
measurement efforts where it makes sense to do so. A QPS user also can 
share its portfolio with others by self-publishing it within QPS on the 
NQF Web site. This feature and the ability to discern which NQF-
endorsed measures are being used in federal programs can provide a rich 
information base to help communities, states, and the federal 
government synchronize their approaches to measuring and improving 
quality.

                                  Deliverables Associated With These Activities
----------------------------------------------------------------------------------------------------------------
                                                                                            Notes/scheduled or
             Description                        Output          Status (as of 1/7/2013)  actual  completion date
----------------------------------------------------------------------------------------------------------------
Measures for use in quality reporting  Measure Applications     Completed..............  February 2012.
 programs under Medicare.               Partnership Pre-
                                        Rulemaking Report:
                                        Input on Measures
                                        Under Consideration by
                                        HHS for 2012
                                        Rulemaking.
MAP report recommending measures that  Final report including   Completed..............  June 1, 2012.
 address the quality issues             potential new
 identified for dual-eligible           performance measures
 beneficiaries.                         to fill gaps in
                                        measurement for dual-
                                        eligible beneficiaries.

[[Page 46712]]

 
MAP report recommending measures for   Final report including   Completed..............  June 1, 2012.
 use in quality reporting for           MAP Coordinating
 Prospective Payment System-exempt      Committee
 cancer hospitals.                      recommendations.
MAP report recommending measures for   Final report including   Completed..............  June 1, 2012.
 use in quality reporting for hospice   MAP Coordinating
 care.                                  Committee
                                        recommendations.
MAP Strategic Plan 2012-2015.........  Final report...........  Completed..............  October 2012.
MAP report detailing families of       Final report...........  Completed..............  October 2012.
 measures for safety, care
 coordination, cardiovascular
 conditions, and diabetes.
----------------------------------------------------------------------------------------------------------------

5. Identifying Measure Gaps and Developing Strategies for Filling Them

    Under section 1890(b)(5)(iv) of the Social Security Act, the entity 
is required to describe gaps in endorsed quality and efficiency 
measures, including measures within priority areas identified by HHS 
under the agency's National Quality Strategy, and where quality and 
efficiency measures are unavailable or inadequate to identify or 
address such gaps. Under section 1890(b)(5)(v) of the Social Security 
Act, NQF is also required to describe areas in which evidence is 
insufficient to support endorsement of quality and efficiency measures 
in priority areas identified by the National Quality Strategy and where 
targeted research may address such gaps.
    Performance measurement science has made important strides in the 
last decade, including addressing new settings and types of providers, 
becoming more responsive to the needs and preferences of varied 
stakeholders, evolving with new technology, and increasingly addressing 
hard-to-measure concepts such as care coordination and appropriateness. 
Despite these gains, measurement gaps persist, either because the 
measures have not yet been developed, or the measures exist but are not 
being used.
    To identify measurement gaps, NQF conducted an extensive analysis 
in 2012 of its current measures portfolio against both the National 
Quality Strategy priority areas and high-impact conditions (both 
Medicare and child health) as required by statute (Social Security Act, 
section 1890(b)(5)(iv)), analyzed stakeholder feedback, and considered 
which NQF-endorsed measures were being used and by which sector. The 
gaps identified below, however, do need to be viewed in the context of 
rising concern about measurement overload and administrative burden. 
While more measures are needed to address high-priority issues, NQF 
continues to remove measures that no longer meet its criteria or where 
performance ``tops out'' to ensure measurement parsimony.
Synthesis of Measure Gaps
    Captured in the 2012 NQF Measure Gap Analysis, this report revealed 
that discussions of measure gaps too often remain at a high conceptual 
level, and that more detailed information is needed to inform next 
steps, whether those steps entail measure development or addressing 
barriers to implementation of existing measures. In addition, while 
there may be non-NQF endorsed measures currently in use that address 
high-priority gap areas, a full assessment of their applicability and 
appropriateness was beyond the scope of this project. Such measures 
should be brought forth for NQF endorsement to assess their importance, 
scientific reliability and validity, usability, and feasibility before 
an assessment of value or recommendations for use can be made. The 
following are high-level syntheses of the measure gaps identified 
through the NQF analysis, presented through the lens of the three aims 
of the NQS.
Better Care
    The lion's share of current NQF-endorsed measures related to better 
care focused on specific conditions. Addressing the gaps identified 
below would provide added input directly from patients about their care 
and could further focus the healthcare system on the needs and 
preferences of patients and families, including the most vulnerable 
patients.
    Patient-reported outcomes (PROs)--To fully assess the quality and 
safety of healthcare, the gap analysis emphasized the importance of 
patient-reported outcomes--any report of the patient's health status 
that comes directly from the patient, without interpretation by a 
clinician or anyone else. Domains for measurement include symptoms and 
symptom burden, health-related quality of life including functional 
status, experience with care, and health-related behaviors. Especially 
important are PRO-based performance measures that can be aggregated 
accurately and reliably to the level of an accountable healthcare 
entity, and that span the full continuum of care.
    Patient-centered care and shared decision-making--To spur the 
healthcare system to be more responsive to patients and families, 
measures are needed that assess whether patient and family treatment 
preferences are identified; whether their psychosocial, cultural, 
spiritual, or healthcare literacy needs are addressed; whether they are 
actively engaged in developing a care plan; and whether their expressed 
preferences and goals for care are met. Measures of decision quality 
are critical for assessing whether patients understand evidence-based 
treatment options and whether they are able to make decisions based on 
information provided by their healthcare practitioner.
    Care coordination and care transitions--Important outcome measures 
are needed to assess whether patients, families, and caregivers believe 
that the overall care coordination process--including the quality of 
communication, care planning, care transitions, and team-based care--
satisfactorily prepared them to manage their care and return to the 
best possible quality of life. The timeliness of access to high-quality 
palliative care or hospice services, including pain and symptom 
management, psychosocial support, and advance care planning also is 
identified as a gap area in need of further attention. Measure gaps 
related to effective medication management and patient adherence, and 
adverse drug events remain.

[[Page 46713]]

    Care for vulnerable populations--A critical gap area to be filled 
includes the ability to measure whether high-quality care is available 
to patients most in need, particularly the vulnerable elderly, 
individuals with multiple chronic conditions and complex care needs, 
critically ill patients, patients receiving end-of-life care, children 
with special needs, residents in long-term care settings, the homeless, 
and people who are dually eligible for Medicare and Medicaid.
Healthy People/Healthy Communities
    Recognizing that the health of the American public is mostly 
attributable to healthy life style behaviors, environment, or social 
status, the following gap areas push the field beyond the traditional 
boundaries of the healthcare delivery system and offer the potential 
for dramatic gains in health for the nation.
    Health and well-being--Measures within and outside of the 
healthcare system are needed to assess health-related quality of life 
and to optimize the population's well-being. Measures that assess the 
burden of illness experienced by patients, families, and caregivers, as 
well as measures of productivity also are important. Community indices 
that measure key factors or social determinants known to significantly 
influence health or drive unnecessary utilization of healthcare 
services are needed to develop community programs that effectively and 
appropriately target resources and interventions to improve population 
health and reduce disparities.
    Preventive care--Composite measures of the highest impact age- and 
sex-appropriate clinical preventive services, particularly for the 
cardiovascular disease priority area, continue to be important measure 
gaps to fill. Oral health was highlighted as an important area in need 
of measures, specifically for the prevention of dental caries, as were 
coordination of long-term support services and psychosocial, behavioral 
health, spiritual, and cultural services. An emerging area of focus for 
measurement is on the extent to which care is coordinated beyond the 
healthcare delivery system--particularly between healthcare, public 
health, and community support services--and how individual 
organizations are held collectively accountable.
    Childhood measures--Measure gaps for child and adolescent health 
emphasized the attainment of developmental milestones, the quality of 
adolescent well-care visits, prevention of accidents and injuries, and 
prevention of risky behaviors. There also is a heightened need for 
measures of childhood obesity in addition to body mass index for more 
effective upstream management, given the risk for development of 
diabetes, cardiovascular disease, and other chronic conditions.
Accessible and Affordable Care
    Affordability is often narrowly construed. The following 
identification of gaps broadens its definition so that affordability is 
viewed through a variety of lenses including the individual and 
society, for example, out-of-pocket costs to patients and families and 
costs to the healthcare system. Further, a commitment to ensuring 
access to affordable, high quality care for all necessitates judicious 
use of resources at the individual level.
    Access to care--In addition to measures that assess insurance 
coverage, the analysis revealed that measure gaps indicative of access 
to needed care are important to address. Important considerations 
include the ability to obtain medications, mental health, oral health, 
and specialty services in a timely fashion. Measures also are needed to 
assess disparities in access and affordability, particularly with 
regard to socioeconomic status, race, and ethnicity, and for vulnerable 
populations.
    Healthcare affordability--Many stakeholders emphasize the need for 
affordability indices that reflect the burden of healthcare costs on 
consumers and that include direct costs (e.g., out-of-pocket expenses, 
personal healthcare expenditures per capita) as well as indirect 
opportunity costs (e.g., productivity, work and school absenteeism, and 
the ``cost of neglect'' of medical and dental care). Efficiency 
measures are needed to benchmark providers on cost and quality as well 
as to quantify the impact of inefficiencies across care settings to 
further target quality improvement efforts. Purchasers and consumers 
continue to emphasize the importance of understanding pricing and 
improved transparency of data through standardized measurement and 
reporting.
    Waste and overuse--Measures that assess the extent to which the 
healthcare system promotes the provision of medical, surgical, and 
diagnostic services that offer little if any value--and that may be 
harmful to patients--are critical to closing gaps in variation. 
Specific areas frequently cited as important for measurement include 
appropriate, patient-centered and patient-directed end-of-life care; 
unnecessary emergency department visits and hospital admissions and 
readmissions (particularly for ambulatory-sensitive conditions); 
inappropriate medication use and polypharmacy; and duplication of or 
inappropriate services and testing, particularly imaging.
Availability of NQF-endorsed Measures
    Although the NQF portfolio increasingly maps to the NQS, its extent 
varies across each of the six NQS priorities. For example, 40 percent 
of NQF measures that map to the NQS at the goal level address patient 
safety, including a wide range of measures related to healthcare-
acquired conditions and hospital readmissions. Yet only 7 percent of 
measures that map at the goal level address patient and family 
engagement, with very few measures to address important areas of shared 
decision making, patient navigation, and patient self-management. 
Likewise, measures to address healthy lifestyle behaviors and community 
interventions to prevent cardiovascular disease upstream also warrant 
increased attention. Specific measures of cost remain a high-priority 
gap area, particularly for purchasers of healthcare.
    NQF's portfolio includes more than 400 condition-specific measures, 
more than 250 of which address the high-impact Medicare conditions. Yet 
only 53 of the measures address the specific high-impact child health 
conditions, and 12 of the high-impact child health conditions do not 
have any specific endorsed measures. While the lack of measures for 
certain conditions may be of interest or concern, future measure 
development should be prioritized to focus on cross-cutting measures 
that apply to patients regardless of their disease process.
NQF Measure Portfolio in Use
    The federal government remains the predominant user of NQF-endorsed 
measures, but a growing number of measures are in use across other 
public-sector programs--including state and local programs--as well as 
in the private sector. More promising is the emerging overlap in 
measure use across these sectors. Further alignment--or use of the same 
measures--offers the potential to significantly reduce measurement 
burden and to simultaneously accelerate improvement by sending 
consistent signals about what is important for providers to focus care 
improvement resources against.
    Overall, 64 measures in the NQF portfolio that address specific NQS 
goals are in concurrent use in federal programs and two or more private 
programs. While the majority of these are safety-related measures, a 
small

[[Page 46714]]

number address aspects of overuse, patient experience, and preventive 
screenings. A nearly equal number of measures that address specific NQS 
goals are not in use in any of the programs analyzed--a missed 
opportunity, particularly for goals related to function and quality of 
life, hospice and palliative care, mental health, and preventive 
services for children. Similarly, the analysis revealed that 57 
measures in the NQF portfolio that address high-impact conditions are 
in concurrent use in federal programs and two or more private programs, 
the majority of which reflect the high-impact Medicare conditions. 
However, 47 measures that address high-impact Medicare or child health 
conditions had no identified use in any of the sectors analyzed. 
Consideration should be given to the potential barriers that prevent 
these measures from being implemented in the field.
The Path Forward
    As the field--the public and private stakeholders committed to 
building a solid foundation for quality improvement--strives to 
continually advance the use of standardized performance measurement, 
there is a strong desire to accelerate efforts to fill, rather than 
just identify, key measurement gaps. This will require making better 
use of the measures already available for key priority areas and 
investing wisely in measure development and endorsement activities to 
fill the most critical gap areas.

6. Looking Forward

    NQF has evolved in the dozen years it has been in existence and 
since it endorsed its first performance measures a decade ago. While 
its focus on improving quality, enhancing safety, and reducing costs by 
endorsing performance measures has remained a constant, its role has 
expanded to include a significant emphasis on getting the various 
stakeholder groups to align with respect to their use of performance 
measures and related improvement efforts. Experience has made it clear 
that sector-by-sector approaches to enhancing healthcare performance 
are ineffective in our decentralized and complex healthcare system, and 
they waste precious healthcare resources and may even do harm.
    Looking ahead, NQF will work together with HHS and the broader 
quality movement to:
     Deepen the alignment between the public and private 
sectors and across stakeholder groups to accelerate progress and reduce 
burden: This relates to measure endorsement and the work of NQF-
convened partnerships and is a core, enduring value of the 
organization;
     Focus more on ``end user'' needs and engagement: NQF will 
enlarge its current collaborative efforts to better incorporate the 
perspectives and values of those at the local level and those on the 
sharp end of healthcare--who ultimately are integrating the needs of 
the delivery system with those who receive and pay for care. Starting 
with the preferences of the end user in mind and systematically 
collecting user feedback about the efficacy of measures are ways to 
engage communities, providers, and other users in the collective goal 
of improving healthcare value.
     Take a more proactive approach to coordinate the measures 
pipeline and remake measure review and endorsement so it is more 
nimble: NQF will not only identify measure gaps but engage developers 
in filling them so that their efforts are streamlined and avoid 
duplication. Simultaneously, NQF plans to set up standing committees so 
that measures can more readily be reviewed.
     Review and endorse ``next generation'' quality measures 
that put the patient first: A key priority is endorsing next-generation 
measures that are more meaningful to patients and families and that 
help track patient outcomes across healthcare settings. NQF is 
committed to moving our nation's healthcare system to be ever more 
responsive to patient preferences and values and believes that richer 
information can play a crucial role;
     Increase the focus on measures that can enhance value: 
Affordability and its relationship to quality will become a focal point 
and better integrated into NQF's future work, starting with defining 
the many aspects of affordability and prioritizing near and longer term 
areas of focus going forward. Given the embryonic stage of 
affordability measures overall, there is much upfront conceptual work 
to be done that will rely on getting broad-based and varied input in 
order to gain a deeper appreciation for how to further measurement in 
the areas of costs, appropriateness, and resource use and how to pair 
such measures with quality metrics in order to assess value.
    NQF is embarking on an exciting agenda that emphasizes enhanced 
alignment and collaboration so as to better integrate end user needs--
all with an eye on evolving our measure portfolio so that it drives the 
healthcare system toward both delivering higher value healthcare and 
incorporating the needs and preferences of patients, payers, and 
purchasers. The goals are clear, and the collective work of the 800 
plus individuals who collaborate with NQF are focused on efforts to 
benefit the U.S. healthcare system and the patients it serves.

Appendix A: 2012 Accomplishments

                                       January 14, 2012 to January 7, 2013
----------------------------------------------------------------------------------------------------------------
                                                                                           Notes/scheduled or
           Description                     Output            Status  (as of 1/7/2013)    actual completion date
----------------------------------------------------------------------------------------------------------------
                I. Facilitating Coordinated Action to Achieve the National Quality Strategy Goals
----------------------------------------------------------------------------------------------------------------
NPP support for Partnership for   4 quarterly convenings    Completed.................  Content of meetings and
 Patients' HHS initiative          for 100+ people each,                                 webinars were captured
 focused on patient safety.        and 3 webinars reaching                               in individual
                                   550+.                                                 summaries.
NPP support for Partnership for   2 public web meetings     Completed.................  Content of meetings and
 Patients' HHS initiative          reaching 500+ and 2                                   calls were captured in
 focused on patient safety.        public conference                                     individual summaries.
                                   calls, reaching 100+.
NPP support for Partnership for   Formed two action teams   Completed.................
 Patients' HHS initiative          around Readmissions and
 focused on patient safety.        Maternal Health. Early
                                   development of
                                   additional action teams
                                   around Million Hearts/
                                   Cardiovascular Health
                                   and Patient & Family
                                   Engagement.

[[Page 46715]]

 
NPP support for Partnership for   Created the Action        Completed.................
 Patients' HHS initiative          Registry, a virtual
 focused on patient safety.        space for organizations
                                   to share their quality
                                   improvement activities--
                                   or ``actions''--around
                                   the six priority areas
                                   of the National Quality
                                   Strategy and make
                                   connections with each
                                   other.
NPP support for Partnership for   Quarterly reports for     Completed.................
 Patients' HHS initiative          HHS.
 focused on patient safety.
----------------------------------------------------------------------------------------------------------------
                              II. Supporting National Healthcare Measurement Needs
----------------------------------------------------------------------------------------------------------------
Surgery measures and maintenance  Two-phase project to      Completed.................  Phase 1: 18 measures
 review.                           endorse new surgery                                   endorsed in December
                                   measures and conduct                                  2011.
                                   maintenance on existing                              NQF Board endorsed 24
                                   NQF-endorsed measures.                                measures in Phase 2 in
                                                                                         January 2012.
                                                                                        Phase 2 addendum
                                                                                         endorsed 9 measures in
                                                                                         May 2012.
                                                                                        51 endorsed measures
                                                                                         total, 42 maintenance.
Efficiency and resource-use       Endorsed measures of      Completed.................  Imaging Efficiency
 measures.                         imaging efficiency;                                   (Complete)
                                   white paper drafted;                                 --6 imaging efficiency
                                   endorsed measures of                                  measures endorsed in
                                   healthcare efficiency.                                February 2011.
                                                                                        --1 imaging efficiency
                                                                                         measure was recommended
                                                                                         to be combined with an
                                                                                         existing NQF measure
                                                                                         and was endorsed in
                                                                                         April 2011.
                                                                                        Efficiency--Resource Use
                                                                                         (Complete).
                                                                                        Cycle 1: 4 measures
                                                                                         endorsed in January
                                                                                         2012.
                                                                                        Cycle 2: 4 measures
                                                                                         endorsed in April 2012.
                                                                                        --8 total measures
                                                                                         endorsed, zero
                                                                                         maintenance.
Cancer measures and maintenance   Project to endorse new    Completed.................  Phase 1: 22 measures
 review.                           cancer measures and                                   endorsed October 2012,
                                   conduct maintenance on                                18 maintenance.
                                   existing NQF-endorsed                                Phase 2: 16 measures
                                   measures.                                             endorsed in October
                                                                                         2012, 10 maintenance.
Perinatal measures and            Project to endorse new    Completed.................  14 perinatal measures
 maintenance review.               perinatal measures and                                endorsed April 2012, 12
                                   conduct maintenance on                                maintenance.
                                   existing NQF-endorsed
                                   measures.
Renal measures and maintenance    Project to endorse new    Completed.................  12 renal measures
 review.                           renal measures and                                    endorsed April 2012,
                                   conduct maintenance on                                nine maintenance.
                                   existing NQF-endorsed
                                   measures.
Pulmonary/critical-care measures  Project to endorse new    In progress...............  19 pulmonary/critical-
 and maintenance review.           pulmonary/critical-care                               care measures endorsed
                                   measures, and conduct                                 July 2012, 16
                                   maintenance on existing                               maintenance. One
                                   NQF-endorsed measures.                                additional measure
                                                                                         endorsed in January
                                                                                         2013, with two final
                                                                                         measures still under
                                                                                         review.
Palliative and end-of-life care.  Project to endorse new    Completed.................  14 palliative and end-of-
                                   palliative and end-of-                                life care measures
                                   life care measures and                                endorsed February 2012,
                                   conduct maintenance on                                2 maintenance.
                                   existing NQF-endorsed
                                   measures.
Care coordination measures and    Set of endorsed care      Completed.................  12 care coordination
 maintenance review.               coordination measures.                                measures endorsed
                                                                                         August 2012, 12
                                                                                         maintenance.
Population Health Phase 1:        Set of endorsed measures  Completed.................  19 population health
 Prevention measures and           for preventative                                      measures endorsed May
 maintenance measures review.      services.                                             2012, 17 maintenance.
Population health Phase 2:        Commissioned paper        Completed.................  Five measures also
 Population health measures.       addressing population                                 endorsed in October
                                   health measurement                                    2012, 3 maintenance.
                                   issues and set of
                                   endorsed population
                                   health measures, plus
                                   set of endorsed
                                   measures.
Behavioral health measures and    Set of endorsed measures  Phase I completed, phase 2  Phase 1 endorsed 10
 maintenance review.               for behavioral health.    slated for 2013.            measures in October
                                                                                         2012, 4 maintenance.
All-cause readmissions            Set of endorsed all-      Completed.................  Two all-cause
 (expedited Consensus              cause readmission                                     readmissions measures
 Development Process [CDP]         measures.                                             endorsed June 2012,
 review).                                                                                zero maintenance.

[[Page 46716]]

 
Multiple Chronic Conditions       Work plan completed;      Completed.................  May 2012.
 Measurement Framework report      interim report
 analyzing measures being used     available for public
 to gauge quality of care for      comment.
 people with multiple chronic
 conditions.
Patient-reported outcomes (PROs)  Two workshops discussing  Completed.................  Final report completed
 workshops addressing              commissioned papers                                   December 2012.
 prerequisites for endorsed PRO    addressing
 measures.                         methodological
                                   prerequisites for NQF
                                   consideration of PRO
                                   measures for
                                   endorsement.
Oral health.....................  Report that catalogs      Completed.................  July 2012.
                                   oral health measures,
                                   measure concepts,
                                   priorities and gaps in
                                   measurement.
Rapid-cycle CDP improvement       Summary of process        Completed.................  May 2012.
 (measure-endorsement process).    improvement approach,
                                   events, and metrics
                                   used to enhance the
                                   quality and efficiency
                                   of CDP process.
GI/GU Two-Stage CDP.............  Proposed two-stage pilot  Stage 1 completed.........  12 measure concepts
                                   project designed to                                   approved in December
                                   provide early guidance                                2012.
                                   to measure developers
                                   on whether a measure
                                   concept meets NQF's
                                   criterion for
                                   importance to measure
                                   and report before they
                                   invest time and
                                   resources in specifying
                                   and testing a measure.
Patient-safety-complications      Set of endorsed measures  Completed.................  14 measures endorsed
 measures and maintenance review   on complications-                                     June 2012, 14
 (Phase 1).                        related areas.                                        maintenance.
                                                                                        2 additional measures
                                                                                         endorsed August 2012. 2
                                                                                         maintenance.
                                                                                        16 measures total, 16
                                                                                         maintenance.
Infectious disease measures and   Set of endorsed           In progress...............  14 measures endorsed
 maintenance review.               infectious disease                                    January 2013, 10
                                   measures.                                             maintenance. Two
                                                                                         measures still under
                                                                                         review.
Regionalized Emergency Medical    Provide guidance for      Completed.................
 Care Services measure topic       measure development to
 prioritization.                   ASPR's prioritized
                                   areas of (1) ED
                                   crowding, including a
                                   specific focus on
                                   boarding and diversion,
                                   (2) emergency
                                   preparedness, and (3)
                                   surge capacity.
Registry Needs Assessment.......  Hosted a public workshop  Completed.................
                                   that discussed measure
                                   information needs,
                                   requirements, and
                                   potential approaches to
                                   measure information
                                   management, as well as
                                   2 webinars--focused on
                                   measure information
                                   management systems and
                                   a discussion on major
                                   findings of the
                                   workshop, respectively.
                                   Final report summarized
                                   major findings and
                                   included public
                                   feedback.
Common formats for patient        Responsible--on behalf    Completed.................
 safety data.                      of AHRQ--for
                                   coordinating a process
                                   to obtain comments from
                                   stakeholders about the
                                   Common Formats
                                   authorized by the
                                   Patient Safety and
                                   Quality Improvement Act
                                   of 2005.
QDM maintenance.................  Updated the QDM to        Completed.................  Work stopped effective 1/
                                   incorporate additional                                10/13 as a result of
                                   types of measurement                                  amendments made by the
                                   data needed to support                                American Taxpayer
                                   emerging measures. The                                Relief Act.
                                   QDM June 2012 Update
                                   was released in summer
                                   for public comment.
                                  The QDM December 2012
                                   was released in
                                   December based on
                                   feedback from the 2014
                                   Clinical Quality
                                   Measure (CQM)
                                   development cycle for
                                   Meaningful Use Stage 2.
MAT.............................  Non-proprietary, web-     Completed.................  CMS assumed day-to-day
                                   based tool that allows                                responsibilities of the
                                   performance-measure                                   MAT as of January 2013.
                                   developers to specify,
                                   submit, and maintain
                                   electronic measures in
                                   a more streamlined,
                                   efficient, and highly
                                   structured way.

[[Page 46717]]

 
Refinement of the eMeasure        Provided education and    Ongoing...................  Launched and maintained
 Process and Technical             outreach to both HHS                                  the Health IT Knowledge
 Assistance.                       and its contractors,                                  Base which includes
                                   and to the users of                                   frequently asked
                                   QDM, eMeasures, and the                               questions (FAQs) from
                                   Measure Authoring Tool:                               webinars, technical
                                   Measure developers, EHR                               assistance log, user
                                   vendors, and providers                                feedback, etc., a
                                   implementing measures.                                glossary of terms and
                                   This education and                                    links to Health IT
                                   outreach included both                                reports. Updated and
                                   interactive teaching                                  maintained the Measure
                                   through webinars and                                  Authoring Tool (MAT)
                                   live presentations, as                                User Guide. Provided
                                   well as development of                                technical assistance to
                                   technical information                                 HHS/ONC/CMS eMeasure
                                   posted on NQF's Web                                   contractors focusing on
                                   site. Technical support                               topics such as QDM and
                                   was also provided to                                  eMeasure logic in
                                   HHS/CMS/ONC as needed.                                preparation for the
                                                                                         release of MU2.
                                                                                         Participated in
                                                                                         eMeasure support calls
                                                                                         and meeting as
                                                                                         requested by ONC and
                                                                                         CMS.
Commissioned paper on data        Final report and          Completed.................  April 2012.
 sources and readiness of HIT      commissioned paper.
 systems to support care
 coordination.
Critical Paths..................  Examine new measurement   Completed.................  Patient Safety and Care
                                   areas (e.g., care                                     Coordination final
                                   plans) to understand                                  reports completed in
                                   the feasibility of                                    October and November
                                   measuring such areas in                               2012.
                                   an electronic
                                   environment.
eMeasure Learning Collaborative.  Examining issues related  Completed.................  Final report completed
                                   to implementation of                                  in December 2012.
                                   eMeasures with a multi-
                                   stakeholder group in
                                   order to define best
                                   practices and
                                   recommendations to the
                                   Office of the National
                                   Coordinator's Federal
                                   Advisory Committees.
eMeasure feasibility testing....  Review the current state  In progress...............  Draft guidance report to
                                   of feasibility                                        be finalized and
                                   assessment for                                        released for public
                                   eMeasures and identify                                comment. Slated for
                                   a set of principles,                                  completion by 4/5/13.
                                   recommendations, and
                                   criteria for adequate
                                   feasibility assessment.
Composite evaluation guidance...  Reassess NQF's existing   In progress...............  Final report slated for
                                   guidance for evaluating                               completion by 4/5/13.
                                   composites, with
                                   particular
                                   consideration of recent
                                   changes in composite
                                   measure development and
                                   related methodology.
----------------------------------------------------------------------------------------------------------------
                             III. Aligning Accountability Programs to Enhance Value
----------------------------------------------------------------------------------------------------------------
Measures for use in quality       Measure Applications      Completed.................  Completed February 2012.
 reporting programs under          Partnership Pre-
 Medicare.                         Rulemaking Report:
                                   Input on Measures Under
                                   Consideration by HHS
                                   for 2012 Rulemaking.
MAP report recommending measures  Final report including    Completed.................  June 1, 2012.
 that address the quality issues   potential new
 identified for dual-eligible      performance measures to
 beneficiaries.                    fill gaps in
                                   measurement for dual-
                                   eligible beneficiaries.
MAP report recommending measures  Final report including    Completed.................  June 1, 2012.
 for use in quality reporting      MAP Coordinating
 for Prospective Payment System-   Committee
 exempt cancer hospitals.          recommendations.
MAP report recommending measures  Final report including    Completed.................  June 1, 2012.
 for use in quality reporting      MAP Coordinating
 for hospice care.                 Committee
                                   recommendations.
MAP Strategic Plan 2012-2015....  Final report............  Completed.................  October 2012.
MAP report detailing families of  Final report............  Completed.................  October 2012.
 measures for safety, care
 coordination, cardiovascular
 conditions, and diabetes.

[[Page 46718]]

 
                     IV. Identifying Measure Gaps and Developing Strategies for Filling Them
----------------------------------------------------------------------------------------------------------------
Gaps Report.....................  ........................  ..........................  Feedback received on 2/
                                                                                         8. Revised draft due
                                                                                         back on 3/31/13.
----------------------------------------------------------------------------------------------------------------

Appendix B: NQF Board and Management Team

Board of Directors

William L. Roper, MD, MPH (Chair), Dean, School of Medicine, Vice 
Chancellor for Medical Affairs and Chief Executive Officer, UNC Health 
Care System, University of North Carolina at Chapel Hill
Helen Darling, MA (Vice Chair), President, National Business Group on 
Health
Gerald M. Shea (Treasurer and Interim CEO), Assistant to the President 
for External Affairs, AFL-CIO
Lawrence M. Becker, Director, HR Strategic Partnerships, Xerox 
Corporation
JudyAnn Bigby, MD, Secretary, Executive Office of Health & Human 
Services, Commonwealth of Massachusetts
Jack Cochran, MD, FACS, Executive Director, The Permanente Federation
Maureen Corry, Executive Director, Childbirth Connection
Leonardo Cuello, Staff Attorney, National Health Law Program
Joyce Dubow, Senior Health Care Reform Director, AARP Office of the 
Executive Vice-President for Policy and Strategy
Robert Galvin, MD, MBA, Chief Executive Officer, Equity Healthcare, The 
Blackstone Group
Ardis Dee Hoven, MD, Chair, Board of Trustees, American Medical 
Association
Charles N. Kahn III, MPH, President, Federation of American Hospitals
Donald Kemper, Chairman and CEO, Healthwise, Inc.
William Kramer, Executive Director for National Health Policy, Pacific 
Business Group on Health
Harold D. Miller, President and CEO, Network for Regional Healthcare 
Improvement
Elizabeth Mitchell, CEO, Maine Health Management Coalition
Dolores L. Mitchell, Executive Director, Commonwealth of Massachusetts 
Group Insurance Commission
Mary Naylor, Ph.D., RN, FAAN, Director, New Courtland Center for 
Transitions & Health and Marian S. Ware Professor in Gerontology, 
University of Pennsylvania School of Nursing
Debra L. Ness, President, National Partnership for Women & Families
Samuel R. Nussbaum, MD, Executive Vice President and Chief Medical 
Officer, WellPoint, Inc.
J. Marc Overhage, MD, Ph.D., Chief Medical Informatics Officer, Siemens 
Medical Solutions, Inc.
Bernard M. Rosof, MD, Chair, Board of Directors, Huntington Hospital, 
Chair, Physician Consortium for Performance Improvement (PCPI)
John C. Rother, JD, President and CEO, National Coalition on Health 
Care
Bruce Siegel, MD, MPH, President and Chief Executive Officer, National 
Association of Public Hospitals and Health Systems (NAPH)
Joseph R. Swedish, FACHE, President and CEO, Trinity Health
John Tooker, MD, MBA, MACP, Associate Executive Vice President, 
American College of Physicians
Richard J. Umbdenstock, FACHE, President and CEO, American Hospital 
Association

CMS

Patrick Conway, MD, Chief Medical Officer, Centers for Medicare & 
Medicaid Services

AHRQ

Carolyn M. Clancy, MD, Director, Agency for Healthcare Research and 
Quality
Designee: Nancy Wilson, MD, MPH, Senior Advisor to the Director

HRSA

Mary Wakefield, Ph.D., RN, Administrator, Health Resources and Services 
Administration
Designee: Terry Adirim, MD, Director, Office of Special Health Affairs

CDC

Thomas R. Frieden, MD, MPH, Director, Centers for Disease Control and 
Prevention
Designee: Peter A. Briss, MD, MPH, Captain, U.S. Public Health Service, 
Medical Director

EX OFFICIO (NON-VOTING):

Ann Monroe, (Chair, Consensus Standards Approval Committee), President, 
Health Foundation for Western and Central New York
Paul C. Tang, MD, MS, (Chair, Health Information Technology Advisory 
Committee) Vice President and Chief Medical Information Officer Palo 
Alto Medical Foundation

Management Team

Gerald Shea, Interim Chief Executive Officer
Karen Adams, Vice President, National Priorities
Heidi Bossley, Vice President, Performance Measures
Helen Burstin, Senior Vice President, Performance Measures
Ann Greiner, Vice President, Government Relations
Ann Hammersmith, General Counsel
Lisa Hines, Vice President, Member Relations
Rosemary Kennedy, Vice President, Health Information Technology
Nicole Silverman, Vice President, Program Operations
Lindsey Spindle, Senior Vice President, Communications and External 
Affairs
Diane Stollenwerk, Vice President, Stakeholder Collaboration
Jeffrey Tomitz, Chief Financial Officer, Accounting & Finance
Thomas Valuck, Senior Vice President, Strategic Partnerships
Kyle Vickers, Chief Information Office

Appendix C: MAP ``Working'' Measure Selection Criteria

1. Measures Within the Program Measure Set Are NQF-endorsed or Meet the 
Requirements for Expedited Review

    Measures within the program measure set are NQF-endorsed, 
indicating that they have met the following criteria: important to 
measure and report, scientifically acceptable measure properties, 
usable, and feasible. Measures within the program measure set that are 
not NQF-endorsed but meet requirements for expedited review, including 
measures in widespread use and/or tested, may be recommended by MAP, 
contingent on subsequent endorsement. These measures will be submitted 
for expedited review.

Response option: Strongly Agree/Agree/Disagree/Strongly Disagree


[[Page 46719]]


Measures within the program measure set are NQF-endorsed or meet 
requirements for expedited review (including measures in widespread use 
and/or tested)
Additional Implementation Consideration: Individual endorsed measures 
may require additional discussion and may be excluded from the program 
measure set if there is evidence that implementing the measure would 
result in undesirable unintended consequences.

2. Program Measure Set Adequately Addresses Each of the National 
Quality Strategy (NQS) priorities

    Demonstrated by measures addressing each of the National Quality 
Strategy (NQS) priorities:
Subcriterion 2.1 Safer care
Subcriterion 2.2 Effective care coordination
Subcriterion 2.3 Preventing and treating leading causes of mortality 
and morbidity
Subcriterion 2.4 Person- and family-centered care
Subcriterion 2.5 Supporting better health in communities
Subcriterion 2.6 Making care more affordable

Response option for each subcriterion: Strongly Agree/Agree/Disagree/
Strongly Disagree:

NQS priority is adequately addressed in the program measure set

3. Program Measure Set Adequately Addresses High-impact Conditions 
Relevant to the Program's Intended Population(s) (e.g., Children, Adult 
non-Medicare, Older Adults, Dual Eligible Beneficiaries)

    Demonstrated by the program measure set addressing Medicare High-
Impact Conditions; Child Health Conditions and risks; or conditions of 
high prevalence, high disease burden, and high cost relevant to the 
program's intended population(s). (Refer to tables 1 and 2 for Medicare 
High-Impact Conditions and Child Health Conditions determined by the 
NQF Measure Prioritization Advisory Committee.)

Response option: Strongly Agree/Agree/Disagree/Strongly Disagree:

Program measure set adequately addresses high-impact conditions 
relevant to the program.

4. Program Measure Set Promotes Alignment With Specific Program 
Attributes, as Well as Alignment Across Programs

    Demonstrated by a program measure set that is applicable to the 
intended care setting(s), level(s) of analysis, and population(s) 
relevant to the program.
Response option for each subcriterion:

Strongly Agree/Agree/Disagree/Strongly Disagree
Subcriterion 4.1 Program measure set is applicable to the program's 
intended care setting(s)
Subcriterion 4.2 Program measure set is applicable to the program's 
intended level(s) of analysis
Subcriterion 4.3 Program measure set is applicable to the program's 
population(s)

5. Program Measure Set Includes an Appropriate Mix of Measure Types

    Demonstrated by a program measure set that includes an appropriate 
mix of process, outcome, experience of care, cost/resource use/
appropriateness, and structural measures necessary for the specific 
program attributes.

Response option for each subcriterion: Strongly Agree/Agree/Disagree/
Strongly Disagree

Subcriterion 5.1 Outcome measures are adequately represented in the 
program measure set
Subcriterion 5.2 Process measures are adequately represented in the 
program measure set
Subcriterion 5.3 Experience of care measures are adequately represented 
in the program measure set (e.g. patient, family, caregiver)
Subcriterion 5.4 Cost/resource use/appropriateness measures are 
adequately represented in the program measure set
Subcriterion 5.5 Structural measures and measures of access are 
represented in the program measure set when appropriate

6. Program Measure Set Enables Measurement Across the Person-Centered 
Episode of Care \1\
---------------------------------------------------------------------------

    \1\ National Quality Forum (NQF), Measurement Framework: 
Evaluating Efficiency Across Patient-Focused Episodes of Care, 
Washington, DC: NQF; 2010.
---------------------------------------------------------------------------

    Demonstrated by assessment of the person's trajectory across 
providers, settings, and time.

Response option for each subcriterion: Strongly Agree/Agree/Disagree/
Strongly Disagree

Subcriterion 6.1 Measures within the program measure set are applicable 
across relevant providers
Subcriterion 6.2 Measures within the program measure set are applicable 
across relevant settings
Subcriterion 6.3 Program measure set adequately measures patient care 
across time

7. Program Measure Set Includes Considerations for Healthcare 
Disparities \2\
---------------------------------------------------------------------------

    \2\ NQF, Healthcare Disparities Measurement, Washington, DC: 
NQF; 2011.
---------------------------------------------------------------------------

    Demonstrated by a program measure set that promotes equitable 
access and treatment by considering healthcare disparities. Factors 
include addressing race, ethnicity, socioeconomic status, language, 
gender, age disparities, or geographical considerations (e.g., urban 
vs. rural). Program measure set also can address populations at risk 
for healthcare disparities (e.g., people with behavioral/mental 
illness).

Response option for each subcriterion: Strongly Agree/Agree/Disagree/
Strongly Disagree

Subcriterion 7.1 Program measure set includes measures that directly 
assess healthcare disparities (e.g., interpreter services)
Subcriterion 7.2 Program measure set includes measures that are 
sensitive to disparities measurement (e.g., beta blocker treatment 
after a heart attack)

8. Program Measure Set Promotes Parsimony

    Demonstrated by a program measure set that supports efficient 
(i.e., minimum number of measures and the least effort) use of 
resources for data collection and reporting and supports multiple 
programs and measurement applications. The program measure set should 
balance the degree of effort associated with measurement and its 
opportunity to improve quality.

Response option for each subcriterion: Strongly Agree/Agree/Disagree/
Strongly Disagree

Subcriterion 8.1 Program measure set demonstrates efficiency (i.e., 
minimum number of measures and the least burdensome)
Subcriterion 8.2 Program measure set can be used across multiple 
programs or applications (e.g., Meaningful Use, Physician Quality 
Reporting System [PQRS])

              Table 1--National Quality Strategy Priorities
------------------------------------------------------------------------
 
-------------------------------------------------------------------------
1. Making care safer by reducing harm caused in the delivery of care.
2. Ensuring that each person and family is engaged as partners in their
 care.
3. Promoting effective communication and coordination of care.
4. Promoting the most effective prevention and treatment practices for
 the leading causes of mortality, starting with cardiovascular disease.

[[Page 46720]]

 
5. Working with communities to promote wide use of best practices to
 enable healthy living.
6. Making quality care more affordable for individuals, families,
 employers, and governments by developing and spreading new healthcare
 delivery models.
------------------------------------------------------------------------


                     Table 2--High-Impact Conditions
------------------------------------------------------------------------
 
-------------------------------------------------------------------------
Medicare Conditions:
    1. Major Depression.
    2. Congestive Heart Failure.
    3. Ischemic Heart Disease.
    4. Diabetes.
    5. Stroke/Transient Ischemic Attack.
    6. Alzheimer's Disease.
    7. Breast Cancer.
    8. Chronic Obstructive Pulmonary Disease.
    9. Acute Myocardial Infarction.
    10. Colorectal Cancer.
    11. Hip/Pelvic Fracture.
    12. Chronic Renal Disease.
    13. Prostate Cancer.
    14. Rheumatoid Arthritis/Osteoarthritis.
    15. Atrial Fibrillation.
    16. Lung Cancer.
    17. Cataract.
    18. Osteoporosis.
    19. Glaucoma.
    20. Endometrial Cancer.
Child Health Conditions and Risks:
    1. Tobacco Use.
    2. Overweight/Obese (>=85th percentile BMI for age).
    3. Risk of Developmental Delays or Behavioral Problems.
    4. Oral Health.
    5. Diabetes.
    6. Asthma.
    7. Depression.
    8. Behavior or Conduct Problems.
    9. Chronic Ear Infections (3 or more in the past year).
    10. Autism, Asperger's, PDD, ASD.
    11. Developmental Delay (diag.).
    12. Environmental Allergies (hay fever, respiratory or skin
     allergies).
    13. Learning Disability.
    14. Anxiety Problems.
    15. ADD/ADHD.
    16. Vision Problems not Corrected by Glasses.
    17. Bone, Joint, or Muscle Problems.
    18. Migraine Headaches.
    19. Food or Digestive Allergy.
    20. Hearing Problems.
    21. Stuttering, Stammering, or Other Speech Problems.
    22. Brain Injury or Concussion.
    23. Epilepsy or Seizure Disorder.
    24. Tourette Syndrome.
------------------------------------------------------------------------

Appendix D: 2012 NQF Expert Participant Leaders (organized by 
committee)

Behavioral Health Steering Committee

Peter Briss, Co-Chair, National Center for Chronic Disease Prevention 
and Health Promotion
Harold Pincus, Co-Chair, Columbia University

Cancer Steering Committee

Stephen Edge, Co-Chair, Roswell Park Cancer Institute
Stephen Lutz, Chair, Blanchard Valley Regional Cancer Center

Cardiovascular Endorsement Maintenance 2010 Steering Committee

Mary George, Vice Chair, Centers for Disease Control and Prevention
Raymond Gibbons, Chair, Mayo Clinic

Care Coordination Steering Committee

Donald Casey, Co-Chair, Atlantic Health
Gerri Lamb, Co-Chair, Arizona State University

Common Formats Expert Panel

David Classen, Co-Chair, University of Utah School of Medicine
Henry Johnson, Co-Chair, ACS-MIDAS+

Council Leadership

Tanya Alteras, Chair, National Partnership for Women & Families
Maureen Corry, Vice Chair, Childbirth Connection
Deborah Fritz, Vice Chair, GlaxoSmithKline
Seiji Hayashi, Chair, Health Resources and Services Administration
David Hopkins, Chair, Pacific Business Group on Health
Thomas James, Chair, Humana Inc.
Carol Mullin, Chair, Virtua Health
Michael Phelan, Vice Chair, Cleveland Clinic
Louise Probst, Vice Chair, St. Louis Area Business Health Coalition
William Rich, Chair, Northern Virginia Ophthalomology Associates
Richard Salmon, Vice Chair, CIGNA HealthCare
David Shahian, Vice Chair, Massachusetts General Hospital
Kathleen Shoemaker, Chair, Lilly USA, LLC
Hussein Tahan, Vice Chair, New York Presbyterian Healthcare System
Marcia Wilson, Chair, Center for Health Care Quality

CSAC: Consensus Standards Approval Committee

Ann Monroe, Chair, Vice Chair, Health Foundation for Central & Western 
New York
Frank Opelka, Vice Chair, American College of Surgeons

GI & GU Pilot Project Steering Committee

Andrew Baskin, Co-Chair, Aetna
Christopher Saigal, Co-Chair, UCLA Medical Center

Health Information Technology Advisory Committee

J. Marc Overhage, Vice Chair, Siemens Medical Solutions USA, Inc.
Paul Tang, Chair, Palo Alto Medical Foundation

Healthcare Disparities & Cultural Competency Steering Committee

Dennis Andrulis, Co-Chair, Texas Health Institute
Denice Cora-Bramble, Co-Chair, Children's National Medical Center

HITAC Change Control Board

Floyd Eisenberg, Chair, NQF

HITAC Oversight and Testing Workgroup

Michael Lieberman, Chair, Oregon Health and Sciences University

HITAC Quality Data Model Subcommittee

David Bates, Chair, Brigham and Women's Hospital
Caterina Lasome, Co-Chair, iON Informatics, LLC

Infectious Disease Steering Committee

Steven Brotman, Co-Chair, The Advanced Medical Technology Association 
(AdvaMed)
Edward Septimus, Co-Chair, HCA

Leadership Network

William Corley, Chair, Community Health Network

MAP Cardiovascular and Diabetes Care Task Force

Christine Cassel, Chair, American Board of Internal Medicine

MAP Safety and Care Coordination Task Force

Frank Opelka, Chair, American College of Surgeons

MAP Strategy Task Force 2

Charles Kahn, Co-Chair, Federation of American Hospitals
Gerald Shea, Co-Chair, AFL-CIO

Measure Applications Partnership Clinician Workgroup

Mark McClellan, Chair, The Brookings Institute

Measure Applications Partnership Coordinating Committee

George Isham, Co-Chair, HealthPartners
Elizabeth McGlynn, Co-Chair, Kaiser Permanente Center for Effectiveness 
& Safety Research

[[Page 46721]]

Measure Applications Partnership Dual Eligibles Workgroup

Alice Lind, Chair, Center for Health Care Strategies, Inc

Measure Applications Partnership Hospital Workgroup

Frank Opelka, Chair, American College of Surgeons

Measure Applications Partnership PAC-LTC Workgroup

Carol Raphael, Chair, Visiting Nurse Service of New York

Multiple Chronic Conditions Measurement Framework Steering Committee

Caroline Blaum, Co-Chair, DVAMC GRECC Institute of Gerontology
Barbara McCann, Co-Chair, Interim HealthCare Inc.

National Priorities Partnership

Helen Darling, Co-Chair, National Business Group on Health
Bernard Rosof, Co-Chair, American Medical Association-Physician 
Consortium for Performance Improvement

Neurology Steering Committee

David Knowlton, Co-Chair, New Jersey Health Care Quality Institute
David Tirschwell, Co-Chair, University of Washington, Department of 
Neurology

NPP Maternity Action Team

Maureen Corry, Co-Chair, Childbirth Connection
Bernard Rosof, Co-Chair, American Medical Association-Physician 
Consortium for Performance Improvement

NPP Readmissions Action Team

Helen Darling, Co-Chair, National Business Group on Health
Susan Frampton, Co-Chair, Planetree

Oral Health Expert Panel

Paul Glassman, Co-Chair, University of the Pacific School of Dentistry
David Krol, Co-Chair, The Robert Wood Johnson Foundation

Palliative Care and End of Life Care Steering Committee

June Lunney, Co-Chair, Hospice and Palliative Nurses Association
Sean Morrison, Co-Chair, Mount Sinai School of Medicine--Dept. of 
Geriatrics & Palliative Medicine

Patient Safety State Based Reporting Work Group

Michael Doering, Co-Chair, Pennsylvania Patient Safety Authority
Diane Rydrych, Co-Chair, Minnesota Department of Health
Iona Thraen, Co-Chair, Utah Department of Health

Patient Safety-Measures Complications Steering Committee

Pamela Cipriano, Co-Chair, University of Virginia Health System
William Conway, Co-Chair, Henry Ford Health System

Perinatal and Reproductive Health Steering Committee

Laura Riley, Co-Chair, Massachusetts General Hospital
Carol Sakala, Co-Chair, Childbirth Connection

Population Health Steering Committee

Paul Jarris, Co-Chair, Association of State and Territorial Health 
Officers
Kurt Stange, Co-Chair, Case Western Reserve University

Pulmonary Steering Committee

Stephen Grossbart, Co-Chair, Catholic Health Partners
Kevin Weiss, Co-Chair, American Board of Medical Specialties

Readmissions Expedited Review Steering Committee

Sherrie Kaplan, Co-Chair, UC Irvine School of Medicine
Eliot Lazar, Co-Chair, New York Presbyterian Healthcare System

Regionalized Emergency Medical Care Services Steering Committee

Arthur Kellermann, Co-Chair, The RAND Corporation
Andrew Roszak, Co-Chair, HHS\HRSA

Resource Use Project Cancer TAP

David Penson, Chair, Vanderbilt University Medical Center

Resource Use Project Cardio/Diab TAP

Jeptha Curtis, Co-Chair, Yale University School of Medicine
James Rosenzweig, Co-Chair, Boston Medical Center and Boston University 
School of Medicine

Resource Use Project: Bone/Joint TAP

James Weinstein, Chair, Dartmouth-Hitchcock Medical Center

Resource Use Project: Pulmonary TAP

Kurtis Elward, Co-Chair, Family Medicine of Albermarle
Janet Maurer, Co-Chair, American College of Chest Physicians

Appendix E: 2012 NQF Expert Participants (organized by affiliation)

Barbara Kelly--A.F. Williams Family Medicine Center
Joyce Dubow--AARP
Naomi Karp--AARP
Susan Reinhard--AARP
Judith Cahill--Academy of Managed Care Pharmacy
Marissa Schlaifer--Academy of Managed Care Pharmacy
Henry Johnson--ACS-MIDAS+
Madhavi Vemireddy--ActiveHealth Management
Henry Claypool--Administration for Community Living, HHS
Joanne Armstrong--Aetna
Andrew Baskin--Aetna
Thomas Howe--Aetna
Randall Krakauer--Aetna
Patricia McDermott--Aetna
Gerald Shea--AFL-CIO
Marie Kokol--Agency for Health Care Administration
Carolyn Clancy--Agency for Healthcare Research and Quality
Erin Grace--Agency for Healthcare Research and Quality
Darryl Gray--Agency for Healthcare Research and Quality
Ernest Moy--Agency for Healthcare Research and Quality
William Munier--Agency for Healthcare Research and Quality
Mary Nix--Agency for Healthcare Research and Quality
Mamatha Pancholi--Agency for Healthcare Research and Quality
D.E.B. Potter--Agency for Healthcare Research and Quality
Judith Sangl--Agency for Healthcare Research and Quality
Nancy Wilson--Agency for Healthcare Research and Quality
MaryAnne Lindeblad--Aging and Disability Services Administration
Sam Fazio--Alzheimer's Association
Beth Kallmyer--Alzheimer's Association
Julie Lewis--Amedisys
Bruce Bagley--American Academy of Family Physicians
Dennis Saver--American Academy of Family Physicians
Dale Lupu--American Academy of Hospice and Palliative Medicine
Jack Scariano--American Academy of Neurology
Mary Jo Goolsby--American Academy of Nurse Practitioners
Douglas Burton--American Academy of Orthopaedic Surgeons
John Ratliff--American Association of Neurological Surgeons
Christine Zambricki--American Association of Nurse Anesthetists
Margaret Nygren--American Association on Intellectual and Developmental 
Disabilities
Christine Cassel--American Board of Internal Medicine
Lorna Lynn--American Board of Internal Medicine
Denece Kesler--American Board of Medical Specialties

[[Page 46722]]

Kevin Weiss--American Board of Medical Specialties
Larry Gilstrap--American Board of Obstetrics and Gynecology
Mary Maryland--American Cancer Society Illinois Division
Janet Maurer--American College of Chest Physicians
Lisa Moores--American College of Chest Physicians
Lorrie Kaplan--American College of Nurse-Midwives
Sean Currigan--American College of Obstetricians and Gynecologists
Gerald Joseph--American College of Obstetricians and Gynecologists
Sandra Fryhofer--American College of Physicians
Amir Qaseem--American College of Physicians
Don Detmer--American College of Surgeons
Bruce Hall--American College of Surgeons
Frank Opelka--American College of Surgeons
Sally Tyler--American Federation of State, County and Municipal 
Employees
Jennie Hansen--American Geriatrics Society
David Gifford--American Health Care Association
Ruta Kadonoff--American Health Care Association
Naomi Naierman--American Hospice Foundation
Nancy Foster--American Hospital Association
Richard Umbdenstock--American Hospital Association
Kalpana Ramiah--American Institutes for Research
Norman Edelman--American Lung Association
Kendra Hanley--American Medical Association
Delane Heldt--American Medical Association-Physician Consortium for 
Performance Improvement
Bernard Rosof--American Medical Association-Physician Consortium for 
Performance Improvement
James Lett--American Medical Directors Association
Sam Lin--American Medical Group Association
Maureen Dailey--American Nurses Association
Marla Weston--American Nurses Association
Patricia Conway-Morana--American Organization of Nurse Executives
Dianne Jewell--American Physical Therapy Association
Arden Morris--American Society of Colon and Rectal Surgeons
Shekhar Mehta--American Society of Health-System Pharmacists
Janet Brown--American Speech-Language-Hearing Association
Aparna Higgins--America's Health Insurance Plans
Andrea Gelzer--AmeriHealth Mercy Family of Companies
Richard Dutton--Anesthesia Quality Institute
Jay Schukman--Anthem Blue Cross and Blue Shield
Michael Helgeson--Apple Tree Dental
Gerri Lamb--Arizona State University
Craig Gilliam--Arkansas Children's Hospital
Catherine Tapp--Arkansas Department of Health and Human Services
Ann Hendrich--Ascension Health
Sarah Hill--Ascension Health
Joanne Conroy--Association of American Medical Colleges
Marilyn Bowman-Hayes--Association of periOperative Registered Nurses
Paul Jarris--Association of State and Territorial Health Officers
Shawn Polk--Association of State and Territorial Health Officials
Donald Casey--Atlantic Health
Michael Cantine--Atlantic Health
Roger Kurlan--Atlantic Health
Rhonda Anderson--Banner Health System
Ann de Velasco--Baptist Health South Florida
Thomas Giordano--Baylor College of Medicine
Jochen Profit--Baylor College of Medicine
Carl Couch--Baylor Health Care System
Jean De Leon--Baylor Health Care System
Robert Fine--Baylor Health Care System
Robert Watson--Baylor Health Care System
David Hackney--Beth Israel Deaconess Medical Center
Nancy Ridley--Betsy Lehman Center for Patient Safety and Medical Error 
Reduction
Patrick Murray--Better Health Greater Cleveland
Debra Bakerjian--Betty Irene Moore School of Nursing
Tiffany Osborn--BJC HealthCare
Stephen Lutz--Blanchard Valley Regional Cancer Center
Jane Franke--Blue Cross Blue Shield of Massachusetts
Greg Pawlson--BlueCross BlueShield Association
Carol Wilhoit--BlueCross BlueShield of Illinois
Kristine Anderson--BoozAllenHamilton
George Philippides--Boston Medical Center
James Rosenzweig--Boston Medical Center
Jeffrey Samet--Boston University School of Medicine
Lewis Kazis--Boston University School of Public Health
David Bates--Brigham and Women's Hospital
Daniel Forman--Brigham and Women's Hospital
Bruce Koplan--Brigham and Women's Hospital
Jeffrey Greenberg--Brigham and Women's Physicians' Organization
Richard Zane--Brigham Women's Hospital
Barbara Caress--Building Services 32BJ Health Fund
Lisa Shea--Butler Hospital
Carolyn Pare--Buyers Health Care Action Group
Neal Kohatsu--California Department of Health Care Services
Loriann DeMartini--California Department of Public Health
Kathleen O'Malley--California HealthCare Foundation
Ellen Wu--California Pan-Ethnic Health Network
Evelyn Calvillo--California State University
Janet Young--Carilion Health Systems
Jennifer Brandenburg--Carle Foundation Hospital
Suzanne Snyder--Carolinas Rehabilitation
Kurt Stange--Case Western Reserve University
Suzanne Delbanco--Catalyst for Payment Reform
Gail Amundson--Caterpillar Inc.
Stephen Grossbart--Catholic Health Partners
Zab Mosenifar--Cedars Sinai Medical Center
Kimberly Gregory--Cedars-Sinai Medical Center
Michael Langberg--Cedars-Sinai Medical Center
Rekha Murthy--Cedars-Sinai Medical Center
David Palestrant--Cedars-Sinai Medical Center
Marcia Wilson--Center for Health Care Quality, Department of Health 
Policy, George Washington University
Alice Lind--Center for Health Care Strategies, Inc
Elliot Sloane--Center for Healthcare Information Research and Policy
Arthur Levin--Center for Medical Consumers
Alfred Chiplin Jr.--Center for Medicare Advocacy, Inc.
Patricia Nemore--Center for Medicare Advocacy, Inc.
Terrence Batliner--Center for Native Oral Health Research
Diane Meier--Center to Advance Palliative Care
Peter Briss--Centers for Disease Control and Prevention
William Callaghan--Centers for Disease Control and Prevention
Mary George--Centers for Disease Control and Prevention

[[Page 46723]]

Catherine Gordon--Centers for Disease Control and Prevention
Gail Janes--Centers for Disease Control and Prevention
Chesley Richards--Centers for Disease Control and Prevention
Patrick Conway--Centers for Medicare & Medicaid Services
Maria Durham--Centers for Medicare & Medicaid Services
Kate Goodrich--Centers for Medicare & Medicaid Services
Shaheen Halim--Centers for Medicare & Medicaid Services
Shari Ling--Centers for Medicare & Medicaid Services
Cheryl Powell--Centers for Medicare & Medicaid Services
Michael Rapp--Centers for Medicare & Medicaid Services
Ashley Ridlon--Centers for Medicare & Medicaid Services
Marsha Smith--Centers for Medicare & Medicaid Services
Erin Smith--Centers for Medicare & Medicaid Services
Judith Tobin--Centers for Medicare & Medicaid Services
Alisa Ray--Certification Commission for Healthcare Information 
Technology
Parinda Khatri--Cherokee Health Systems
Maureen Corry--Childbirth Connection
Carol Sakala--Childbirth Connection
Ellen Schwalenstocker--Children's Hospital Association
Richard Antonelli--Children's Hospital Boston
Jenifer Lightdale--Children's Hospital Boston
Mark Schuster--Children's Hospital Boston
Trude Haecker--Children's Hospital of Philadelphia
David Einzig--Children's Hospitals and Clinics of Minnesota
Carol Kemper--Children's Mercy Hospital
Denice Cora-Bramble--Children's National Medical Center
David Stockwell--Children's National Medical Center
Joseph Wright--Children's National Medical Center
William Weintraub--Christiana Care Health System
Colette Edwards--CIGNA HealthCare
Mary Kay O'Neill--CIGNA HealthCare
Richard Salmon--CIGNA HealthCare
Uma Kotagal--Cincinnati Children's Hospital Medical Center
Thomas Loyacono--City of Baton Rouge and Parish of East Baton Rouge
Joseph Alvarnas--City of Hope
Jo Ann Brooks--Clarian Health
Jocelyn Bautista--Cleveland Clinic
Sung Hee Leslie Cho--Cleveland Clinic
Irene Katzan--Cleveland Clinic
David Lang--Cleveland Clinic
Thomas Marwick--Cleveland Clinic
Michael Phelan--Cleveland Clinic
Shannon Phillips--Cleveland Clinic
Allan Siperstein--Cleveland Clinic
Sharon Sutherland--Cleveland Clinic
Timothy Gilligan--Cleveland Clinic
Stanley Pestotnik--Cognovant, Inc.
Chris Tonozzi--Colorado Associated Community Health Information 
Enterprise
Kim Johnson--Colorado Department of Public Health and Environment
Wendy Tenzyk--Colorado Public Employees' Retirement Association
Arthur Cooper--Columbia University
Jacqueline Merrill--Columbia University
Bobbie Berkowitz--Columbia University School of Nursing
Lawrence Gottlieb--Commonwealth Care Alliance
Roger Snow--Commonwealth of Massachusetts
Dolores Mitchell--Commonwealth of Massachusetts --Group Insurance 
Commission
William Corley--Community Health Network
Andrea Benin--Connecticut Children's Medical Center
Cheryl Theriault--Connecticut Department of Health
Mary Alice Lee--Connecticut Voices for Children
E. Clarke Ross--Consortium for Citizens with Disabilities
Lawrence Sadwin--Consultant
Adam Thompson--Consultant
Richard Hanke--Consumer Representative
Robert Ellis--Consumers' Checkbook
Robert Krughoff--Consumers' Checkbook
Steven Findlay--Consumers Union
Lisa McGiffert--Consumers Union
Doris Peter--Consumers Union
Andrea Russo--Cooper University Hospital
Russell Acevedo--Crouse Hospital
Dolores Kelleher--D Kelleher Consulting
Richard Goldstein--Dana-Farber Cancer Institute
Saul Weingart--Dana-Farber Cancer Institute
John Wasson--Dartmouth-Hitchcock Medical Center
James Weinstein--Dartmouth-Hitchcock Medical Center
Linda Wilkinson--Dartmouth-Hitchcock Medical Center
Erik Pupo--Deloitte Consulting, LLP
Richard Albert--Denver Health Medical Center
Edward Havranek--Denver Health Medical Center
Philip Mehler--Denver Health Medical Center
Feseha Woldu--Department of Health and Human Services
Mary Sieggreen--Detroit Medical Center
Margaret Campbell--Detroit Receiving Hospital
Sharon Baskerville--District of Columbia Primary Care Association
Steve Morgenstern--Dow Chemical Company
Gwendolen Buhr--Duke University Health System
Sean O'Brien--Duke University Health System
John Clarke--ECRI Institute
Kathleen Shoemaker--Eli Lilly and Company
Nicole Tapay--Eli Lilly and Company
AnnMarie Papa--Emergency Nurses Association
Kathleen Szumanski--Emergency Nurses Association
Ricardo Martinez--Emory University School of Medicine
Amit Popat--Epic Systems Corp
Stanley Davis--Fairview Health Services
Brent Asplin--Fairview Medical Group
Kathleen Kelly--Family Caregiver Alliance
Kurtis Elward--Family Medicine of Albermarle
Allen McCullough--Fayette County Public Safety
Charles Kahn--Federation of American Hospitals
Nick Nudell--FirstWatch Solutions, Inc.
Joseph Ouslander--Florida Atlantic University
Laurie Burke--Food and Drug Administration
Jay Crowley--Food and Drug Administration
Behnaz Minaei--Food and Drug Administration
Terrie Reed--Food and Drug Administration
Terry Rogers--Foundation for Health Care Quality
Dwight Kloth--Fox Chase Cancer Center
Barbara Levy--Franciscan Health System
Dana Alexander--GE Healthcare
Brandon Savage--GE Healthcare
James Walker--Geisinger Health System
Andrew Guccione--George Mason University
Mayri Leslie--George Washington University
Robert Graham--George Washington University--School of Public Health
Michael Stoto--Georgetown University
Leslee Pool--Georgia Department of Health and Human Resources+D306
Rohit Borker--GlaxoSmithKline
Deborah Fritz--GlaxoSmithKline
Brenda Parker--GlaxoSmithKline
Richard Stanford--GlaxoSmithKline
John Derr--Golden Living, LLC
Connie Steed--Greenville Hospital System
Jason Colquitt--Greenway Medical Technologies

[[Page 46724]]

Anne Cohen--Harbage Consulting
John Gore--Harborview Medical Center
Ronald Maier--Harborview Medical Center
Paula Minton Foltz--Harborview Medical Center
David Spach--Harborview Medical Center
David Tirschwell--Harborview Medical Center
Jeffrey Greenwald--Harvard Medical School
Elsbeth Kalenderian--Harvard School of Dental Medicine
Ashish Jha--Harvard School of Public Health
Christine Klotz--Health Foundation for Central & Western New York
Ann Monroe--Health Foundation for Central & Western New York
Lyn Paget--Health Policy Partners
Ahmed Calvo--Health Resources and Services Administration
Ian Corbridge--Health Resources and Services Administration
Chris DeGraw--Health Resources and Services Administration
Leonard Epstein--Health Resources and Services Administration
Reem Ghandour--Health Resources and Services Administration
Seiji Hayashi--Health Resources and Services Administration
Sarah Linde-Feucht--Health Resources and Services Administration
Michael Lu--Health Resources and Services Administration
Samantha Meklir--Health Resources and Services Administration
Andrew Roszak--Health Resources and Services Administration
Mary Wakefield--Health Resources and Services Administration
John Seibel--HealthInsight New Mexico
Juliana Preston--HealthInsight Utah
Beth Averbeck--HealthPartners
David Gesko--HealthPartners
George Isham--HealthPartners
Thomas Kottke--HealthPartners
Thomas Von Sternberg--HealthPartners
Rick Luetkemeyer--HealthStrategy
Leslie Kelly Hall--Healthwise
Diane Limbo--Healthy Smiles for Kids of Orange County
John Pellicone--Helen Hayes Hospital
William Conway--Henry Ford Health System
Vanita Pindolia--Henry Ford Health System
Elizabeth Gilbertson--HEREIU Welfare Fund
Mary Blank--Highmark
Rubin Cohen--Hofstra University School of Medicine
June Lunney--Hospice and Palliative Nurses Association
Gail Austin Cooney--Hospice of Palm Beach County/Spectrum Health Inc.
Hayley Burgess--Hospital Corporation of America
Edward Septimus--Hospital Corporation of America
Louis Hoccheiser--Humana Inc.
Thomas James--Humana Inc.
Thomas James--Humana Inc.
Bryan Loy--Humana Inc.
Charles Stemple--Humana Inc.
Fredrik Tolin--Humana Inc.
Kyu Rhee--IBM
Mary Driscoll--Illinois Department of Public Health
Richard Snyder--Independence Blue Cross
Steve Udvarhelyi--Independence Blue Cross
Christopher Lamer--Indian Health Service
Steven Counsell--Indiana University School of Medicine
Floyd Fowler--Informed Medical Decision Making Foundation
Paula Graling--Inova Fairfax Hospital
Donald Goldmann--Institute for Healthcare Improvement
Sue Gullo--Institute for Healthcare Improvement
David Radley--Institute for Healthcare Improvement
Matthew Grissinger--Institute for Safe Medication Practices
Christina Michalek--Institute for Safe Medication Practices
Dolores Yanagihara--Integrated Healthcare Association
Allison Jackson--Intel
Barbara McCann--Interim HealthCare Inc.
Elizabeth Hammond--Intermountain Healthcare
Laura Heerman Langford--Intermountain Healthcare
Teri Kiehn--Intermountain Healthcare
Caterina Lasome--iON Informatics, LLC
Bob Russell--Iowa Department of Public Health
Meg Nugent--Iowa Healthcare Collaborative
Lance Roberts--Iowa Healthcare Collaborative
Nancy Zionts--Jewish Healthcare Foundation
Lisa Tripp--John Marshall Law School
Colleen Barry--Johns Hopkins Health System
Cynthia Boyd--Johns Hopkins Health System
Bruce Leff--Johns Hopkins Health System
Christoph Lehmann--Johns Hopkins Health System
Matthew McNabney--Johns Hopkins Health System
Robert Miller--Johns Hopkins Health System
Aaron Milstone--Johns Hopkins Health System
Lori Paine--Johns Hopkins Health System
Albert Wu--Johns Hopkins Health System
Patricia Abbott--Johns Hopkins University School of Nursing
David Domann--Johnson & Johnson Health Care Systems, Inc.
Christina Farup--Johnson & Johnson Health Care Systems, Inc.
Andy Amster--Kaiser Permanente
Amy Compton-Phillips--Kaiser Permanente
Douglas Corley--Kaiser Permanente
Sue Elam--Kaiser Permanente
Jamie Ferguson--Kaiser Permanente
Helen Lau--Kaiser Permanente
David Magid--Kaiser Permanente
Helene Martel--Kaiser Permanente
Ted Palen--Kaiser Permanente
David Pating--Kaiser Permanente
Elizabeth Paxton--Kaiser Permanente
Michael Schatz--Kaiser Permanente
Matt Stiefel--Kaiser Permanente
Jim Bellows--Kaiser Permanente
Jann Dorman--Kaiser Permanente
Elizabeth McGlynn--Kaiser Permanente
Lynn Searles--Kansas Department of Health and Environment
A.M. Barrett--Kessler Foundation
Bruce Pomeranz--Kessler Institute for Rehabilitation
Sean Muldoon--Kindred Healthcare
Laura Linebach--LA Care Health Plan
Rocco Ricciardi--Lahey Clinic Medical Center
Suma Thomas--Lahey Clinic Medical Center
Lauren Murray--Lamaze International
Paul Casale--Lancaster General Hospital
Cheryl Phillips--LeadingAge
Ian Chuang--Lockton Companies, LLC
Rebekah Gee--LSU School of Public Health
Anne Flanagan--Maine Department of Health
Elizabeth Mitchell--Maine Health Management Coalition
Ted Rooney--Maine Quality Counts
Scott Berns--March of Dimes
Cynthia Pellegrini--March of Dimes
Amit Acharya--Marshfield Clinic
Renee Webster--Maryland Department of Health
Elizabeth Daake--Massachusetts Department of Health
Joseph Betancourt--Massachusetts General Hospital
Liliana Bordeianou--Massachusetts General Hospital
Raymond Chung--Massachusetts General Hospital
Timothy Ferris--Massachusetts General Hospital
Elizabeth Mort--Massachusetts General Hospital
Laura Riley--Massachusetts General Hospital
Laura Riley--Massachusetts General Hospital
Karen Sepucha--Massachusetts General Hospital
David Shahian--Massachusetts General Hospital

[[Page 46725]]

David Torchiana--Massachusetts General Physicians Organization
David Polakoff--MassHealth
Robert Cima--Mayo Clinic
Pamela Foster--Mayo Clinic
Raymond Gibbons--Mayo Clinic
Catherine Roberts--Mayo Clinic
Eric Tangalos--Mayo Clinic
Karlene Phillips--Mayo Clinic
Gary Wingrove--Mayo Clinic
Charles Denk--MCH Epidemiology Program
Ginny Meadows--McKesson Corporation
Caroline Doebbeling--MDwise
Nicholas Sears--MedAssets, Inc.
Linus Santo Tomas--Medical College of Wisconsin
Peter Havens--Medical College of Wisconsin and Froedtert Hospital
Dana King--Medical University of South Carolina
Gail Stuart--Medical University of South Carolina
Zahid Butt--Medisolv, Inc.
Charlotte Alexander--Memorial Hermann Healthcare System
Roy Beasley--Memorial Hermann Healthcare System
M. Michael Shabot--Memorial Hermann Healthcare System
Lourdes Cuellar--Memorial Hermann Healthcare System--TIRR
David Pfister--Memorial Sloan-Kettering Cancer Center
Cristie Travis--Memphis Business Group on Health
Luther Clark--Merck & Co., Inc
Jennifer Bailit--MetroHealth Medical Center
Robin Shivley--Michigan Department of Health, EMS, and Trauma Systems
Michael O'Toole--Midwest Heart Specialists, Ltd.
Collette Pitzen--Minnesota Community Measurement
Diane Rydrych--Minnesota Department of Health
Vallire Hooper--Mission Hospital
Karen Fields--Moffitt Cancer Center
Jason Adelman--Montefiore Medical Center
Daniel Labovitz--Montefiore Medical Center
Helen Haskell--Mothers Against Medical Error
Leslie Zun--Mount Sinai Hospital
Peter Elkin--Mount Sinai Medical Center
R. Sean Morrison--Mount Sinai School of Medicine
Sean Morrison--Mount Sinai School of Medicine
Andrew Snyder--National Academy for State Health Policy
Gail Hunt--National Alliance for Caregiving
David Stevens--National Association of Community Health Centers
Robert Pestronk--National Association of County & City Health Officials
Denise Love--National Association of Health Data Organizations
Jane Hooker--National Association of Public Hospitals and Health 
Systems
Vickie Sears--National Association of Public Hospitals and Health 
Systems
Bruce Siegel--National Association of Public Hospitals and Health 
Systems
Jill Steinbruegge--National Association of Public Hospitals and Health 
Systems
Joan Zlotnik--National Association of Social Workers
Charles Moseley--National Association of State Directors of 
Developmental Disabilities Services
Martha Roherty--National Association of States United for Aging and 
Disabilities
Colleen Bruce--National Business Coalition on Health
Andrew Webber--National Business Coalition on Health
Dennis White--National Business Coalition on Health
Penney Berryman--National Business Group on Health
Helen Darling--National Business Group on Health
Pamela Kalen--National Business Group on Health
Sarah Brown--National Campaign to Prevent Teen and Unplanned Pregnancy
Steven Clauser--National Cancer Institute
Suzanne Heurtin-Roberts--National Cancer Institute
Linda Kinsinger--National Center for Health Promotion and Disease 
Prevention
Carol Allred--National Coalition for Women with Heart Disease
Mary Barton--National Committee for Quality Assurance
Margaret O'Kane--National Committee for Quality Assurance
Aldo Tinoco--National Committee for Quality Assurance
Phyllis Torda--National Committee for Quality Assurance
Michael Lardiere--National Council for Community Behavioral Healthcare
Nancy Whitelaw--National Council on Aging
Howard Kirkwood--National EMS Management Association
Keith Mason--National Forum for Heart Disease and Stroke Prevention
Brad Finnegan--National Governors Association
Marcia Thomas-Brown--National Health IT Collaborative for the 
Underserved
Leonardo Cuello--National Health Law Program
Deborah Reid--National Health Law Program
Mara Youdelman--National Health Law Program
Elena Rios--National Hispanic Medical Association
Carol Spence--National Hospice and Palliative Care Organization
Charles Homer--National Initiative for Children's Healthcare Quality
Jennifer Ustianov--National Initiative for Children's Healthcare 
Quality
Michael Lauer--National Institutes of Health
Marcel Salive--National Institutes of Health
Salina Waddy--National Institutes of Health
Adam Burrows--National PACE Association
Peter Schmidt--National Parkinson Foundation, Inc.
Tanya Alteras--National Partnership for Women & Families
Christine Bechtel--National Partnership for Women & Families
Debra Ness--National Partnership for Women & Families
Lee Partridge--National Partnership for Women & Families
Eva Powell--National Partnership for Women & Families
Kalahn Taylor-Clark--National Partnership for Women & Families
Janet Corrigan--National Quality Forum
Floyd Eisenberg--National Quality Forum
Laura Miller--National Quality Forum
Brock Slabach--National Rural Health Association
Robert Robin--Native Americans for Community Action, Inc.
Kathryn Blake--Nemours Foundation
Stephen Lawless--Nemours Foundation
Raj Sheth--Nemours Foundation
Mary Ann Clark--Neocure Group
Harold Miller--Network for Regional Healthcare Improvement
Bobbette Bond--Nevada Healthcare Policy Group LLC
Jay Kvam--Nevada State Health Division
Jose Montero--New Hampshire Department of Health and Human Services
Christine Stearns--New Jersey Business & Industry Association
Margaret Lumia--New Jersey Department of Health and Senior Services
David Knowlton--New Jersey Health Care Quality Institute
Ann Marie Sullivan--New York City Health and Hospitals Corporation
Eliot Lazar--New York Presbyterian Healthcare System
Harold Pincus--New York Presbyterian Healthcare System
Hussein Tahan--New York Presbyterian Healthcare System
Foster Gesten--New York State Department of Health

[[Page 46726]]

Norman Otsuka--New York University Hospital for Joint Diseases
Madeline Naegle--New York University, American Nurses Association
J. Emilio Carrillo--New York-Presbyterian Community Health Plan
Scott MacLean--Newton-Wellesley Hospital
Gregory Kapinos--North Shore-Long Island Jewish Health System
Louis Potters--North Shore-Long Island Jewish Health System
Kristofer Smith--North Shore-Long Island Jewish Health System
Jeffrey Susman--Northeast Ohio Medical University
William Rich--Northern Virginia Ophthalmology Associates
David Baker--Northwestern University
Romana Hasnain-Wynia--Northwestern University
David Stumpf--Northwestern University
Jane Sullivan--Northwestern University Feinberg School of Medicine
Mark Williams--Northwestern University Feinberg School of Medicine
Mary Jean Schumann--Nursing Alliance for Quality Care
Russell Leftwich--Office of eHealth Initiatives, State of Tennessee
Frank Johnson--Office of Employee Health & Benefits, State of Maine
Stephanie Mika--Office of the Assistant Secretary for Planning & 
Evaluation, HHS
Thomas Tsang--Office of the Governor, Hawaii
Jesse James--Office of the National Coordinator for Health Information 
Technology
Kevin Larsen--Office of the National Coordinator for Health Information 
Technology
Jacob Reider--Office of the National Coordinator for Health Information 
Technology
Joshua Seidman--Office of the National Coordinator for Health 
Information Technology
Allen Traylor--Office of the National Coordinator for Health 
Information Technology
Kaliyah Shaheen--Ohio Department of Health
Bernadette Melnyk--Ohio State University
Susan Moffatt-Bruce--Ohio State University
Michael Sayre--Ohio State University
Patrick Ross--Ohio State University Comprehensive Cancer Center--James 
Cancer Hospital
Gerene Bauldoff--Ohio State University, School of Nursing
Douglas Nee--OptiMed,Inc.
Mark Leenay--OptumHealth
Michael Lieberman--Oregon Health and Sciences University
Sydney Edlund--Oregon Patient Safety Commission
Roger Herr--Outcome Concept Systems
Kate Chenok--Pacific Business Group on Health
Emma Hoo--Pacific Business Group on Health
David Hopkins--Pacific Business Group on Health
Jennifer Huff--Pacific Business Group on Health
William Kramer--Pacific Business Group on Health
Seena Haines--Palm Beach Atlantic University
Paul Tang--Palo Alto Medical Foundation
Sue Pickens--Parkland Health & Hospital System
Michael Mirro--Parkview Health
Blackford Middleton--Partners HealthCare System, Inc.
Jason Spangler--Partnership for Prevention
Lori Frank--Patient Centered Outcomes Research Institute
Marci Nielsen--Patient Centered Primary Care Collaborative
Ron Stock--PeaceHealth Oregon Region
Chris Snyder--Peninsula Regional Medical Center
Peter Dillon--Penn State Hershey Medical Center
Michael Doering--Pennsylvania Patient Safety Authority
Eileen Kennedy--Pepco Holdings, Inc
Michael Ibara--Pfizer
Eleanor Perfetto--Pfizer
Laura Cranston--Pharmacy Quality Alliance
Kathleen Brady--Philadelphia Department of Public Health
Tina Cronin--Piedmont Medical Center
Susan Frampton--Planetree
Michael Lepore--Planetree
Richard Bankowitz--Premier healthcare alliance
Gina Pugliese--Premier healthcare alliance
Dennis Kaldenberg--Press Ganey Associates
Larry Cohen--Prevention Institute
James Lee--Providence Everett Medical Center
Robert Hellrigel--Providence Health & Services
Ron Bialek--Public Health Foundation
Mary Pittman--Public Health Institute
Louis Diamond--QHC Advisory Group, LLC
Dawn Fitzgerald--Qsource
Sharon Hibay--Quality Insights of Pennsylvania
Bonnie Paris--Quality Quest for Health of Illinois
David Seidenwurm--Radiological Associates of Sacramento Medical Group, 
Inc.
Leona Cuttler--Rainbow Babies and Children's Hospital
Arthur Kellermann--RAND Corporation
Debra Saliba--RAND Corporation
Kathleen Aller--Recommind, Inc.
Mary Van de Kamp--RehabCare
Darlene Skorski--Rhode Island Department of Health--Office of 
Facilities Regulation
David Krol--Robert Wood Johnson Foundation
Carey Smoak--Roche Laboratories, Inc.
Stephen Edge--Roswell Park Cancer Institute
Kathleen Lohr--RTI International
Ruth Kleinpell--Rush University Medical Center
Shannon Sims--Rush University Medical Center
Victoria Nahum--Safe Care Campaign
James Dunford--San Diego Fire-Rescue
Paul Merguerian--Seattle Children's Hospital
Rita Mangione-Smith--Seattle Children's Research Institute
Charissa Raynor--Service Employees International Union
Dale Shaller--Shaller Consulting Group
Karen Nielsen--Siemens Medical Solutions USA
J. Marc Overhage--Siemens Medical Solutions USA
Christopher Smiley--Smiley Family Dentistry, PC
Richard Bringewatt--SNP Alliance
William Grobman--Society for Maternal-Fetal Medicine
Kate Menard--Society for Maternal-Fetal Medicine
Mitchell Levy--Society of Critical Care Medicine
Janet Nagamine--Society of Hospital Medicine
Wendy Nickel--Society of Hospital Medicine
Howard Barnebey--Specialty Eyecare Centre
Jerad Widman--Spring Hill Family Medicine
Dennis Rivenburgh--St Anthony's
Mohamad Fakih--St. John Hospital and Medical Center
Kathleen Rice Simpson--St. John's Mercy Health Care
Joseph Laver--St. Jude Children's Research Hospital
Louise Probst--St. Louis Area Business Health Coalition
Mark Sanz--St. Patrick Hospital
Risha Gidwani--Stanford University Medical Center
John Morton--Stanford University Medical Center
Marc Leib--State of Arizona Medicaid Program
Ruth Leslie--State of New York Department of Health
John Maese--Staten Island University Hospital

[[Page 46727]]

Bruce Auerbach--Sturdy Memorial Hospital
Amina Chaudhry--Substance Abuse and Mental Health Services 
Administration
Frances Cotter--Substance Abuse and Mental Health Services 
Administration
Pamela Hyde--Substance Abuse and Mental Health Services Administration
Rita Vandivort-Warren--Substance Abuse and Mental Health Services 
Administration
Thomas File--Summa Health System
Tina Picchi--Supportive Care Coalition
Lois Cross--Sutter Health
A. John Blair--Taconic IPA, Inc.
Chad Bennett--Telligen
Julie Kuhle--Telligen
Liz Johnson--Tenet Healthcare Corporation
Ann Reed--Tennessee Department of Health
William Glomb--Texas Health and Human Services Commission
Dennis Andrulis--Texas Health Institute
Steven Brotman--The Advanced Medical Technology Association
Cheryl DeMars--The Alliance
Mark McClellan--The Brookings Institute
Anne-Marie Audet--The Commonwealth Fund
Mary Jane Koren--The Commonwealth Fund
Eugene Nelson--The Dartmouth Institute
Jesse Pines--The George Washington University Medical Center
Gerard Castro--The Joint Commission
Mark Chassin--The Joint Commission
Patricia Craig--The Joint Commission
Patricia Kurtz--The Joint Commission
Jerod Loeb--The Joint Commission
Crystal Riley--The Joint Commission
Heather Sherman--The Joint Commission
Margaret VanAmringe--The Joint Commission
Ann Watt--The Joint Commission
Susan Yendro--The Joint Commission
Leah Binder--The Leapfrog Group
Barbara Rudolph--The Leapfrog Group
Nadine Gracia--The Office of Minority Health
Mady Chalk--Treatment Research Institute
Paul Conlon--Trinity Health
Tami Mark--Truven Health Analytics
Randel Johnson--U.S. Chamber of Commerce
Salma Lemtouni--U.S. Food and Drug Administration
Philip Schoenfeld--UM Medical School
Jordan Eisenstock--UMass Memorial Medical Center
Devorah Rich--United Auto Workers Retiree Medical Benefits Trust
Rhonda Robinson Beale--United Behavioral Health
Barbara Corn--UnitedHealth Group
Rhonda Medows--UnitedHealth Group
Renae Stafford--University North Carolina
Alayne Markland--University of Alabama at Birmingham
Robert Weech-Maldonado--University of Alabama at Birmingham
Doug Campos-Outcalt--University of Arizona College of Medicine
Steven Chen--University of California Davis
Francis Lu--University of California Davis
Richard White--University of California Davis
Solomon Liao--University of California Irvine
Sherrie Kaplan--University of California Irvine School of Medicine
John Kusske--University of California Irvine School of Medicine
Nasim Afsar-manesh--University of California Los Angeles
Jim Crall--University of California Los Angeles
Bonnie Zima--University of California Los Angeles Center for Health 
Services & Society
Christopher Saigal--University of California Los Angeles Medical Center
Theodore Ganiats--University of California San Diego
Charlene Harrington--University of California San Francisco
Louise Walter--University of California San Francisco
Nancy Donaldson--University of California San Francisco School of 
Nursing
Marshall Chin--University of Chicago
William McDade--University of Chicago
William Dale--University of Chicago Medical Center
Nancy Lowe--University of Colorado Denver
Mark Metersky--University of Connecticut Health Center
Ramon Bautista--University of Florida HSC/Jacksonville
Tim Williamson--University of Kansas Medical Center
Katherine Reeder--University of Kansas School of Nursing
Judith Warren--University of Kansas School of Nursing
Joanna Sikkema--University of Miami, School of Nursing and Health 
Studies
William Barsan--University of Michigan Hospitals and Health Centers
James Carpenter--University of Michigan Hospitals and Health Centers
Elaine Chottiner--University of Michigan Hospitals and Health Centers
Curtis Collins--University of Michigan Hospitals and Health Centers
Karen Farris--University of Michigan Hospitals and Health Centers
Ella Kazerooni--University of Michigan Hospitals and Health Centers
Janet Larson--University of Michigan Hospitals and Health Centers
Jean Malouin--University of Michigan Hospitals and Health Centers
Marc Moote--University of Michigan Hospitals and Health Centers
Anne Pelletier Cameron--University of Michigan Hospitals and Health 
Centers
Linda Lindeke--University of Minnesota Amplatz Children's Hospital
Ira Moscovice--University of Minnesota Rural Health Research Center
Kristi Anne Henderson--University of Mississippi Medical Center
Bonnie Wakefield--University of Missouri
John Fildes--University of Nevada Las Vegas Medical Center
Ethan Basch--University of North Carolina at Chapel Hill
Jessica Lee--University of North Carolina at Chapel Hill
Sidney Smith--University of North Carolina at Chapel Hill
David Weber--University of North Carolina at Chapel Hill
Lynn Wegner--University of North Carolina School of Medicine
Lawrence Marks--University of North Carolina, School of Medicine
Dale Bratzler--University of Oklahoma Health Sciences Center
Mark Wolraich--University of Oklahoma Health Sciences Center
Judith Hibbard--University of Oregon
Leah Marcotte--University of Pennsylvania
Brendan Carr--University of Pennsylvania Health System
Lee Fleisher--University of Pennsylvania Health System
Jerry Johnson--University of Pennsylvania Health System
Frank Leone--University of Pennsylvania Health System
David Casarett--University of Pennsylvania School of Medicine
Kathryn Bowles--University of Pennsylvania School of Nursing
Nancy Hanrahan--University of Pennsylvania School of Nursing
Therese Richmond--University of Pennsylvania, School of Nursing
Douglas White--University of Pittsburgh
Donald Yealy--University of Pittsburgh Medical Center
Carl Sirio--University of Pittsburgh School of Medicine
Heidi Donovan--University of Pittsburgh School of Nursing

[[Page 46728]]

Laurent Glance--University of Rochester
Kevin Fiscella--University of Rochester School of Medicine
Jeffrey Beal--University of South Florida
Barbara Turner--University of Texas Health Science Center at San 
Antonio
Eduardo Bruera--University of Texas MD Anderson Cancer Center
Kenneth Ottenbacher--University of Texas Medical Branch at Galveston
Ethan Halm--University of Texas Southwestern Medical Center
Mambarambath Jaleel--University of Texas Southwestern Medical Center
Kathy Rinnert--University of Texas Southwestern Medical Center
Craig Rubin--University of Texas Southwestern Medical School
Victoria Jordan--University of Texas-MD Anderson Cancer Center
John Skibber--University of Texas-MD Anderson Cancer Center
Barbara Summers--University of Texas-MD Anderson Cancer Center
Ronald Walters--University of Texas-MD Anderson Cancer Center
Amy Hessel--University of Texas-MD Anderson Medical Center
Paul Glassman--University of the Pacific School of Dentistry
David Classen--University of Utah School of Medicine
Michael Farber--University of Vermont College of Medicine
Pamela Cipriano--University of Virginia Health System
Rachel Grob--University of Wisconsin Center for Patient Partnerships
Elizabeth Jacobs--University of Wisconsin, Department of Medicine
Patricia Brennan--University of Wisconsin-Madison
Tracy Schroepfer--University of Wisconsin-Madison
Christine Hunter--US Office of Personnel Management
John O'Brien--US Office of Personnel Management
Iona Thraen--Utah Department of Health
Jim Smith--Utica College
David Penson--Vanderbilt University Medical Center
W. Stuart Reynolds--Vanderbilt University Medical Center
Peter Almenoff--Veterans Health Administration
Caroline Blaum--Veterans Health Administration
John Duda--Veterans Health Administration
Stephan Fihn--Veterans Health Administration
Joseph Francis--Veterans Health Administration
Vivienne Halpern--Veterans Health Administration
Marcia Insley--Veterans Health Administration
Michael Kelley--Veterans Health Administration
Daniel Kivlahan--Veterans Health Administration
Robert Petzel--Veterans Health Administration
Patricia Quigley--Veterans Health Administration
Scott Shreve--Veterans Health Administration
Patricia Sinnott--Veterans Health Administration
Donna Washington--Veterans Health Administration
Edward Gill--Virginia Commonwealth University Medical Center
Cathie Furman--Virginia Mason Medical Center
Johannes Koch--Virginia Mason Medical Center
Jolynn Suko--Virginia Mason Medical Center
Carol Mullin--Virtua Health
Margaret Terry--Visiting Nurse Associations of America
Carol Raphael--Visiting Nurse Service of New York
Robert Rosati--Visiting Nurse Service of New York
William Frohna--Washington Hospital Center
Linda Furkay--Washington State Department of Health
David Mancuso--Washington State Department of Social & Health Services
Jeffery Thompson--Washington State Medicaid
Michael Kaplitt--Weill Cornell Medical College
Aron Halfin--WellPoint
Richard Hastreiter--WellPoint
Jennifer Malin--WellPoint
Sarah Sampsel--WellPoint
Grace Ting--WellPoint
Tracy Wang--WellPoint
Alonzo White--WellPoint
Christy Whetsell--West Virginia University Hospitals
Frank Ghinassi--Western Psychiatric Institute & Clinic of the 
University of Pittsburgh Medical Center
Lori Nichols--Whatcom Health Information Network
Christopher Queram--Wisconsin Collaborative for Healthcare Quality
John Bott--Wisconsin Department of Employee Trust Funds
Lois Sater--Wisconsin Division of Public Health
Nancy Faller--Wound, Ostomy and Continence Nurses Society
Jeptha Curtis--Yale New Haven Health System
Elizabeth Drye--Yale New Haven Health System
Marcella Nunez-Smith--Yale New Haven Health System
Patrick O'Connor--Yale New Haven Health System
Mary Tinetti--Yale New Haven Health System
Patricia Button--Zynx Health
David Rhew--Zynx Health

Appendix F: National Quality Forum--Background

    Despite the hard work of many, there is broad recognition that our 
healthcare system can do a better job on quality, safety, and 
affordability. This reality, in the context of a cost-conscious 
economy, has re-energized a national commitment to simultaneously 
improve care and responsibly constrain healthcare costs. State leaders, 
local governments, a broad swath of federal healthcare agencies, and an 
increasing number of other public- and private-sector organizations 
that constitute the quality movement are at the center of that 
resurgence. NQF is a public service organization that helps unite all 
of these organizations in their pursuit to make healthcare better, 
safer, and affordable.
    Established in 1999 as the standard-setting organization for 
healthcare performance measures, NQF today has a much-broadened mission 
to:
     Build consensus on national priorities and goals for 
performance improvement, and work in partnership with the public and 
private sectors to achieve them.
     Endorse and maintain best-in-class standards for measuring 
and publicly reporting on healthcare performance quality.
     Promote the attainment of national healthcare improvement 
goals and the use of standardized measures through education and 
outreach programs.
    NQF is recognized as a voluntary consensus standard-setting 
organization under the National Technology Transfer and Advancement Act 
of 1995. Its process for reaching consensus adheres to the Office of 
Management and Budget's formal definition of consensus.\31\
    The NQF Board of Directors governs the organization and is composed 
of 31 voting members--key public- and private-sector leaders who 
represent major stakeholders in America's healthcare system. Consumers 
and those who purchase healthcare hold a simple majority of the at-
large seats (see Appendix B). In 2012, NQF convened more than 800 
hundred experts across every stakeholder group who contributed their 
time, experience, and insights to measure-review, measure-selection, 
and priority-setting committees (see Appendix E).
    In recent years as part of a close working partnership with HHS, 
the

[[Page 46729]]

variety of NQF-endorsed measures has greatly expanded to address most 
settings of care, conditions, and provider types. NQF's measure 
portfolio includes measures of clinical process, patient experience of 
care, the actual outcomes of care, the costs and resources that go into 
providing care, as well as select structural measures. The portfolio is 
being enhanced with advanced measures, such as patient-reported 
outcomes and cross-cutting care-coordination measures. At the same 
time, NQF carefully manages its portfolio to be lean, retiring measures 
that no longer meet the more rigorous criteria. In the past year alone, 
430 measures were submitted to NQF and 301, or nearly 70 percent, were 
endorsed. This endorsement rate--or ratio of submitted to endorsed 
measures--reflects NQF's efforts to systematically raise the bar on 
performance measurement and to fill key measurement gap areas even as 
it aggressively seeks to reduce the burden on providers by eliminating 
duplicative measures that add unnecessary data collection and 
administrative workload.

       Percentage of Outcome Measures in NQF Portfolio, 2010-2012
------------------------------------------------------------------------
                                                           Percentage of
                                                              outcome
                          Year                              measures in
                                                             portfolio
------------------------------------------------------------------------
2010....................................................              18
2011....................................................              24
2012....................................................              27
------------------------------------------------------------------------

    To be NQF endorsed, a measure must capture a process or outcome 
that is important to measure and report, be scientifically acceptable, 
be feasible to collect, and provide useful results. NQF conducts an 
eight-step, consensus-based process for reviewing measures and other 
standards; this process has been continually improved over a decade, 
and is as follows:
    1. Call for Nominations allows anyone to suggest a candidate for 
the committee that will oversee the project. Committees are diverse, 
often encompassing experts in a particular field, providers, 
scientists, and consumers. After selection, NQF posts committee rosters 
on its Web site to solicit public comments on the composition of the 
panel and makes adjustments as needed to ensure balanced 
representation.
    2. Call for Measures starts a 30-day period for developers to 
submit a measure or practice through NQF's online submission forms.
    3. Steering Committee Review puts submitted measures to a four-part 
test to ensure they reflect sound science, will be useful to providers 
and patients, and will make a difference in improving quality. The 
expert steering committee conducts this detailed review in open 
sessions, each of which starts a limited period for public comment.
    4. Public Comment solicits input from anyone who wishes to respond 
to a draft report that outlines the steering committee's assessment of 
measures for possible endorsement. The steering committee may request a 
revision to the proposed measures.
    5. Member Vote asks NQF members to review the draft report and cast 
their votes on the endorsement of measures.
    6. CSAC Review marks the point at which the NQF Consensus Standards 
Approval Committee (CSAC) deliberates on the merits of the measure and 
the issues raised during the review process, and makes a recommendation 
on endorsement to the Board of Directors. The CSAC includes consumers, 
purchasers, healthcare professionals, and others. It provides the big 
picture to ensure that standards are being consistently assessed from 
project to project.
    7. Board Ratification asks for review and ratification by the NQF 
Board of Directors of measures recommended for endorsement.
    8. Appeal opens a period when anyone can appeal the Board's 
decision.
    Review committees comprise multiple stakeholders; consumer 
organizations and individual patients are equal partners with 
clinicians and other stakeholders throughout the process. There is a 
strong commitment to transparency: NQF invites public participation at 
every step, ranging from nominations for committees to comments and 
votes on specific measures. Endorsed measures are re-evaluated every 
three years to ensure their continuing relevance with current science 
and their actual use and usefulness in the field, and to determine 
whether they continue to represent the best in class compared to new 
measures. At any time, NQF can also conduct an ad hoc review of a 
measure if there is evidence of unintended consequences related to 
measurement or emerging clinical evidence that should result in a 
change to the measure.
    Measures included in the NQF portfolio are developed and maintained 
by about 65 different organizations including the Centers for Medicare 
and Medicaid Services (CMS), the National Committee for Quality 
Assurance (NCQA), the Physician Consortium for Performance Improvement, 
convened by the American Medical Association (AMA-PCPI), Ingenix, The 
Joint Commission, American College of Surgeons (ACS), Bridges to 
Excellence, Cleveland Clinic, Minnesota Community Measurement, and 
Pharmacy Quality Alliance.
    Many public- and private-sector leaders contributed to developing 
NQF's multi-stakeholder consensus process in the measure-endorsement 
realm. In recognition of this unique public service, HHS is required 
under statute to contract with a consensus-based entity, and contracted 
with NQF to convene diverse stakeholder groups to advise the public 
sector on priorities for healthcare improvement, related implementation 
strategies, and selection of measures to both drive these strategies 
and gauge results. The NQF-convened NPP and MAP and their published 
reports are tangible outcomes of this work. An equally important 
outcome of these partnerships is the ongoing alignment across 
stakeholder groups and across public- and private-sector leaders about 
which levers are most powerful in both improving healthcare performance 
and making the delivery system more patient centered.
    NQF was initially funded primarily through grants from major 
philanthropic foundations, including the Robert Wood Johnson Foundation 
and the Commonwealth Fund. NQF in turn built a strong membership base 
across all those who care about advancing healthcare quality; 
membership dues continue to provide annual funding for NQF's work.
    In 2012, NQF received $4.43 million a year in membership dues, an 
amount equaling 18 percent of its total budget. When combined with 
private foundation funding, 23 percent of NQF's budget comes from the 
private sector, with the remainder of its funding stemming from the 
public sector. In addition, the value of uncompensated donated time in 
2012--some 55,000 hours of work done on a volunteer basis by healthcare 
leaders and experts--is conservatively estimated to equal another $4 
million in private funding for NQF's work. Scaling up NQF's capacity 
became a necessity when the public sector, in its role as the largest 
American healthcare purchaser, made a serious commitment to buying 
healthcare based on value. This policy direction immediately generated 
the need for a more sustainable, steady resource that stood ready to 
regularly review and endorse performance measures.
    NQF has been fortunate to have received support from the federal 
government for more than 10 years, particularly since 2008 when federal 
leaders strongly committed themselves

[[Page 46730]]

to designing and implementing a value-driven agenda for healthcare. 
More specifically:
     MIPPA has provided NQF with $10 million annually over a 
four-year period starting in 2009, which was extended for FY 2013 by 
HR8 (PL 112-240). These funds--awarded to NQF through a competitive 
process--support the organization's efforts to identify priority areas 
for improvement, endorse and update related performance measures, 
foster the transition to an electronic environment, and report annually 
to Congress on the status and progress to date of this effort.
    ACA has provided NQF with support of about $10 million annually, 
starting in 2011. Under Section 3014, Congress directed HHS to contract 
with ``the consensus-based entity under contract'' to provide multi-
stakeholder input into the NQS, as well as input to the Secretary of 
HHS on the selection of measures for use in various quality programs 
that utilize the federal rulemaking process for measure selection.

IV. Secretarial Comments on the Annual Report to Congress

    This 2013 Annual Report describes NQF's work in 2012 to fulfill the 
requirements specified in section 1890 of the Social Security Act. This 
section of the Social Security Act requires the Secretary of the 
Department of Health and Human Services to ``have in effect a contract 
with a consensus-based entity, such as the National Quality Forum,'' to 
perform certain duties including those related to performance 
measurement and NQS priorities. The Social Security Act also requires 
by not later than March 1 of each year (beginning with 2009), that the 
CBE shall submit to Congress and the Secretary of the Department of 
Health and Human Services a report containing a description of:

    (i) Implementation of quality and efficiency measurement 
initiatives under the Social Security Act and the coordination of 
such initiatives with quality and efficiency initiatives implemented 
by other payers;
    (ii) recommendations on an integrated national strategy and 
priorities for health care performance measurement;
    (iii) performance of its duties required under its contract with 
HHS;
    (iv) gaps in endorsed quality and efficiency measures, and where 
quality and efficiency measures are unavailable or inadequate to 
identify or address such gaps;
    (v) areas in which evidence is insufficient to support 
endorsement of quality and efficiency measures in priority areas 
identified by the Secretary under the national strategy and where 
targeted research may address such gaps; and
    (vi) convening multi-stakeholder groups to provide input on: 1) 
The selection of quality and efficiency measures for use in various 
Medicare programs, in reporting performance information to the 
public; and in other health care programs; and 2) national 
priorities for improvement in population health and the delivery of 
health care services for consideration under the national quality 
strategy.

    This 2013 report fulfills the statutory requirement for the annual 
report described above and describes the results of work that NQF, as 
the CBE, undertook in 2012.
    For example, in 2012, NQF managed its portfolio of more than 700 
endorsed measures by replacing some measures with improved measures; 
removing measures that were no longer effective or where the evidence 
base had evolved; and expanding the portfolio to address well-
recognized measurement gaps. NQF reviewed 430 submitted measures and 
endorsed 301 of them. This set included 81 new measures and 220 
measures that maintained their endorsement after being considered in 
light of new evidence and/or against new competing measures submitted 
to NQF for consideration. The newly endorsed measures align with needs 
identified in the NQS and address several critical areas, including 
patient outcomes, underserved populations, healthcare disparities, and 
hospital readmissions.
    In 2012, NQF's National Priorities Partnership (NPP), a 
collaborative public-private partnership, focused on how to advance 
patient safety by aligning its work with HHS' ``Partnership for 
Patients'' initiative. Through a series of web-based and in-person 
meetings, nearly 2,700 participants from multiple sectors learned about 
and shared new improvement approaches, information, tools, and 
professional connections to improve health care safety. The NPP also 
developed action plans to focus a range of national and local 
organizations in diverse sectors on how to align efforts to reduce 
preventable readmissions and improve maternity care, and created a web-
based ``action registry'' to track improvement activities focused on 
readmissions and maternity care to enable learning across participants. 
Launched in the fourth quarter of 2012, by March 2013, the registry 
housed over 50 actions by 30 different organizations.
    In 2012, NQF also continued its work to facilitate the electronic 
reporting of quality measures using electronic health records (EHRs) 
that health care providers across the nation are adopting. NQF's work 
on these ``eMeasures'' included standardizing data elements so the same 
quality of care information can be collected from different EHRs. NQF 
also convened an eMeasure Learning Collaborative to help multiple 
parties address barriers to developing and implementing eMeasures.
    NQF's Measure Applications Partnership (MAP) provided multi-
stakeholder input to HHS about the potential use of quality measures in 
more than 17 different Medicare quality reporting and performance 
programs and the Medicare and Medicaid Electronic Health Record (EHR) 
Incentive Program. This input was critical to HHS programs. At the same 
time, MAP released its Families of Measures report, which defined 
measure families in four key areas--safety, care coordination, 
cardiovascular, and diabetes care--with the goal of promoting more 
cohesion and integration of care regardless of setting, provider, level 
of care intensity, or timing of care.
    In 2012, NQF also conducted an analysis of its current measures 
portfolio against both the NQS priority areas and high-impact Medicare 
and child health conditions. This analysis found that while many NQF 
measures address patient safety, fewer measures address patient and 
family engagement. For example, measures of shared decision-making, 
patient navigation and self-management, healthy lifestyle behaviors, 
community interventions to improve health, and access, cost, and 
resource use are significantly less prevalent than safety measures. The 
analysis also found gaps in measures of preventive care, patient-
reported outcomes (particularly quality of life and functional status), 
appropriateness (particularly for specialty care), access to timely 
palliative care, and health and healthcare disparities. Additionally, 
the analysis revealed the need for better population-level measures to 
assess improvements in health and healthcare. And, while certain high-
impact conditions common to adults have an abundance of measures--e.g., 
cardiovascular disease, end-stage renal disease, and diabetes--many of 
the high-impact childhood conditions have few or no NQF-endorsed 
measures.
    These and the other activities described in the Annual Report 
reflect the wide scope of work required for sound measurement of health 
care quality--and the accompanying hard work needed for the continued 
improvement of health care. HHS thanks NQF for its hard work and 
submission of this report.

[[Page 46731]]

V. Future Steps

    The work reflected in this annual report was produced under HHS' 
initial four-year contract to NQF which was executed in 2009 and will 
expire in 2013.
    To continue to fulfill the statutory requirement for a contract 
with a consensus-based entity, HHS competitively procured a new 
contract with NQF in September 2012. Through this new contract, NQF 
will continue to perform the statutory activities for the CBE described 
above in support of HHS' efforts to achieve the aims of the NQS-- 
better care, healthier people and communities, and affordable care.

VI. Collection of Information Requirements

    This document does not impose information collection and 
recordkeeping requirements. Consequently, it need not be reviewed by 
the Office of Management and Budget under the authority of the 
Paperwork Reduction Act of 1995 (44 U.S.C. 35)
    \1\ Measure Applications Partnership. Pre-Rulemaking Report: 
Input on Measures Under Consideration by HHS for 2012 Rulemaking. 
Washington, DC: National Quality Forum, 2013.
    \2\ National Quality Forum. NQF's Portfolio of Measures: Who is 
Using it, and how is it Evolving? Washington, DC: National Quality 
Forum, January 2012.
    \3\ National Quality Forum. NQF Measure Portfolio Report. 
Washington, DC: National Quality Forum, November 2012.
    \4\ Damberg CL, Sorbero ME, Lovejoy SL et al. An Evaluation of 
the Use of Performance measures in Health Care. Santa Monica, CA: 
RAND Corporation; 2011. Available at http:[sol][sol]www.rand.org/
content/dam/rand/pubs/technical_reports/2011/RAND_TR1148.pdf. 
Accessed December 2012.
    \5\ Main E, Oshiro B, Chagolla B, Bingham D, Dang-Kilduff L, and 
Kowalewski L. Elimination of Non-medically Indicated (Elective) 
Deliveries Before 39 Weeks Gestational Age. (California Maternal 
Quality Care Collaborative Toolkit to Transform Maternity Care). 
Developed under contract 08-85012 with the California 
Department of Public Health; Maternal, Child and Adolescent Health 
Division; First edition published by March of Dimes, July 2010.
    \6\ Childbirth Connection. Vaginal or Cesarean Birth: What Is at 
Stake for Women and Babies? New York: Childbirth Connection; 2012. 
Available at http:[sol][sol]transform.childbirthconnection.org/
reports/cesarean.
    \7\ Sakala C, Corry MP. Evidence-Based Maternity Care: What It 
Is and What It Can Achieve. New York: Milbank Memorial Fund in 
collaboration with Childbirth Connection and Reforming States Group; 
2008. Available at http:[sol][sol]www.childbirthconnection.org/pdfs/
evidence-based-maternity-care.pdf.
    \8\ Jencks SF, Williams MV, Coleman EA, Rehospitalizations among 
patients in the Medicare fee-for-service program, New Engl J Med, 
2009;360(14):1420-1421.
    \9\ Pennsylvania Patient Safety Advisory. Leveraging healthcare 
policy changes to decrease hospital 30-day readmission rates, Pa 
Patient Saf Advis, 2010 March;7(1):1-8.
    \10\ Medicare Payment Advisory Commission. Report to Congress: 
Promoting Greater Efficiency in Medicare. Washington, DC; 2007. Pp. 
103-199.
    \11\ Saliba D, Kington R, Buchanan J, et al., Appropriateness of 
the decision to transfer nursing facility residents to hospital, J 
Am Geriatr Soc, 2000;48:154-163.
    \12\ Medicare Payment Advisory Commission (MEDPAC). Report to 
the Congress: Reforming the Delivery System. Washington, DC: MedPAC; 
2008. Available at http:[sol][sol]medpac.gov/documents/Jun08--
EntireReport.pdf. Accessed October 2011.
    \13\ Institute of Medicine. To Err is Human. Washington, DC: 
National Academies Press; 2001.
    \14\ Banks J, et al., Disease and disadvantage in the United 
States and in England, JAMA, 2006;295(17):2037-2045.
    \15\ Hoyert DL, et al., Annual summary of vital statistics: 
2004, Pediatrics, 2006; 117(1):168-183.
    \16\ Weiss JE, Mushinski M, International mortality rates and 
life expectancy: selected countries, Statistical Bulletin--
Metropolitan Life Insurance Company, 1999;80(1):13-21.
    \17\ Department of Health and Human Services (HHS), Office of 
the Assistant Secretary for Health (ASH), Initiatives, Washington, 
DC: HHS, ASH: 2011. Available at http:[sol][sol]www.hhs.gov/ophs/
initiatives/mcc/. Last accessed December 2011.
    \18\ Thorpe KE, Howard DH, The rise in spending among Medicare 
beneficiaries: the role of chronic disease prevalence and changes in 
treatment intensity, Health Aff, 2006;25(5):w378-w388.
    \19\ Gijsen R, Hoeymans N, Schellevis FG, et al., Causes and 
consequences of comorbidity: a review, J Clin Epidemiol, 
2001;54(7):661-674.
    \20\ Boult C, Wieland GD, Comprehensive primary care for older 
patients with multiple chronic conditions: ``nobody rushes you 
through'', JAMA, 2010;304(17):1936-1943.
    \21\ Parekh AK, Barton MB, The challenge of multiple comorbidity 
for the US health care system, JAMA, 2010;303(13):1303-1304.
    \22\ Wolff JL, Starfield B, Anderson G, Prevalence, 
expenditures, and complications of multiple chronic conditions in 
the elderly, Arch Intern Med, 2002;162(20):2269-2276.
    \23\ Boyd CM, Boult C, Shadmi E, et al., Guided care for 
multimorbid older adults, Gerontologist, 2007;47(5):697-704.
    \24\ Institute of Medicine (IOM). Unequal Treatment: Confronting 
Racial and Ethnic Disparities in Health Care. Washington, DC: 
National Academies Press; 2003. Available at 
http:[sol][sol]www.nap.edu/openbook. php?isbn=030908265X. Last 
accessed August 2012.
    \25\ Measure Applications Partnership. Pre-Rulemaking Report: 
Input on Measures Under Consideration by HHS for 2012 Rulemaking. 
Washington, DC: National Quality Forum, 2013.
    \26\ Measure Applications Partnership. Pre-Rulemaking Report: 
Input on Measures Under Consideration by HHS for 2012 Rulemaking. 
Washington, DC: National Quality Forum, 2013.
    \27\ Damberg CL, Sorbero ME, Lovejoy SL et al. An Evaluation of 
the Use of Performance measures in Health Care. Santa Monica, CA: 
RAND Corporation; 2011. Available at http:[sol][sol]www.rand.org/
content/dam/rand/pubs/technical_reports/2011/RAND_TR1148.pdf. 
Accessed December 2012.
    \28\ National Quality Forum. NQF's Portfolio of Measures: Who is 
Using it, and how is it Evolving? Washington, DC: National Quality 
Forum, January 2012.
    \29\ National Quality Forum. NQF Measure Portfolio Report. 
Washington, DC: National Quality Forum, November 2012.
    \30\ See rwjf.org/en/about-rwjf/newsroom/interactives/71857.html.
    \31\ The White House, U.S. Office of Management and Budget 
(OMB). Circular No. A-119, February 10, 1998. Washington, DC: OMB; 
1998. Available at www.whitehouse.gov/omb/circulars_a119/. Last 
accessed January 2012.

    Dated: July 25, 2013.
Kathleen Sebelius,
Secretary, Department of Health and Human Services.
[FR Doc. 2013-18478 Filed 7-31-13; 8:45 am]
BILLING CODE 4150-05-P