Medicare Program; Evaluation Criteria and Standards for Quality Improvement Program Contracts, 42331-42336 [05-14505]
Download as PDF
<![CDATA[
Federal Register / Vol. 70, No. 140 / Friday, July 22, 2005 / Notices
relate to physicians’ services, identified
by the Secretary. Council members are
expected to participate in all meetings.
Section 1868(a)(3) of the Act provides
for payment of expenses and a per diem
allowance for Council members at a rate
equal to payment provided members of
other advisory committees. In addition
to making these payments, the
Department of Health and Human
Services and CMS provide management
and support services to the Council. The
Secretary will appoint new members to
the Council from among those
candidates determined to have the
expertise required to meet specific
agency needs in a manner to ensure
appropriate balance of the Council’s
membership.
The Council held its first meeting on
May 11, 1992. The current members are:
Ronald Castellanos, M.D. Chairperson;
Jose Azocar, M.D.; M. Leroy Sprang,
M.D.; Rebecca Gaughan, M.D.; Peter
Grimm, D.O.; Carlos R. Hamilton, M.D.;
Dennis K. Iglar, M.D.; Joe Johnson, D.C.;
Christopher Leggett, M.D.; Barbara
McAneny, M.D.; Geraldine O’Shea,
D.O.; Laura B. Powers, M.D.; Gregory J.
Przybylski, M.D.; Anthony Senagore,
M.D.; and Robert L. Urata, M.D.
The meeting will commence with the
swearing-in of one Council member.
The Council’s Executive Director will
give a status report and the CMS
responses to the recommendations made
by the Council at the May 23, 2005
meeting and prior meeting
recommendations. Additionally, an
update will be provided on the
Physician Regulatory Issues Team. In
accordance with the Council charter, we
are requesting assistance with the
following agenda topics:
• Competitive Acquisition for Drugs.
• Physician Fee Schedule Proposed
Rule.
• Part D Prescription Drug Program.
• Outpatient Proposed Rule.
• Surgical Care Improvement
Partnership Program.
• Alliance for Cardiac Care
Excellence Program.
• NPI-Outreach and Implementation.
For additional information and
clarification on these topics, contact the
DFO as provided in the FOR FURTHER
INFORMATION CONTACT section of this
notice. Individual physicians or medical
organizations that represent physicians
wishing to make a 5-minute oral
presentation on agenda issues must
contact the DFO by 12 noon, e.d.t.,
August 5, 2005, to be scheduled.
Testimony is limited to agenda topics
only. The number of oral presentations
may be limited by the time available. A
written copy of the presenter’s oral
remarks must be submitted to Kelly
VerDate jul<14>2003
19:28 Jul 21, 2005
Jkt 205001
Buchanan, DFO, no later than 12 noon,
e.d.t., August 5, 2005, for distribution to
Council members for review prior to the
meeting. Physicians and medical
organizations not scheduled to speak
may also submit written comments to
the DFO for distribution no later than
noon, e.d.t., August 5, 2005. The
meeting is open to the public, but
attendance is limited to the space
available.
Special Accommodations: Individuals
requiring sign language interpretation or
other special accommodation must
contact the DFO by e-mail at
PPAC@cms.hhs.gov or by telephone at
(410) 786–6132 at least 10 days before
the meeting.
Authority: (Section 1868 of the Social
Security Act (42 U.S.C. 1395ee) and section
10(a) of Pub. L. 92–463 (5 U.S.C. App. 2,
section 10(a).)
Dated: July 11, 2005.
Mark McClellan,
Administrator, Centers for Medicare &
Medicaid Services.
[FR Doc. 05–14154 Filed 7–21–05; 8:45 am]
BILLING CODE 4120–01–P
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Centers for Medicare & Medicaid
Services
Statement of Organization, Functions,
and Delegations of Authority
Part F., Section F.70. (Order of
Succession) of the Statement of
Organization, Functions, and
Delegations of Authority for the
Department of Health and Human
Services, Centers for Medicare &
Medicaid Services (CMS), (Federal
Register, Vol. 49, No. 174, p. 35251,
dated September 6, 1984) is hereby
rescinded and replaced by the following
new Section F.70.
F.70. Order of Succession
During any period when the
Administrator, Centers for Medicare &
Medicaid Services (CMS), has died,
resigned, or otherwise become unable to
perform the functions and duties of the
office of the Administrator, CMS, the
following officers, in the order listed,
shall act for and perform the functions
and duties of the office of
Administrator, CMS, until such time the
Administrator, CMS, again becomes
available, a permanent successor is
appointed, or the temporary successor is
otherwise relieved:
1. Deputy Administrator.
2. Chief Operating Officer.
PO 00000
Frm 00033
Fmt 4703
Sfmt 4703
42331
3. Director, Center for Medicare
Management.
4. Deputy Chief Operating Officer.
5. Director & Chief Financial Officer,
Office of Financial Management.
6. Deputy Director, Center for
Medicare Management.
7. Deputy Director, Office of Financial
Management.
The authority to act as the
Administrator, CMS, must be exercised
in accordance with the provisions of the
Federal Vacancies and Reform Act of
1998 (‘‘the Vacancies Act’’), 5 U.S.C.
3345 et seq. The ‘‘Acting’’ title is
applicable and reserved only in
instances in which the CMS
Administrator position is vacant. In
accordance with the Vacancies Act, the
Deputy Administrator is herein
designated as the first assistant for CMS.
During a planned absence, the
Administrator, CMS, may designate an
individual to serve as ‘‘operationally in
charge.’’ No individual who is serving
in an ‘‘operationally in charge’’ capacity
shall exercise this authority unless he or
she is herein designated as a delegatee.
This authority is limited to
maintaining the Agency’s essential
functions and restoring the Agency’s
normal business functions under the
CMS Continuity of Operations Plan
(COOP).
Dated: June 16, 2005.
Mark B. McClellan,
Administrator, Centers for Medicare &
Medicaid Services.
[FR Doc. 05–14148 Filed 7–21–05; 8:45 am]
BILLING CODE 4120–01–M
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Centers for Medicare & Medicaid
Services
[CMS–3142–FN]
Medicare Program; Evaluation Criteria
and Standards for Quality
Improvement Program Contracts
Centers for Medicare &
Medicaid Services (CMS), HHS.
ACTION: Final notice.
AGENCY:
SUMMARY: This final notice describes the
evaluation criteria we will use to
evaluate the Quality Improvement
Organizations (QIOs) under their
contracts with us, for efficiency and
effectiveness in accordance with the
Social Security Act. These evaluation
criteria are based on the tasks and
related subtasks set forth in the QIO’s
Scope of Work (SOW). The current 7th
SOW includes Tasks 1 through 4, with
subtasks included under all tasks,
E:\FR\FM\22JYN1.SGM
22JYN1
42332
Federal Register / Vol. 70, No. 140 / Friday, July 22, 2005 / Notices
excluding Task 4. QIOs were awarded
contracts for the 7th SOW, or 7th
Round, for 3 years, with staggered
starting dates beginning August 2002,
November 2002, and February 2003.
This final notice also responds to the
public comments received regarding the
evaluation criteria published in July
2004.
DATES: Effective August 22, 2005.
FOR FURTHER INFORMATION CONTACT:
Maria Hammel, (410) 786–1775.
SUPPLEMENTARY INFORMATION:
I. Background
The Peer Review Improvement Act of
1982 (Title I, Subtitle C of Public Law
97–248) amended Part B of Title XI of
the Social Security Act (the Act) to
establish the Peer Review Organization
(PRO) program. The PRO program (now
called the Quality Improvement
Organization (QIO) program) was
established to redirect, simplify, and
enhance the cost-effectiveness and
efficiency of the medical peer review
process. Sections 1152 and 1153 of the
Act define the types of organizations
eligible to become QIOs, and establish
certain limitations and priorities
regarding QIO contracting.
The Secretary enters into contracts
with QIOs to perform three broad
functions:
• Improve quality of care for
beneficiaries by ensuring that
beneficiary care meets professionally
recognized standards of health care;
• Protect the integrity of the Medicare
Trust Fund by ensuring that Medicare
only pays for services and items that are
reasonable and medically necessary and
that are provided in the most
economical setting;
• Protect beneficiaries by
expeditiously addressing individual
cases such as beneficiary quality of care
complaints, contested hospital issued
notices of noncoverage (HINNs), alleged
Emergency Medical Treatment and
Labor Act (EMTALA) violations (patient
dumping), and other statutory
responsibilities.
Section 1154 of the Act requires that
QIOs review those services furnished by
physicians; other health care
practitioners; and institutional and noninstitutional providers of health care
services, including health maintenance
organizations and competitive medical
plans. Section 109 of the Medicare
Prescription Drug, Improvement, and
Modernization Act of 2003 (MMA), Pub.
L. 108–173, amended section 1154(a)(1)
of the Act to expand the scope of review
of QIOs to include Medicare Advantage
Organizations and prescription drug
sponsors. Section 109 of the MMA also
VerDate jul<14>2003
19:28 Jul 21, 2005
Jkt 205001
created a new section 1154(a)(17) of the
Act, which requires QIOs to offer to
providers, practitioners, Medicare
Advantage Plans, and prescription drug
sponsors quality improvement
assistance pertaining to prescription
drug therapy. We will not evaluate QIOs
on these provisions in the current Scope
of Work (SOW) because these provisions
of sections 1154(a)(1) and (a)(17) of the
Act were not included in the contract.
Section 1153(h)(2) of the Act requires
the Secretary to publish in the Federal
Register the general criteria and
standards that would be used to
evaluate the efficient and effective
performance of contract obligations by
QIOs and to provide the opportunity for
public comment. The QIO contracts for
the 7th SOW were awarded for 3 years
with starting dates staggered into three
approximately equal groups (rounds)
starting August 2002, November 2002,
and February 2003, respectively.
II. Provisions of the Notice With
Comment
On July 23, 2004, we published a
notice with comment in the Federal
Register titled ‘‘Medicare Program;
Evaluation Criteria and Standards for
Quality Improvement Organizations.’’
The comment period for this notice
closed on August 23, 2004. The
evaluation criteria published in the
notice are currently being used to
evaluate QIO performance on the 7th
SOW. The evaluation criteria is listed
here for the reader’s convenience. No
modifications were made to the
evaluation criteria based on comments
provided in response to the notice.
A. Measuring QIO Performance
Under the 7th Round contracts, QIOs
are responsible for completing tasks in
the following four areas, with additional
subtasks contained in the first three
areas:
Task 1—Improving Beneficiary Safety
and Health Through Clinical Quality
Improvement
a. Nursing Home
b. Home Health
c. Hospital
d. Physician Office
e. Underserved and Rural Beneficiaries
f. Medicare+Choice Organizations
(M+COs), now called Medicare
Advantage Organizations (MAs)
Task 2—Improving Beneficiary Safety
and Health Through Information and
Communications
a. Promoting the Use of Performance
Data
b. Transitioning to Hospital-Generated
Data
PO 00000
Frm 00034
Fmt 4703
Sfmt 4703
c. Other Mandated Communications
Activities
Task 3—Improving Beneficiary Safety
and Health Through Medicare
Beneficiary Protection Activities
a. Beneficiary Complaint Response
Program
b. Hospital Payment Monitoring Review
Program
c. All Other Beneficiary Protection
Activities
Task 4—Improving Beneficiary Safety
and Health Through Developmental
Activities (Special Studies defined as
work that we direct a QIO to perform or
work that a QIO elects to perform with
our approval that is not currently
defined in the Tasks, but falls within the
scope of the contract and section 1154
of the Act).
Under this contract, to merit having
its contract renewed non-competitively,
the QIO must meet the performance
criteria (including a score of 1.0 or
greater for Tasks 1a through 1e and 2b)
on 10 of 12 subtasks (9 of 11 for States
with no MA plans) of Tasks 1 through
3 of the 7th SOW. To renew the QIO’s
contract non-competitively for both of
the subtasks that do not meet the
criteria, the QIO must have: (1)
Achieved a score of 0.6 or better on all
quantitative subtasks, and (2) for the
remaining subtasks only, in the
judgment of the Project Officer, the QIO
expended a reasonable effort to address
these subtasks, and developed and
implemented an appropriate initial
work plan. The work plan must have
been assessed by the Project Officer
during the contract period to determine
if it was achieving results likely to lead
to success in meeting contractual
performance expectations and had made
appropriate adjustments to its work plan
based on these results.
To be considered successful (that is,
meeting the criteria outlined in the J–7
found at https://www.cms.hhs.gov/qio/
2.asp), though not meriting a noncompetitive renewal, the QIO must meet
the performance criteria (including a
score of 1.0 or greater for Tasks 1a
through 1e and 2b) on 9 of 12 subtasks
(8 of 11 for States with no MA plans) of
Tasks 1 through 3 of the 7th Round
Contract. For the subtasks that do not
meet the criteria, the QIO must—
• Achieve a score of 0.6 or better on
all quantitative subtasks;
• For the remaining subtasks only, in
the judgment of the Project Officer, the
QIO has expended a reasonable effort to
address these subtasks, developed and
implemented an appropriate initial
work plan that was assessed by the
Project Officer during the contract
period to determine if it was achieving
E:\FR\FM\22JYN1.SGM
22JYN1
Federal Register / Vol. 70, No. 140 / Friday, July 22, 2005 / Notices
results likely to lead to success in
meeting contractual performance
expectations, and had made appropriate
adjustments to its work plan based on
these results; and
• Failed to meet the criteria in no
more than two subtasks of any one task.
For Task 4, except as provided in
Task 3b that is evaluated by the Task
Leader, all special studies approved
under this task will be evaluated
individually, based on study-specific
evaluation criteria. The QIO’s success or
failure on a special study will not be
factored into the evaluation of the QIO’s
work under Tasks 1 through 3.
However, meeting the minimum
performance standards does not
guarantee a noncompetitive renewal of
the QIO’s contract. For example, an
organization within a particular State
meeting the definition of a QIO may
express interest in competing for a
contract currently held by a QIO from
outside that State, according to section
1153(i) of the Act. In this case, we will
compete the contract despite acceptable
performance by the current QIO. We
will make a final decision on renewal/
non-renewal by the end of the 30th
month of the 7th Round contract. We
will issue a ‘‘Notice of Intent to Nonrenew the QIO Contract’’ letter to all
QIOs that do not meet the minimum
performance standards no later than the
end of the 33rd month of the contract.
The QIO will be considered to have met
minimum performance standards if the
QIO had demonstrated acceptable
performance in each Task area as
specified in section III of this notice,
Standards for Minimum Performance.
If the QIO has not met the criteria to
merit a noncompetitive renewal, it will
be notified of our intention not to renew
its contract and will be informed of its
right to request an opportunity to
provide information about its
performance under the contract to a
CMS-wide panel. The panel includes
representatives from each of the four
QIO Regional Offices and the Central
Office. The QIO’s Project Officer will
not be eligible to represent the Regional
Office on the panel when it reviews the
work of his or her QIO. However, the
Project Officer will be available to
answer any questions. Also, the QIO
will be given the opportunity to provide
additional information. The panel will
have the right to create its own
procedures, but must apply them
consistently to all QIOs. At a minimum,
the panel will use the criteria listed
below for all Tasks:
• The degree of collaboration the QIO
exhibited with the Quality Improvement
Organization Support Centers (QIOSCs)
and other QIOs, both by sharing the
VerDate jul<14>2003
19:28 Jul 21, 2005
Jkt 205001
lessons and tools it developed and by
adopting practices and tools developed
by other QIOs.
• Whether the QIO was a new
contractor in the 7th SOW.
• Whether specific identifiable
circumstances uniquely interfered with
the QIO’s efforts.
• Evidence suggesting that the QIO
has done exceptional work in one or
more of the other Task areas.
• Any other issues that the panel may
deem relevant.
Upon completion of its review, the
panel will recommend a final
disposition of the QIO’s contract
renewal to the Director of CMS’ Office
of Clinical Standards and Quality
(OCSQ).
B. Standards for Minimum Performance
General Criteria
We will evaluate the QIO’s
performance on each sub-task by some
combination of the following elements:
• Statewide improvement on the
quality measure(s).
• Improvement on the quality of care
measure(s) among a group of identified
participants as defined within each
subtask.
• Satisfaction among providers and
practitioners regarding their interaction
with the QIO.
Satisfaction will be assessed using a
survey, the purpose of which will be to:
• Measure satisfaction as one
component of the QIO’s evaluation.
• Identify opportunities where the
QIO can improve satisfaction.
Task 1 (including subtasks a through
e) and subtask 2b will be evaluated
quantitatively. The QIO’s success will
be measured by assessing its relative
improvement on each evaluation
criterion. The term ‘‘improvement’’ as
used in the 7th Round Contract will be
defined mathematically to mean the
relative reduction in the failure rate.
The expected minimum improvement
level, as determined by our management
and defined in the J–7 at
http:www.cms.hhs.gov/qio/2.asp, will
serve as the reference point for each
calculated relative improvement.
In a number of the Task 1 subtasks,
statewide improvement will be averaged
with the improvement among a set of
identified participant providers. In these
cases, we have set a target percentage of
identified participant providers. The
relative weights of the statewide
improvement and of identified
participants’ improvement will combine
to equal 80 percent of the subtask’s
weight, and will be a function of the
percentage of the target percentage (up
to 150 percent) that the QIO identifies
PO 00000
Frm 00035
Fmt 4703
Sfmt 4703
42333
as participants. Tasks 1f, 2a, 2c and all
of Task 3 will be evaluated by the
Project Officer using qualitative
measures based on information
provided in reports developed from data
provided by the QIOs on the QIO’s
status to date.
C. Task Specific Standards
1. Task 1—Improving Beneficiary Safety
and Health Through Clinical Quality
Improvement
a. Task 1a—Nursing Home Quality
Improvement
The QIO will be held accountable for
improvement in the quality of care
measure rates for all nursing homes in
the State and for identified participant
nursing homes. QIOs will be evaluated
based on the following components:
Statewide improvement on the set of
three to five publicly reported quality of
care measures that the QIO has selected
in consultation with stakeholders,
improvement in the selected nursing
home publicly reported quality of care
measures for identified participants, and
nursing home satisfaction based on a
survey of identified participating
nursing homes. To view the weighting
criteria for each component, go to
https://www.cms.hhs.gov/qio/2.asp for a
copy of the J–7.
b. Task 1b—Home Health Quality
Improvement
The QIO will be held accountable for
improvement in the Outcome Based
Quality Improvement (OBQI) quality of
care measure rates for a set of home
health agencies that are identified
participants. The QIOs will be evaluated
based on the following components: The
extent to which the number of
participating home health agencies,
with significant improvement in a
targeted outcome, equals or exceeds 30
percent of the total number of home
health agencies in the State, and the
identified participant satisfaction that
will be measured by a survey of
identified participant home health
agencies using a composite measure of
satisfaction that reflects the type of
activities that QIOs are expected to have
undertaken with these providers.
c. Task 1c—Hospital Quality
Improvement
QIOs will be evaluated on the
following criteria: Statewide
improvement on the quality of care
measures listed in the 7th Round
Contract, and hospital satisfaction based
on feedback from the hospitals in the
State. To view the specific criteria, go to
https://www.cms.hhs.gov/qio/2.asp for a
copy of the J–7.
E:\FR\FM\22JYN1.SGM
22JYN1
42334
Federal Register / Vol. 70, No. 140 / Friday, July 22, 2005 / Notices
d. Task 1d—Physician Office Quality
Improvement
QIOs will be evaluated based on the
following general criteria: statewide
improvement on quality of care
measures, improvement on diabetes and
cancer screening quality of care
measures for identified participant
physicians, and physician satisfaction
based on feedback from physician
designees in the State who participated
with the QIO. To view the specific
criteria for this task, go to https://
www.cms.hhs.gov/qio/2.asp for a copy
of the J–7.
2. Task 2—Improving Beneficiary Safety
and Health Through Information and
Communications
a. Task 2a—Promoting the Use of
Performance Data
QIO success will be assessed on the
timely completion and submission of a
project work plan, timely completion
and submission of all required reports
and deliverables, and the extent to
which the QIO uses information we
have provided as well as any other
feedback the QIO receives to refine its
project activities to achieve the desired
outcome.
e. Task 1e—Underserved and Rural
Beneficiaries Quality Improvement
The QIO’s work on this task will be
primarily evaluated on the success of
the QIO’s efforts to reduce disparity
between the targeted underserved group
and their geographically relevant nonunderserved reference group from
baseline to re-measurement. To be
judged to have performed minimally
successful on this task, the QIO must
demonstrate disparity reduction. QIOs
will also be evaluated on three factors
that collectively demonstrate knowledge
generated by the QIO about the
underserved target group, the
interventions planned upon the basis of
that knowledge, the use of literature on
effective interventions, and by
demonstrating the effectiveness of their
interventions through analyses
comparing the intervention group and a
contrast group. To view the specific
criteria for this task, go to https://
www.cms.hhs.gov/qio/2.asp for a copy
of the J–7.
f. Task 1f—Medicare + Choice
Organizations (M+COs) (Now Called
Medicare Advantage Organizations
(MAs)) Quality Improvement
QIOs will be expected to have
demonstrated appropriate activity to
include MAs in Tasks 1a to 1e as
determined by the Project Officer. We
will survey MAs that have worked with
the QIO using a composite measure of
satisfaction that reflects the types of
activities that QIOs are expected to have
undertaken with these organizations.
We will further use the results of the
Medicare+Choice Quality Review
Organizations (M+CQRO) or
accreditation organization evaluation of
the Quality Assessment and
Performance Improvement (QAPI)
projects to determine if expected
improvement was demonstrated.
VerDate jul<14>2003
19:28 Jul 21, 2005
Jkt 205001
b. Task 2b—Transitioning to HospitalGenerated Data
The evaluation for this task will be
based on the following elements:
• We will determine the
completeness of the assessment survey
information for each hospital.
• We will review hospital data
submitted to the national repository via
QualityNet Exchange to determine the
proportion of hospitals within the State
that have implemented a data
abstraction system to abstract quality of
care measures.
• We will review hospital satisfaction
with the QIO data abstraction support.
To view specific criteria for this task, go
to https://www.cms.hhs.gov/qio/2.asp for
a copy of the J–7.
c. Task 2c—Other Mandated
Communication Activities
QIO success on this task will be
assessed on the following elements: The
establishment and use of a Consumer
Advisory Council to advise and provide
guidance regarding consumer related
activities, the QIO’s success at
broadening consumer representation on
the QIO Board of Directors, the
successful operation of a beneficiary
helpline, and the publication and
distribution of an annual report.
3. Task 3—Improving Beneficiary Safety
and Health Through Medicare
Beneficiary Protection Activities
a. Task 3a—Beneficiary Complaint
Response Program
QIO success will be assessed by the
timeliness of completed reviews, quality
improvement activities as the result of
beneficiary complaints, reliability of the
review of cases as determined by QIO
assessment of the review
determinations, and beneficiary
satisfaction with the complaint process.
b. Task 3b—Hospital Payment
Monitoring Review Program
The QIO must complete reviews
within the prescribed timeframes. The
PO 00000
Frm 00036
Fmt 4703
Sfmt 4703
QIO must also meet one of the following
criteria: with respect to the absolute
payment error rate, the follow-up
payment error rate must be no greater
than 1.5 standard errors above the
baseline error rate, or the QIO must have
made acceptable progress in improving
provider performance in relation to all
projects approved or directed by us.
c. Task 3c—Other Beneficiary Protection
Activities
The QIO will be assessed on the
timeliness of reviews for HINN/
NODMAR, EMTALA review, other case
review activities and post review
activities.
III. Analysis of and Responses to Public
Comments and Provisions of the Final
Notice
We received several public comments
on the 2004 Federal Register notice
with comment period.
Comment: One commenter expressed
concern over the hospital satisfaction
survey in Task 1c. The commenter
noted that some hospitals have changed
to acute care hospitals late in the SOW.
The commenter believes this does not
provide the QIO ample opportunity to
work with the hospital before the
hospital completes the satisfaction
survey. The commenter recommended
that we establish a cut-off date for new
entries as acute care hospitals
participating in the satisfaction survey.
Response: While we understand the
concern that hospitals with only recent
experience in acute care could have an
impact on the hospital satisfaction
survey, we do not believe that it would
be a significant impact for the 7th SOW.
The Task 1c satisfaction scores from the
first two rounds appear to support our
position. All QIOs in the first two
rounds received scores that met or
exceeded the 80 percent passing
threshold. The suggestion to include a
cut-off date is a reasonable one that we
can consider for subsequent Scopes of
Work. We intend to evaluate all rounds
for the current SOW identically.
Comment: One commenter expressed
concern about the project plan
requirements in Task 2a. Specifically,
the commenter stated that the task only
required a project plan for the Nursing
Home Quality Initiative. The commenter
requested more specific language in the
evaluation criteria to address this issue.
Response: For the 7th SOW, we are
requiring only one formal project plan
for the Nursing Home Quality Initiative.
A deliverable has not been added for
subsequent plans. QIOs will not be held
accountable for failing to deliver project
plans that are not required deliverables
for the task.
E:\FR\FM\22JYN1.SGM
22JYN1
Federal Register / Vol. 70, No. 140 / Friday, July 22, 2005 / Notices
Comment: One commenter stated that
there are no historical data to
demonstrate that nursing homes’
thresholds and home health thresholds
are achievable or realistic.
Response: We believe that the
thresholds are achievable for most QIOs.
The results of the 1st Round 28-month
evaluations show that the majority of
the QIOs (87 percent) achieved or
exceeded the target performance.
Therefore, there is no indication that
these thresholds should be changed.
Comment: One commenter stated that
the tasks to be evaluated subjectively
would be less ambiguous if the
components of the evaluation were
known before the start of the SOW.
Response: We agree with this
comment. However, QIOs were
provided a copy of the J–7 before the
start of the SOW. The tool used to do
the actual evaluation was based on the
materials provided in the J–7, and did
not include any criteria or standards not
in the SOW. We will produce the tool
for the 8th SOW early in the contract
period. It will be distributed to QIOs as
soon as it is available.
Comment: Three commenters
questioned how statistical significance
could be calculated for home health
agencies with a small number of
episodes of care.
Response: We use the Fisher’s exact
test to calculate statistical significance
for agency outcomes with 10 to 30
episodes of care. This test does not
require a large sample to estimate
statistical significance. More
information on this test can be found in
Categorical Data Analysis by Alan
Agresti. Additionally, we tested the
impact of small HHAs by recalculating
evaluation results. Excluding all HHAs
with fewer than 30 episodes of care did
not substantively improve the overall
evaluation results. Based on this
information, we decided not to modify
the 1b evaluation criteria.
Comment: One commenter questioned
how we determined the home health
task denominator for the 30 percent.
Response: The home health
denominator is made up of two
components. It includes identified
participants and non-identified
participants. Identified participants are
defined as all home health agencies that
submitted an OBQI plan of action (POA)
and have at least a one 3-bar OBQI
report for any reporting period ending at
least 12 months after the POA
submission date. A 3-bar report allows
the HHA to compare current outcome
rates to prior year outcome rates and
national outcome rates. Non-identified
participants are defined as having no
OBQI plan of action submitted, but with
VerDate jul<14>2003
19:28 Jul 21, 2005
Jkt 205001
a 3-bar OBQI report for the reporting
period ending in the 24th month of the
contract. This definition recognizes the
dynamic nature of the home health
industry, and counts only agencies with
sufficient caseload during the 24
months included in the 3-bar report. We
believe that this definition provides
QIOs with the best opportunity to
successfully pass the evaluation, while
including all agencies operating with a
sufficient caseload during a large part of
the SOW.
Comment: One commenter stated that
many of the Task 1c hospital indicators
will have a small number in the
denominator. The commenter stated
that by collecting the same number of
cases for all States, the precision and
confidence interval is much smaller for
a large State, thereby making the
evaluation of the QIO less accurate.
Response: The assumption on the part
of the commenter is not completely
accurate. The evaluation score equally
weights the four conditions for hospital
public reporting (see https://
www.cms.hhs.gov/quality/hospital for
list of conditions) to provide a more
robust estimate of quality improvement.
Three of the four conditions have large
enough samples so that sample size (not
population size) is the primary
determinant driving the precision of the
estimates. Acute Myocardial Infarction
measures, one of the four conditions,
with systematically small samples are
weighted accordingly to minimize the
impact of any unreliable estimates on
the overall evaluation. AMI is the only
one of the four conditions with
systematically small samples. It is
weighted accordingly to minimize the
impact of any unreliable estimates on
the overall evaluation.
Comment: One commenter stated that
the Task 2b evaluation should not be
considered under the quantitative
evaluation criteria. The commenter
stated that the largest weighted criterion
for this task is related to the Reporting
Hospital Quality Data for Annual
Payment Update (RHQDAPU), which
does not have a quantitative
measurement.
Response: The RHQDAPU criterion
for this subtask is dichotomous in
nature and requires that QIOs contact all
hospitals in their State and assist them
in their data submission into the
Standard Data Processing System
Clinical Warehouse. QIOs must also
document their communication and
assistance with all hospitals,
participating and non-participating.
Although this task does involve some
activities that may be evaluated in a
qualitative manner, the majority of the
activities are quantitative in nature.
PO 00000
Frm 00037
Fmt 4703
Sfmt 4703
42335
Therefore, we have chosen to evaluate
this task quantitatively.
Comment: One commenter expressed
concern over the lateness of data for
Task 1d. The commenter believes that
this has made it nearly impossible to
assess the effectiveness of the QIO
interventions, or to identify other areas
for intervention.
Response: We recognize that time lags
can hinder the QIO’s technical
assistance to providers in the outpatient
setting. We have set the baseline period
to allow QIOs to work with providers
during the transition period between
SOWs. Much of this work is reflected in
the next SOW’s evaluation results. The
relative stability of QIOs in their States
lessens the impact of the time lag.
Comment: One commenter suggested
that we change the evaluation criteria in
the J–7 for Task 1e to make them the
same as the evaluation criteria that were
originally developed for their QIO’s
improvement project.
Response: We assume the commenter
is referring to the use of sub-county
targeting in the evaluation of this Task.
We have already modified the
evaluation on this Task to allow subcounty targeting. This modification to
the evaluation was approved by the
Project Officers in the beginning of the
SOW. We do not anticipate any further
changes at this point.
Comment: One commenter suggested
that Task 3 activities be elevated to a
higher position in the SOW. This
commenter believes the current Task 3
should be Task 1 or Task 2 to increase
its importance in the contract.
Response: We agree that all of the
Tasks performed by the QIOs are
important to foster quality improvement
in the health care delivered to Medicare
beneficiaries. The evaluation criteria
reflect this belief. Task 3 comprises 3
out of 12 subtasks evaluated by us. QIOs
must successfully perform Task 3 work
in order to be granted non-competitive
contract renewal. We believe that the
stringent evaluation criteria in place for
this task reflect the importance of the
work.
Comment: One commenter asked
about the provider satisfaction survey
and how we plan to use the survey if the
QIO does not have a sufficient sample
size.
Response: Identifying opportunities
for improvement is part of a quality
improvement feedback cycle. We
believe that the results of the
satisfaction survey are useful to QIOs in
identifying quality improvement
opportunities. CMS and its statistical
contractor have provided all QIOs with
detailed information about their
satisfaction survey results. The
E:\FR\FM\22JYN1.SGM
22JYN1
42336
Federal Register / Vol. 70, No. 140 / Friday, July 22, 2005 / Notices
statistical contractor will also write a
national analysis of the survey results to
identify opportunities for QIO program
improvement as a whole. In the few
instances with insufficient sample size,
we use the actual satisfaction rate to
evaluate QIO performance. However, we
grant QIOs a passing evaluation score
when the overall evaluation status (that
is, pass vs. fail) is sensitive to this
potentially unreliable rate. Usually this
rate does not affect a QIO’s overall
evaluation status on a particular
subtask, since its relative weight is
small in a subtask’s evaluation.
Comment: One commenter stated that,
with the development of the Excel
spreadsheet to evaluate the qualitative
tasks, these tasks are no longer
qualitative. They are now being
evaluated in a quantitative way.
Response: The Excel tool allows
Project Officers to subjectively evaluate
QIO performance in the qualitative
tasks. It was developed in response to
concerns from QIOs about inter-region
variation in the 6th SOW. It uses the
same evaluation criterion provided in
the J–7, and is not intended to make the
evaluation quantitative in nature.
Rather, it gives some consistency to the
subjective review by the Project
Officers. We agree that this tool should
be provided to QIOs as early as possible
in the contract cycle. We will strive to
provide this tool to the QIOs as early as
possible for the 8th SOW.
Comment: One commenter stated that
a great deal of effort was put into the
National Voluntary Hospital Reporting
Initiative (NVHRI), but this effort was
not included in the evaluation criteria.
Response: We appreciate the fact that
the NVHRI did require some additional
effort on the part of the hospitals.
However, participation could not be
included in the evaluation criteria
because this was a voluntary program
on the part of hospitals. The voluntary
nature of the program requires a
different approach by the QIO than is
required by the other subtasks and
deliverables of the contract.
Comment: One commenter stated that
for those States with 100 percent
participation in hospital public
reporting, the Hospital Generated Data
(HGD) Survey is redundant. The
commenter stated that the same
information may be obtained through
both sources.
Response: We have been careful to
avoid redundant activities for both
providers and QIOs. The HGD Survey
does not determine if a hospital is a
reporting hospital. Instead, it assesses
the hospital’s ability to collect data.
Therefore both the survey and the actual
VerDate jul<14>2003
19:28 Jul 21, 2005
Jkt 205001
hospital reporting are necessary and
provide different information to us.
Comment: One commenter questioned
the evaluation criteria for Task 3b. In
the J–7, the term ‘‘reliability’’ is used.
The guidance document states that the
QIO will be evaluated based on both
‘‘reliability’’ and ‘‘validity of review.’’
This commenter also requested
clarification as to why Tasks 3a and 3b
require reliability while Task 3c does
not require validity for evaluation.
Response: The reliability of the
review is the primary criterion for
evaluating this component of the task.
We will ensure consistency in
documents released for the 8th SOW.
The evaluation criteria were chosen for
each subtask in Task 3 based on the
appropriateness for the task.
Comment: One commenter expressed
concern over using Medicare physician
billing as the method to measure the
rate of statewide and identified
participants’ improvement in quality
care measures for Task 1d.
Response: We are investigating this
method of measuring improvement for
the Round 1 evaluations, and have so far
found nothing large-scale or systematic
that would alter evaluation results for
Task 1d. We believe that the evaluation
measures are relatively stable and
reliable estimates, and that billing issues
as a whole do not contribute significant
bias to these estimates. We understand
the limitations of using billing
information to estimate quality
improvement, and are working to
minimize its impact by identifying these
problems and reporting questionable
billing issues to the appropriate parties.
We are adopting the provisions of the
notice with comment as final.
IV. Executive Order 12866 Statement
In accordance with the provisions of
Executive Order 12866, this notice with
comment period was not reviewed by
the Office of Management and Budget.
Authority: Section 1153 of the Social
Security Act (42 U.S.C. 1320c–2).
(Catalog of Federal Domestic Assistance
Program No. 93.774, Medicare—
Supplementary Medical Insurance Program)
Dated: March 14, 2005.
Mark B. McClellan,
Administrator, Centers for Medicare &
Medicaid Services.
[FR Doc. 05–14505 Filed 7–21–05; 8:45 am]
BILLING CODE 4120–01–P
PO 00000
Frm 00038
Fmt 4703
Sfmt 4703
DEPARTMENT OF HEALTH AND
HUMAN SERVICES
Administration for Children and
Families
Administration on Children, Youth and
Families; Family and Youth Services
Bureau; Notice of the Availability of
Financial Assistance and Request for
Applications To Establish and Operate
the National Domestic Violence Hotline
Announcement Type: Grant.
Funding Opportunity Number: HHS–
2005–ACF–ACYF–EV–0039.
CFDA Number: 93.592.
Due Date for Applications: August 22,
2005.
Executive Summary: The
Administration for Children and
Families (ACF), Administration on
Children, Youth and Families (ACYF)
announces the availability of funds in
fiscal year 2005 for the award of one
grant on a competitive basis to operate
a national, toll-free telephone hotline to
provide information and assistance to
victims of domestic violence.
I. Funding Opportunity Description
Authorizing Statutes and Regulations:
The Family Violence Prevention and
Services Act (the Act) was originally
enacted in sections 301–316 of Title III
of the ‘‘Child Abuse Amendments of
1984’’ (Pub. L. 98–457, 10/9/84). The
Act was most recently amended by the
‘‘Keeping Children and Families Safe
Act of 2003’’ (Pub. L. 108–36).
Supplementary Information: In
accordance with amendments to the Act
enacted by Pub. L. 108–36, ACF will
award grants to one or more private,
non-profit entities to assist in the
establishment and operation of a highly
secure Internet website to provide
information and assistance to victims of
domestic violence. A separate
announcement regarding these awards
will be issued at a future date.
Program and Focus Area: The
purpose of the National Domestic
Violence Hotline (Hotline) is to provide
information and referral services,
counseling, and assistance to victims of
domestic violence, their children and
other family members, and others
affected by such violence; and enable
them to find safety and protection in
crisis situations. The successful
applicant will be required to provide
telephonic assistance on a 24 hours-perday, seven days-a-week basis
throughout the continental United
States, Alaska, Hawaii, the
Commonwealth of Puerto Rico, and the
U.S. Virgin Islands.
E:\FR\FM\22JYN1.SGM
22JYN1
]]>Agencies
[Federal Register Volume 70, Number 140 (Friday, July 22, 2005)]
[Notices]
[Pages 42331-42336]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 05-14505]
-----------------------------------------------------------------------
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Centers for Medicare & Medicaid Services
[CMS-3142-FN]
Medicare Program; Evaluation Criteria and Standards for Quality
Improvement Program Contracts
AGENCY: Centers for Medicare & Medicaid Services (CMS), HHS.
ACTION: Final notice.
-----------------------------------------------------------------------
SUMMARY: This final notice describes the evaluation criteria we will
use to evaluate the Quality Improvement Organizations (QIOs) under
their contracts with us, for efficiency and effectiveness in accordance
with the Social Security Act. These evaluation criteria are based on
the tasks and related subtasks set forth in the QIO's Scope of Work
(SOW). The current 7th SOW includes Tasks 1 through 4, with subtasks
included under all tasks,
[[Page 42332]]
excluding Task 4. QIOs were awarded contracts for the 7th SOW, or 7th
Round, for 3 years, with staggered starting dates beginning August
2002, November 2002, and February 2003. This final notice also responds
to the public comments received regarding the evaluation criteria
published in July 2004.
DATES: Effective August 22, 2005.
FOR FURTHER INFORMATION CONTACT: Maria Hammel, (410) 786-1775.
SUPPLEMENTARY INFORMATION:
I. Background
The Peer Review Improvement Act of 1982 (Title I, Subtitle C of
Public Law 97-248) amended Part B of Title XI of the Social Security
Act (the Act) to establish the Peer Review Organization (PRO) program.
The PRO program (now called the Quality Improvement Organization (QIO)
program) was established to redirect, simplify, and enhance the cost-
effectiveness and efficiency of the medical peer review process.
Sections 1152 and 1153 of the Act define the types of organizations
eligible to become QIOs, and establish certain limitations and
priorities regarding QIO contracting.
The Secretary enters into contracts with QIOs to perform three
broad functions:
Improve quality of care for beneficiaries by ensuring that
beneficiary care meets professionally recognized standards of health
care;
Protect the integrity of the Medicare Trust Fund by
ensuring that Medicare only pays for services and items that are
reasonable and medically necessary and that are provided in the most
economical setting;
Protect beneficiaries by expeditiously addressing
individual cases such as beneficiary quality of care complaints,
contested hospital issued notices of noncoverage (HINNs), alleged
Emergency Medical Treatment and Labor Act (EMTALA) violations (patient
dumping), and other statutory responsibilities.
Section 1154 of the Act requires that QIOs review those services
furnished by physicians; other health care practitioners; and
institutional and non-institutional providers of health care services,
including health maintenance organizations and competitive medical
plans. Section 109 of the Medicare Prescription Drug, Improvement, and
Modernization Act of 2003 (MMA), Pub. L. 108-173, amended section
1154(a)(1) of the Act to expand the scope of review of QIOs to include
Medicare Advantage Organizations and prescription drug sponsors.
Section 109 of the MMA also created a new section 1154(a)(17) of the
Act, which requires QIOs to offer to providers, practitioners, Medicare
Advantage Plans, and prescription drug sponsors quality improvement
assistance pertaining to prescription drug therapy. We will not
evaluate QIOs on these provisions in the current Scope of Work (SOW)
because these provisions of sections 1154(a)(1) and (a)(17) of the Act
were not included in the contract.
Section 1153(h)(2) of the Act requires the Secretary to publish in
the Federal Register the general criteria and standards that would be
used to evaluate the efficient and effective performance of contract
obligations by QIOs and to provide the opportunity for public comment.
The QIO contracts for the 7th SOW were awarded for 3 years with
starting dates staggered into three approximately equal groups (rounds)
starting August 2002, November 2002, and February 2003, respectively.
II. Provisions of the Notice With Comment
On July 23, 2004, we published a notice with comment in the Federal
Register titled ``Medicare Program; Evaluation Criteria and Standards
for Quality Improvement Organizations.'' The comment period for this
notice closed on August 23, 2004. The evaluation criteria published in
the notice are currently being used to evaluate QIO performance on the
7th SOW. The evaluation criteria is listed here for the reader's
convenience. No modifications were made to the evaluation criteria
based on comments provided in response to the notice.
A. Measuring QIO Performance
Under the 7th Round contracts, QIOs are responsible for completing
tasks in the following four areas, with additional subtasks contained
in the first three areas:
Task 1--Improving Beneficiary Safety and Health Through Clinical
Quality Improvement
a. Nursing Home
b. Home Health
c. Hospital
d. Physician Office
e. Underserved and Rural Beneficiaries
f. Medicare+Choice Organizations (M+COs), now called Medicare Advantage
Organizations (MAs)
Task 2--Improving Beneficiary Safety and Health Through Information and
Communications
a. Promoting the Use of Performance Data
b. Transitioning to Hospital-Generated Data
c. Other Mandated Communications Activities
Task 3--Improving Beneficiary Safety and Health Through Medicare
Beneficiary Protection Activities
a. Beneficiary Complaint Response Program
b. Hospital Payment Monitoring Review Program
c. All Other Beneficiary Protection Activities
Task 4--Improving Beneficiary Safety and Health Through
Developmental Activities (Special Studies defined as work that we
direct a QIO to perform or work that a QIO elects to perform with our
approval that is not currently defined in the Tasks, but falls within
the scope of the contract and section 1154 of the Act).
Under this contract, to merit having its contract renewed non-
competitively, the QIO must meet the performance criteria (including a
score of 1.0 or greater for Tasks 1a through 1e and 2b) on 10 of 12
subtasks (9 of 11 for States with no MA plans) of Tasks 1 through 3 of
the 7th SOW. To renew the QIO's contract non-competitively for both of
the subtasks that do not meet the criteria, the QIO must have: (1)
Achieved a score of 0.6 or better on all quantitative subtasks, and (2)
for the remaining subtasks only, in the judgment of the Project
Officer, the QIO expended a reasonable effort to address these
subtasks, and developed and implemented an appropriate initial work
plan. The work plan must have been assessed by the Project Officer
during the contract period to determine if it was achieving results
likely to lead to success in meeting contractual performance
expectations and had made appropriate adjustments to its work plan
based on these results.
To be considered successful (that is, meeting the criteria outlined
in the J-7 found at https://www.cms.hhs.gov/qio/2.asp), though not
meriting a non-competitive renewal, the QIO must meet the performance
criteria (including a score of 1.0 or greater for Tasks 1a through 1e
and 2b) on 9 of 12 subtasks (8 of 11 for States with no MA plans) of
Tasks 1 through 3 of the 7th Round Contract. For the subtasks that do
not meet the criteria, the QIO must--
Achieve a score of 0.6 or better on all quantitative
subtasks;
For the remaining subtasks only, in the judgment of the
Project Officer, the QIO has expended a reasonable effort to address
these subtasks, developed and implemented an appropriate initial work
plan that was assessed by the Project Officer during the contract
period to determine if it was achieving
[[Page 42333]]
results likely to lead to success in meeting contractual performance
expectations, and had made appropriate adjustments to its work plan
based on these results; and
Failed to meet the criteria in no more than two subtasks
of any one task.
For Task 4, except as provided in Task 3b that is evaluated by the
Task Leader, all special studies approved under this task will be
evaluated individually, based on study-specific evaluation criteria.
The QIO's success or failure on a special study will not be factored
into the evaluation of the QIO's work under Tasks 1 through 3.
However, meeting the minimum performance standards does not
guarantee a noncompetitive renewal of the QIO's contract. For example,
an organization within a particular State meeting the definition of a
QIO may express interest in competing for a contract currently held by
a QIO from outside that State, according to section 1153(i) of the Act.
In this case, we will compete the contract despite acceptable
performance by the current QIO. We will make a final decision on
renewal/non-renewal by the end of the 30th month of the 7th Round
contract. We will issue a ``Notice of Intent to Non-renew the QIO
Contract'' letter to all QIOs that do not meet the minimum performance
standards no later than the end of the 33rd month of the contract. The
QIO will be considered to have met minimum performance standards if the
QIO had demonstrated acceptable performance in each Task area as
specified in section III of this notice, Standards for Minimum
Performance.
If the QIO has not met the criteria to merit a noncompetitive
renewal, it will be notified of our intention not to renew its contract
and will be informed of its right to request an opportunity to provide
information about its performance under the contract to a CMS-wide
panel. The panel includes representatives from each of the four QIO
Regional Offices and the Central Office. The QIO's Project Officer will
not be eligible to represent the Regional Office on the panel when it
reviews the work of his or her QIO. However, the Project Officer will
be available to answer any questions. Also, the QIO will be given the
opportunity to provide additional information. The panel will have the
right to create its own procedures, but must apply them consistently to
all QIOs. At a minimum, the panel will use the criteria listed below
for all Tasks:
The degree of collaboration the QIO exhibited with the
Quality Improvement Organization Support Centers (QIOSCs) and other
QIOs, both by sharing the lessons and tools it developed and by
adopting practices and tools developed by other QIOs.
Whether the QIO was a new contractor in the 7th SOW.
Whether specific identifiable circumstances uniquely
interfered with the QIO's efforts.
Evidence suggesting that the QIO has done exceptional work
in one or more of the other Task areas.
Any other issues that the panel may deem relevant.
Upon completion of its review, the panel will recommend a final
disposition of the QIO's contract renewal to the Director of CMS'
Office of Clinical Standards and Quality (OCSQ).
B. Standards for Minimum Performance
General Criteria
We will evaluate the QIO's performance on each sub-task by some
combination of the following elements:
Statewide improvement on the quality measure(s).
Improvement on the quality of care measure(s) among a
group of identified participants as defined within each subtask.
Satisfaction among providers and practitioners regarding
their interaction with the QIO.
Satisfaction will be assessed using a survey, the purpose of which
will be to:
Measure satisfaction as one component of the QIO's
evaluation.
Identify opportunities where the QIO can improve
satisfaction.
Task 1 (including subtasks a through e) and subtask 2b will be
evaluated quantitatively. The QIO's success will be measured by
assessing its relative improvement on each evaluation criterion. The
term ``improvement'' as used in the 7th Round Contract will be defined
mathematically to mean the relative reduction in the failure rate. The
expected minimum improvement level, as determined by our management and
defined in the J-7 at http:www.cms.hhs.gov/qio/2.asp, will serve as the
reference point for each calculated relative improvement.
In a number of the Task 1 subtasks, statewide improvement will be
averaged with the improvement among a set of identified participant
providers. In these cases, we have set a target percentage of
identified participant providers. The relative weights of the statewide
improvement and of identified participants' improvement will combine to
equal 80 percent of the subtask's weight, and will be a function of the
percentage of the target percentage (up to 150 percent) that the QIO
identifies as participants. Tasks 1f, 2a, 2c and all of Task 3 will be
evaluated by the Project Officer using qualitative measures based on
information provided in reports developed from data provided by the
QIOs on the QIO's status to date.
C. Task Specific Standards
1. Task 1--Improving Beneficiary Safety and Health Through Clinical
Quality Improvement
a. Task 1a--Nursing Home Quality Improvement
The QIO will be held accountable for improvement in the quality of
care measure rates for all nursing homes in the State and for
identified participant nursing homes. QIOs will be evaluated based on
the following components: Statewide improvement on the set of three to
five publicly reported quality of care measures that the QIO has
selected in consultation with stakeholders, improvement in the selected
nursing home publicly reported quality of care measures for identified
participants, and nursing home satisfaction based on a survey of
identified participating nursing homes. To view the weighting criteria
for each component, go to https://www.cms.hhs.gov/qio/2.asp for a copy
of the J-7.
b. Task 1b--Home Health Quality Improvement
The QIO will be held accountable for improvement in the Outcome
Based Quality Improvement (OBQI) quality of care measure rates for a
set of home health agencies that are identified participants. The QIOs
will be evaluated based on the following components: The extent to
which the number of participating home health agencies, with
significant improvement in a targeted outcome, equals or exceeds 30
percent of the total number of home health agencies in the State, and
the identified participant satisfaction that will be measured by a
survey of identified participant home health agencies using a composite
measure of satisfaction that reflects the type of activities that QIOs
are expected to have undertaken with these providers.
c. Task 1c--Hospital Quality Improvement
QIOs will be evaluated on the following criteria: Statewide
improvement on the quality of care measures listed in the 7th Round
Contract, and hospital satisfaction based on feedback from the
hospitals in the State. To view the specific criteria, go to https://
www.cms.hhs.gov/qio/2.asp for a copy of the J-7.
[[Page 42334]]
d. Task 1d--Physician Office Quality Improvement
QIOs will be evaluated based on the following general criteria:
statewide improvement on quality of care measures, improvement on
diabetes and cancer screening quality of care measures for identified
participant physicians, and physician satisfaction based on feedback
from physician designees in the State who participated with the QIO. To
view the specific criteria for this task, go to https://www.cms.hhs.gov/
qio/2.asp for a copy of the J-7.
e. Task 1e--Underserved and Rural Beneficiaries Quality Improvement
The QIO's work on this task will be primarily evaluated on the
success of the QIO's efforts to reduce disparity between the targeted
underserved group and their geographically relevant non-underserved
reference group from baseline to re-measurement. To be judged to have
performed minimally successful on this task, the QIO must demonstrate
disparity reduction. QIOs will also be evaluated on three factors that
collectively demonstrate knowledge generated by the QIO about the
underserved target group, the interventions planned upon the basis of
that knowledge, the use of literature on effective interventions, and
by demonstrating the effectiveness of their interventions through
analyses comparing the intervention group and a contrast group. To view
the specific criteria for this task, go to https://www.cms.hhs.gov/qio/
2.asp for a copy of the J-7.
f. Task 1f--Medicare + Choice Organizations (M+COs) (Now Called
Medicare Advantage Organizations (MAs)) Quality Improvement
QIOs will be expected to have demonstrated appropriate activity to
include MAs in Tasks 1a to 1e as determined by the Project Officer. We
will survey MAs that have worked with the QIO using a composite measure
of satisfaction that reflects the types of activities that QIOs are
expected to have undertaken with these organizations. We will further
use the results of the Medicare+Choice Quality Review Organizations
(M+CQRO) or accreditation organization evaluation of the Quality
Assessment and Performance Improvement (QAPI) projects to determine if
expected improvement was demonstrated.
2. Task 2--Improving Beneficiary Safety and Health Through Information
and Communications
a. Task 2a--Promoting the Use of Performance Data
QIO success will be assessed on the timely completion and
submission of a project work plan, timely completion and submission of
all required reports and deliverables, and the extent to which the QIO
uses information we have provided as well as any other feedback the QIO
receives to refine its project activities to achieve the desired
outcome.
b. Task 2b--Transitioning to Hospital-Generated Data
The evaluation for this task will be based on the following
elements:
We will determine the completeness of the assessment
survey information for each hospital.
We will review hospital data submitted to the national
repository via QualityNet Exchange to determine the proportion of
hospitals within the State that have implemented a data abstraction
system to abstract quality of care measures.
We will review hospital satisfaction with the QIO data
abstraction support. To view specific criteria for this task, go to
https://www.cms.hhs.gov/qio/2.asp for a copy of the J-7.
c. Task 2c--Other Mandated Communication Activities
QIO success on this task will be assessed on the following
elements: The establishment and use of a Consumer Advisory Council to
advise and provide guidance regarding consumer related activities, the
QIO's success at broadening consumer representation on the QIO Board of
Directors, the successful operation of a beneficiary helpline, and the
publication and distribution of an annual report.
3. Task 3--Improving Beneficiary Safety and Health Through Medicare
Beneficiary Protection Activities
a. Task 3a--Beneficiary Complaint Response Program
QIO success will be assessed by the timeliness of completed
reviews, quality improvement activities as the result of beneficiary
complaints, reliability of the review of cases as determined by QIO
assessment of the review determinations, and beneficiary satisfaction
with the complaint process.
b. Task 3b--Hospital Payment Monitoring Review Program
The QIO must complete reviews within the prescribed timeframes. The
QIO must also meet one of the following criteria: with respect to the
absolute payment error rate, the follow-up payment error rate must be
no greater than 1.5 standard errors above the baseline error rate, or
the QIO must have made acceptable progress in improving provider
performance in relation to all projects approved or directed by us.
c. Task 3c--Other Beneficiary Protection Activities
The QIO will be assessed on the timeliness of reviews for HINN/
NODMAR, EMTALA review, other case review activities and post review
activities.
III. Analysis of and Responses to Public Comments and Provisions of the
Final Notice
We received several public comments on the 2004 Federal Register
notice with comment period.
Comment: One commenter expressed concern over the hospital
satisfaction survey in Task 1c. The commenter noted that some hospitals
have changed to acute care hospitals late in the SOW. The commenter
believes this does not provide the QIO ample opportunity to work with
the hospital before the hospital completes the satisfaction survey. The
commenter recommended that we establish a cut-off date for new entries
as acute care hospitals participating in the satisfaction survey.
Response: While we understand the concern that hospitals with only
recent experience in acute care could have an impact on the hospital
satisfaction survey, we do not believe that it would be a significant
impact for the 7th SOW. The Task 1c satisfaction scores from the first
two rounds appear to support our position. All QIOs in the first two
rounds received scores that met or exceeded the 80 percent passing
threshold. The suggestion to include a cut-off date is a reasonable one
that we can consider for subsequent Scopes of Work. We intend to
evaluate all rounds for the current SOW identically.
Comment: One commenter expressed concern about the project plan
requirements in Task 2a. Specifically, the commenter stated that the
task only required a project plan for the Nursing Home Quality
Initiative. The commenter requested more specific language in the
evaluation criteria to address this issue.
Response: For the 7th SOW, we are requiring only one formal project
plan for the Nursing Home Quality Initiative. A deliverable has not
been added for subsequent plans. QIOs will not be held accountable for
failing to deliver project plans that are not required deliverables for
the task.
[[Page 42335]]
Comment: One commenter stated that there are no historical data to
demonstrate that nursing homes' thresholds and home health thresholds
are achievable or realistic.
Response: We believe that the thresholds are achievable for most
QIOs. The results of the 1st Round 28-month evaluations show that the
majority of the QIOs (87 percent) achieved or exceeded the target
performance. Therefore, there is no indication that these thresholds
should be changed.
Comment: One commenter stated that the tasks to be evaluated
subjectively would be less ambiguous if the components of the
evaluation were known before the start of the SOW.
Response: We agree with this comment. However, QIOs were provided a
copy of the J-7 before the start of the SOW. The tool used to do the
actual evaluation was based on the materials provided in the J-7, and
did not include any criteria or standards not in the SOW. We will
produce the tool for the 8th SOW early in the contract period. It will
be distributed to QIOs as soon as it is available.
Comment: Three commenters questioned how statistical significance
could be calculated for home health agencies with a small number of
episodes of care.
Response: We use the Fisher's exact test to calculate statistical
significance for agency outcomes with 10 to 30 episodes of care. This
test does not require a large sample to estimate statistical
significance. More information on this test can be found in Categorical
Data Analysis by Alan Agresti. Additionally, we tested the impact of
small HHAs by recalculating evaluation results. Excluding all HHAs with
fewer than 30 episodes of care did not substantively improve the
overall evaluation results. Based on this information, we decided not
to modify the 1b evaluation criteria.
Comment: One commenter questioned how we determined the home health
task denominator for the 30 percent.
Response: The home health denominator is made up of two components.
It includes identified participants and non-identified participants.
Identified participants are defined as all home health agencies that
submitted an OBQI plan of action (POA) and have at least a one 3-bar
OBQI report for any reporting period ending at least 12 months after
the POA submission date. A 3-bar report allows the HHA to compare
current outcome rates to prior year outcome rates and national outcome
rates. Non-identified participants are defined as having no OBQI plan
of action submitted, but with a 3-bar OBQI report for the reporting
period ending in the 24th month of the contract. This definition
recognizes the dynamic nature of the home health industry, and counts
only agencies with sufficient caseload during the 24 months included in
the 3-bar report. We believe that this definition provides QIOs with
the best opportunity to successfully pass the evaluation, while
including all agencies operating with a sufficient caseload during a
large part of the SOW.
Comment: One commenter stated that many of the Task 1c hospital
indicators will have a small number in the denominator. The commenter
stated that by collecting the same number of cases for all States, the
precision and confidence interval is much smaller for a large State,
thereby making the evaluation of the QIO less accurate.
Response: The assumption on the part of the commenter is not
completely accurate. The evaluation score equally weights the four
conditions for hospital public reporting (see https://www.cms.hhs.gov/
quality/hospital for list of conditions) to provide a more robust
estimate of quality improvement. Three of the four conditions have
large enough samples so that sample size (not population size) is the
primary determinant driving the precision of the estimates. Acute
Myocardial Infarction measures, one of the four conditions, with
systematically small samples are weighted accordingly to minimize the
impact of any unreliable estimates on the overall evaluation. AMI is
the only one of the four conditions with systematically small samples.
It is weighted accordingly to minimize the impact of any unreliable
estimates on the overall evaluation.
Comment: One commenter stated that the Task 2b evaluation should
not be considered under the quantitative evaluation criteria. The
commenter stated that the largest weighted criterion for this task is
related to the Reporting Hospital Quality Data for Annual Payment
Update (RHQDAPU), which does not have a quantitative measurement.
Response: The RHQDAPU criterion for this subtask is dichotomous in
nature and requires that QIOs contact all hospitals in their State and
assist them in their data submission into the Standard Data Processing
System Clinical Warehouse. QIOs must also document their communication
and assistance with all hospitals, participating and non-participating.
Although this task does involve some activities that may be evaluated
in a qualitative manner, the majority of the activities are
quantitative in nature. Therefore, we have chosen to evaluate this task
quantitatively.
Comment: One commenter expressed concern over the lateness of data
for Task 1d. The commenter believes that this has made it nearly
impossible to assess the effectiveness of the QIO interventions, or to
identify other areas for intervention.
Response: We recognize that time lags can hinder the QIO's
technical assistance to providers in the outpatient setting. We have
set the baseline period to allow QIOs to work with providers during the
transition period between SOWs. Much of this work is reflected in the
next SOW's evaluation results. The relative stability of QIOs in their
States lessens the impact of the time lag.
Comment: One commenter suggested that we change the evaluation
criteria in the J-7 for Task 1e to make them the same as the evaluation
criteria that were originally developed for their QIO's improvement
project.
Response: We assume the commenter is referring to the use of sub-
county targeting in the evaluation of this Task. We have already
modified the evaluation on this Task to allow sub-county targeting.
This modification to the evaluation was approved by the Project
Officers in the beginning of the SOW. We do not anticipate any further
changes at this point.
Comment: One commenter suggested that Task 3 activities be elevated
to a higher position in the SOW. This commenter believes the current
Task 3 should be Task 1 or Task 2 to increase its importance in the
contract.
Response: We agree that all of the Tasks performed by the QIOs are
important to foster quality improvement in the health care delivered to
Medicare beneficiaries. The evaluation criteria reflect this belief.
Task 3 comprises 3 out of 12 subtasks evaluated by us. QIOs must
successfully perform Task 3 work in order to be granted non-competitive
contract renewal. We believe that the stringent evaluation criteria in
place for this task reflect the importance of the work.
Comment: One commenter asked about the provider satisfaction survey
and how we plan to use the survey if the QIO does not have a sufficient
sample size.
Response: Identifying opportunities for improvement is part of a
quality improvement feedback cycle. We believe that the results of the
satisfaction survey are useful to QIOs in identifying quality
improvement opportunities. CMS and its statistical contractor have
provided all QIOs with detailed information about their satisfaction
survey results. The
[[Page 42336]]
statistical contractor will also write a national analysis of the
survey results to identify opportunities for QIO program improvement as
a whole. In the few instances with insufficient sample size, we use the
actual satisfaction rate to evaluate QIO performance. However, we grant
QIOs a passing evaluation score when the overall evaluation status
(that is, pass vs. fail) is sensitive to this potentially unreliable
rate. Usually this rate does not affect a QIO's overall evaluation
status on a particular subtask, since its relative weight is small in a
subtask's evaluation.
Comment: One commenter stated that, with the development of the
Excel spreadsheet to evaluate the qualitative tasks, these tasks are no
longer qualitative. They are now being evaluated in a quantitative way.
Response: The Excel tool allows Project Officers to subjectively
evaluate QIO performance in the qualitative tasks. It was developed in
response to concerns from QIOs about inter-region variation in the 6th
SOW. It uses the same evaluation criterion provided in the J-7, and is
not intended to make the evaluation quantitative in nature. Rather, it
gives some consistency to the subjective review by the Project
Officers. We agree that this tool should be provided to QIOs as early
as possible in the contract cycle. We will strive to provide this tool
to the QIOs as early as possible for the 8th SOW.
Comment: One commenter stated that a great deal of effort was put
into the National Voluntary Hospital Reporting Initiative (NVHRI), but
this effort was not included in the evaluation criteria.
Response: We appreciate the fact that the NVHRI did require some
additional effort on the part of the hospitals. However, participation
could not be included in the evaluation criteria because this was a
voluntary program on the part of hospitals. The voluntary nature of the
program requires a different approach by the QIO than is required by
the other subtasks and deliverables of the contract.
Comment: One commenter stated that for those States with 100
percent participation in hospital public reporting, the Hospital
Generated Data (HGD) Survey is redundant. The commenter stated that the
same information may be obtained through both sources.
Response: We have been careful to avoid redundant activities for
both providers and QIOs. The HGD Survey does not determine if a
hospital is a reporting hospital. Instead, it assesses the hospital's
ability to collect data. Therefore both the survey and the actual
hospital reporting are necessary and provide different information to
us.
Comment: One commenter questioned the evaluation criteria for Task
3b. In the J-7, the term ``reliability'' is used. The guidance document
states that the QIO will be evaluated based on both ``reliability'' and
``validity of review.'' This commenter also requested clarification as
to why Tasks 3a and 3b require reliability while Task 3c does not
require validity for evaluation.
Response: The reliability of the review is the primary criterion
for evaluating this component of the task. We will ensure consistency
in documents released for the 8th SOW. The evaluation criteria were
chosen for each subtask in Task 3 based on the appropriateness for the
task.
Comment: One commenter expressed concern over using Medicare
physician billing as the method to measure the rate of statewide and
identified participants' improvement in quality care measures for Task
1d.
Response: We are investigating this method of measuring improvement
for the Round 1 evaluations, and have so far found nothing large-scale
or systematic that would alter evaluation results for Task 1d. We
believe that the evaluation measures are relatively stable and reliable
estimates, and that billing issues as a whole do not contribute
significant bias to these estimates. We understand the limitations of
using billing information to estimate quality improvement, and are
working to minimize its impact by identifying these problems and
reporting questionable billing issues to the appropriate parties.
We are adopting the provisions of the notice with comment as final.
IV. Executive Order 12866 Statement
In accordance with the provisions of Executive Order 12866, this
notice with comment period was not reviewed by the Office of Management
and Budget.
Authority: Section 1153 of the Social Security Act (42 U.S.C.
1320c-2).
(Catalog of Federal Domestic Assistance Program No. 93.774,
Medicare--Supplementary Medical Insurance Program)
Dated: March 14, 2005.
Mark B. McClellan,
Administrator, Centers for Medicare & Medicaid Services.
[FR Doc. 05-14505 Filed 7-21-05; 8:45 am]
BILLING CODE 4120-01-P
