Request for Information Regarding the 21st Century Cures Act Electronic Health Record Reporting Program, 42913-42919 [2018-18297]

Agencies

• DEPARTMENT OF HEALTH AND HUMAN SERVICES

[Federal Register Volume 83, Number 165 (Friday, August 24, 2018)]
[Notices]
[Pages 42913-42919]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2018-18297]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES


Request for Information Regarding the 21st Century Cures Act 
Electronic Health Record Reporting Program

AGENCY: Office of the National Coordinator for Health Information 
Technology (ONC), HHS.

ACTION: Request for information.

-----------------------------------------------------------------------

SUMMARY: This request for information (RFI) seeks input from the public 
regarding the Electronic Health Record (EHR) Reporting Program 
established as Section 4002 of the 21st Century Cures Act (Cures Act) 
codified Section 3009A in Title XXX of the Public Health Service Act 
(PHSA). This RFI is a first step toward implementing the statute. Its 
responses will be used to inform subsequent discussions among 
stakeholders and future work toward the development of reporting 
criteria under the EHR Reporting Program.

DATES: To be assured consideration, written or electronic comments must 
be received at one of the addresses provided below, no later than 5 
p.m. on October 17, 2018.

ADDRESSES: The public should address written comments on the proposed 
system of records to https://www.regulations.gov or to the HHS Office of 
Security and Strategic Information (OSSI), 200 Independence Avenue SW, 
Washington, DC 20201.
     Federal eRulemaking Portal: Follow the instructions for 
submitting comments. Attachments should be in Microsoft Word, Microsoft 
Excel, or Adobe PDF; however, we prefer Microsoft Word.
     Regular, Express, or Overnight Mail: Department of Health 
and Human Services, Office of the National Coordinator for Health 
Information Technology, Attention: EHR Reporting Program Request for 
Information, Mary E. Switzer Building, Mail Stop: 7033A, 330 C Street 
SW, Washington, DC 20201. Please submit one original and two copies.
     Hand Delivery or Courier: Office of the National 
Coordinator for Health Information Technology, Attention: EHR Reporting 
Program Request for Information, Mary E. Switzer Building, Mail Stop: 
7033A, 330 C Street SW, Washington, DC 20201. Please submit one 
original and two copies. (Because access to the interior of the Mary E. 
Switzer Building is not readily available to persons without federal 
government identification, commenters are encouraged to leave their 
comments in the mail drop slots located in the main lobby of the 
building.)
    Enhancing the Public Comment Experience: To facilitate public 
comment on this RFI, a copy will be made available in Microsoft Word 
format on ONC's website (https://www.healthit.gov).
    Inspection of Public Comments: All comments received before the 
close of the comment period will be available for public inspection, 
including any personally identifiable or confidential business 
information that is included in a comment. Please do not include 
anything in your comment submission that you do not wish to share with 
the general public. Such information includes, but is not limited to: A 
person's social security number; date of birth; driver's license 
number; state identification number or foreign country equivalent; 
passport number; financial account number; credit or debit card number; 
any personal health information; or any business information that could 
be considered proprietary. We will post all comments that are received 
before the close of the comment period at https://www.regulations.gov.
    Comments received timely will also be available for public 
inspection, generally beginning approximately 3 weeks after publication 
of a document at Office of the National Coordinator for Health 
Information Technology, 330 C Street SW, Room 7033A, Washington, DC 
20201. Contact Michael Wittie, listed below, to arrange for inspection.
    Docket: For access to the docket to read background documents or 
comments received, go to https://www.regulations.gov or the Department 
of Health and Human Services, Office of the National Coordinator for 
Health Information Technology, Mary E. Switzer Building, Mail Stop: 
7033A, 330 C Street SW, Washington, DC 20201 (call ahead to the contact 
listed below to arrange for inspection).

FOR FURTHER INFORMATION CONTACT: Michael Wittie, Office of Policy, 
Office of the National Coordinator for Health Information Technology, 
202-690-7151, [email protected] or Lauren Richie, Office of 
Policy, Office of the National Coordinator for Health Information 
Technology, 202-690-7151, [email protected].

SUPPLEMENTARY INFORMATION: 

I. Background

    The Secretary has delegated authority to the Office of the National 
Coordinator for Health Information Technology (ONC) to carry out the 
provisions of sections 4002(a) and 4002(c) of the Cures Act. Section 
4002(a) creates PHSA section 3001(c)(5)(D) and instructs the Secretary 
to ``require, as a condition of certification and maintenance of 
certification'' that health IT developers satisfy certain requirements, 
including submitting ``reporting criteria in accordance with section 
3009A(b).'' Section 4002(c) creates PHSA Section 3009A and requires the 
Secretary to develop an ``Electronic Health Record Reporting Program'' 
(EHR Reporting Program or Program). Section 3009A also calls on the 
Secretary to lead a public, transparent process to establish the 
``reporting criteria'' associated with the EHR Reporting Program. 
Section 3009A directs the Secretary to award

[[Page 42914]]

grants, contracts, or agreements to independent entities to support the 
EHR Reporting Program. For the purposes of this RFI and the Program, 
the term ``certified health IT'' includes the full range of potential 
technologies, functions, and systems for which HHS has adopted 
standards, implementation specifications, and certification criteria 
under the ONC Health IT Certification Program.\1\ ONC will engage a 
contractor to convene stakeholders and use the responses to this RFI to 
inform stakeholder discussion in order to formally develop these 
criteria.
---------------------------------------------------------------------------

    \1\ For further discussion, see the DEFINITIONS FOR CERTIFIED 
HEALTH IT AND CEHRT section of the 2016 Report on the Feasibility of 
Mechanisms to Assist Providers in Comparing and Selecting Certified 
EHR Technology Products (https://www.healthit.gov/sites/default/files/macraehrpct_final_4-2016.pdf).
---------------------------------------------------------------------------

    The Cures Act requires the EHR Reporting Program's reporting 
criteria to address the following five categories: Security; 
interoperability; usability and user-centered design; conformance to 
certification testing; and other categories, as appropriate to measure 
the performance of certified EHR technology. The Cures Act also 
suggests several other categories for consideration, including, but not 
limited to: Enabling users to order and view results of laboratory 
tests, imaging tests, and other diagnostic tests; exchanging data with 
clinical registries; accessing and exchanging data from medical 
devices, health information exchanges, and other health care providers; 
accessing and exchanging data held by federal, state, and local 
agencies.
    For the purposes of this RFI, we have focused our questions on the 
five mandatory categories from the Cures Act. However, the public is 
welcome to comment on any of the additional categories noted by the 
Cures Act (please consult section 3009A(a)(3)(B)).

The ONC Health IT Certification Program

    The ONC Health IT Certification Program provides a process to 
support certifying health information technology (health IT) to the 
appropriate standards, implementation specifications, and certification 
criteria that have been adopted by the Secretary.\2\ As a result, since 
2015, nearly all hospitals and most physicians used health IT certified 
under the ONC Health IT Certification Program.\3\ The 2015 Edition 
certification criteria is the most recent edition of certification 
criteria adopted by the Secretary for use in the ONC Health IT 
Certification Program.
---------------------------------------------------------------------------

    \2\ Understanding Certified Health IT. https://www.healthit.gov/sites/default/files/understanding-certified-;health-it-2.pdf.
    \3\ https://dashboard.healthit.gov/apps/health-information-technology-data-summaries.php?state=National&cat9=all+data&cat1=ehr+adoption.
---------------------------------------------------------------------------

II. Solicitation of Comments

    This RFI includes two main sections for public comment:
     Cross-cutting: Requests input on priorities on the 
intersection of health IT product-related reporting criteria and 
healthcare provider reporting criteria; and
     Categories: Requests input on specific focus areas, 
including the reporting criteria categories required by the Cures Act.
    In reviewing the RFI questions, commenters should consider existing 
sources of information about health IT products. Commenters should also 
consider how reporting criteria for different stakeholders could be 
constructed based on their differing perspectives, especially, for 
example, since health IT developers will be required to respond to 
reporting criteria for their product(s) in order to maintain the 
product's certification. To prevent duplication of efforts, commenters 
should consider what other information is lacking from the existing 
sources about health IT products and what the reporting criteria under 
the EHR Reporting Program could uniquely contribute.
    Overall, we seek input about reporting criteria that will be used 
to:
     Show distinct, measurable differences between products;
     Describe the functionalities of health IT products varying 
by the setting where implemented (e.g., primary versus specialty care);
     Provide timely and reliable information in ways not unduly 
burdensome to users or to small and start-up developers;
     Comparatively inform acquisition,\4\ upgrade, and 
customization decisions that best support end users' needs beyond 
currently available information; and
---------------------------------------------------------------------------

    \4\ In this RFI, all references to acquisition of certified 
health IT include purchasing, licensing, and other methods of 
obtaining technology.
---------------------------------------------------------------------------

     Support analysis for industry trends with respect to 
interoperability and other types of user experiences.
    ONC is especially interested in feedback targeting users in 
ambulatory and small practice settings, where providers typically do 
not have substantial time and resources to conduct broad market 
research. To reduce data collection burden, ONC also seeks input on the 
availability and applicability of existing data sources that could be 
used to report on this information (e.g., the reporting criteria). 
Finally, ONC seeks input on the most efficient processes to minimize 
stakeholder burden for collecting and reporting the information.

III. Cross-Cutting Topics

    Existing Data Sources:
    In 2016, ONC released the Report to Congress on the Feasibility of 
Mechanisms to Assist Providers in Comparing and Selecting Certified EHR 
Technology Products (EHR Compare Report). The report was based on 
market analysis and insight from subject matter experts including the 
ONC Certified Technology Comparison Task Force of the Joint Health IT 
Policy and Health IT Standards Committees. It described mechanisms for 
improving the health care community's ability to compare and select 
certified health IT. The report identified and described existing 
sources of health IT comparison data, as well as gaps in the 
information available with possible mechanisms to address those gaps. 
The sources identified in the report are listed in Appendix A.
    ONC is interested in stakeholders' input on currently existing 
sources of health IT comparison data as well as gaps in such 
information since the EHR Compare Report release.
    Questions:
     Please identify any sources of health IT comparison 
information that were not in the EHR Compare Report that would be 
helpful as potential reporting criteria are considered. In addition, 
please comment on whether any of the sources of health IT comparison 
information that were available at the time of the EHR Compare Report 
have changed notably or are no longer available.
     Which, if any, of these sources are particularly relevant 
or should be considered as they relate to certified health IT for 
ambulatory and small practice settings?
    Given the wide range of data that is reported to HHS and other 
agencies, we seek to avoid duplicate reporting through the EHR 
Reporting Program. We are interested in stakeholders' input on 
information already available from health IT acquisition decision 
makers and users who report to Federal programs that could be re-used 
and factored into the EHR Reporting Program. We are particularly 
interested in any data reported by providers participating in Centers 
for Medicare & Medicaid Services (CMS) programs since they can be 
considered verified users of certified health IT.
    Questions:

[[Page 42915]]

     What, if any, types of information reported by providers 
as part of their participation in HHS programs would be useful for the 
EHR Reporting Program (e.g., to inform health IT acquisition, upgrade, 
or customization decisions)?
     What data reported to State agencies (e.g., Medicaid EHR 
Incentive Program data), if available nationally, would be useful for 
the EHR Reporting Program?
    Data Reported by Health IT Developers versus End-Users:
    User-reported data can help assess interoperability,\5\ the 
usability of information that is exchanged, and the accessibility of 
that information to end users. There may also be areas where it would 
be useful to obtain both qualitative end user experiences as well as 
qualitative information from the developers on the same aspect of a 
particular EHR Reporting Program criterion, such as interoperability. 
Such information may provide insights into how well a certified health 
IT product is performing from both perspectives. However, there may be 
criteria where developers, as opposed to acquisition decision makers 
and end users, would serve as the primary source of information.
---------------------------------------------------------------------------

    \5\ Section 4003 of the Cures Act provides that 
interoperability: (1) Enables secure exchange and use of electronic 
health information without special effort on the part of the user; 
(2) allows for complete access, exchange, and use of all 
electronically accessible health information for authorized use; and 
(C) does not constitute information blocking.
---------------------------------------------------------------------------

    Questions:
     What types of reporting criteria should developers of 
certified health IT report about their certified health IT products:
    [cir] That would be important to use in identifying trends, 
assessing interoperability and successful exchange of health care 
information, and supporting assessment of user experiences?
    [cir] That would be valuable to those acquiring health IT in making 
health IT acquisition, upgrade, or customization decisions that best 
support end users' needs?
     What types of reporting criteria for health care 
providers, patients, and other users of certified health IT products 
would be most useful in making technology acquisition, upgrade, or 
customization decisions to best support end users' needs?
     What kinds of user-reported information are health IT 
acquisition decision makers using now; how are they used in comparing 
systems; and do they remain relevant today?
     What types of reporting criteria would be useful to obtain 
from both developers and end users to inform health IT comparisons? 
What about these types of reporting criteria makes them particularly 
amenable to reporting from both the developer and end user perspective?
    User-Reported Criteria:
    The Cures Act calls for collecting EHR Reporting Program reporting 
criteria information from health care providers, patients, and other 
users of certified EHR technology, as well as from developers. As 
addressed in the EHR Compare Report, there are currently private sector 
resources where users can provide and view reviews of health IT 
products. However, the resources may be improved upon because they are 
not comprehensive, reflective of verified users' views, nor accessible 
and affordable to all.
    ONC is interested in input about what user-submitted information 
would make the EHR Reporting Program a valuable addition to the 
existing landscape of market research and analysis. ONC is also 
interested in feedback on what factors might influence end users' 
decisions to report more easily.
    Questions:
     How can data be collected without creating or increasing 
burden on providers?
     What recommendations do stakeholders have to improve the 
timeliness of the data so there are not significant lags between its 
collection and publication?
     Describe the value, if any, in an EHR Reporting Program 
function that would display reviews from existing sources, or provided 
a current list with hyperlinks to access them.
     Discuss the benefits and limitations of requiring users be 
verified before submitting reviews. What should be required for such 
verification?
     Which reporting criteria are applicable generally across 
all providers? What reporting criteria would require customization 
across different provider types and specialties, including small 
practices and those in underserved areas?
     For what settings (e.g., hospitals, primary care 
physicians, or specialties) would comparable information on certified 
health IT be most helpful? If naming several settings, please list in 
your order of priority.
     How helpful are qualitative user reviews (such as `star 
ratings' or Likert scales) compared to objective reports (e.g., that a 
system works as expected with quantifiable measures)? Which specific 
types of information are better reflected in one of these formats or 
another?
     How could HHS encourage clinicians, patients, and other 
users to share their experiences with certified health IT?
     Which particular reporting mechanisms, if any, should be 
avoided?
    Health IT Developer-Reported Criteria:
    The Cures Act requires that health IT developers report information 
on certified health IT as a condition of certification and maintenance 
of certification under the ONC Health IT Certification Program. A 
common set of criteria reported by health IT developers could help 
acquisition decision makers compare across products to make more 
informed decisions that best support end users' needs. Such reporting 
criteria could also be used to establish a consistent set of metrics to 
provide a baseline and identify trends over time in key focus areas 
associated with health IT use and interoperability.
    However, there may be information that uniform reporting criteria 
may not adequately reflect, particularly in health IT targeted towards 
smaller or specialized settings with specific needs. A mixed approach 
that blends common and optional sets of reporting criteria may better 
address the needs of providers and developers of varying sizes and 
settings.
    Questions:
     If you have used the certified health IT product data 
available on the ONC Certified Health IT Products List (CHPL) to 
compare products (e.g., to inform acquisition, upgrade, or 
customization decisions), what information was most helpful and what 
was missing? If providing a brief list of the information, please 
prioritize the information from most helpful to least helpful also 
considering their grouping into categories in Section IV.
     Would a common set of criteria reported on by all 
developers of certified health IT, or a mixed approach blending common 
and optional sets of criteria, be more effective as we implement the 
EHR Reporting Program?
     What developer-reported criteria are particularly 
relevant, or not relevant, to health IT users and acquisition decision 
makers in the ambulatory and small practice settings?
     Which criteria topics might be especially burdensome or 
difficult for a small or new developer to report on?
     What types of criteria might introduce bias (e.g., unfair 
advantage) in favor of larger, established developers or in favor of 
small or new developers?
     In what ways can different health IT deployment 
architectures be accommodated? For instance, are there certain types of 
criteria that cloud-based

[[Page 42916]]

certified health IT developers would be better able to report on versus 
those who are not cloud-based? How might this affect generating and 
reporting information on criteria?

IV. Categories for the EHR Reporting Program

    The Cures Act requires the following categories to be addressed 
when it comes to EHR Reporting Program reporting criteria. Please 
consult the end of this RFI section for specific questions on the many 
other reporting criteria categories suggested by the Cures Act.
     Security;
     Usability and user-centered design;
     Interoperability;
     Conformance to certification testing; and
     Other categories, as appropriate to measure the 
performance of certified EHR technology.
    Questions:
     What categories of reporting criteria are end users most 
interested in (e.g., security, usability and user-centered design, 
interoperability, conformance to certification testing)? Please list by 
priority.
    Security:
    The ONC Health IT Certification Program supports the privacy and 
security of electronic health information by establishing a detailed 
set of requirements that health IT developers must meet for their 
products to be certified to the Privacy and Security certification 
criteria. Implementation of these capabilities can also help certified 
health IT users meet certain Health Insurance Portability and 
Accountability Act (HIPAA) compliance requirements.
    Questions:
     What reporting criteria could provide information on 
meaningful differences between products in the ease and effectiveness 
that they enable end users to meet their security and privacy needs?
     Describe other useful security and privacy features or 
functions that a certified health IT product may offer beyond those 
required by HIPAA and the ONC Health IT Certification Program, such as 
functions related to requirements under 42 CFR part 2.
    What information about a certified health IT product's security and 
privacy capabilities and performance have acquisition decision makers 
used to inform decisions about acquisitions, upgrades, or use to best 
support end users' needs? How has that information helped inform 
decision-making? What other information would be useful in comparing 
certified health IT products on security and privacy (e.g., 
compatibility with newer security technologies such as biometrics)?
    Usability and User-Centered Design:
    Usability centers on the extent to which a system supports a user 
to efficiently and effectively achieve their desired goals.\6\ Poor 
usability of health IT systems can contribute to clinician burden and 
physician burnout,\7\ and problems with usability may lead to risks to 
patient safety and end user error.\8\ \9\ Traditionally, usability 
assessment involves analyses of clinician workflow, including error 
rates and time spent on specific tasks. Usability assessments use 
methods that include conducting time-motion studies \10\ and other 
qualitative measures that gather end users' input about their 
experiences using the system.\11\
---------------------------------------------------------------------------

    \6\ ISO 9241-11; https://www.iso.org/obp/ui/#iso:std:iso:ts:20282:-2:ed-2:v1:en.
    \7\ Friedberg M, et. al. Factors Affecting Physician 
Professional Satisfaction and Their Implications for Patient Care, 
Health Systems, and Health Policy. RAND Corporation, 2013. https://www.rand.org/content/dam/rand/pubs/research_reports/RR400/RR439/RAND_RR439.pdf.
    \8\ NISTIR 7804-1, https://nvlpubs.nist.gov/nistpubs/ir/2015/NIST.IR.7804-1.pdf.
    \9\ ONC Health IT Playbook. https://www.healthit.gov/playbook/electronic-health-records/#section-1-4.
    \10\ Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders 
L, Westbrook J, Tutty M, Blike G. Allocation of Physician Time in 
Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann 
Intern Med. 2016 Dec 6;165(11):753-760.
    \11\ Friedberg MW, Chen PG, Van Busum KR, Aunon F, Pham C, 
Caloyeras J, Mattke S, Pitchforth E, Quigley DD, Brook RH, Crosson 
FJ, Tutty M. Factors Affecting Physician Professional Satisfaction 
and Their Implications for Patient Care, Health Systems, and Health 
Policy. Rand Health Q. 2014 Dec 1;3(4):1.
---------------------------------------------------------------------------

    User-Centered Design (UCD) is a type of development process that 
can improve system usability. UCD considers users' needs during each 
stage of system design and development, and is designed to lead to more 
usable end products. To have their products certified, health IT 
developers must attest that they employed a UCD process and report the 
results of usability testing on certain technical functions. The 
results--including measures such as time to perform certain tasks, the 
number of individuals used in the testing, and Likert scale scores that 
rate usability of the technical functions--are available on the 
CHPL.\12\ The UCD process implemented will likely vary based on the 
health IT functionality certified.\13\ Thus, it may be difficult to use 
certification results to assess and compare effective use of UCD across 
all certified health IT products.
---------------------------------------------------------------------------

    \12\ Certified Health IT Product List. https://chpl.healthit.gov/#/search.
    \13\ Medstar National Center for Human Factors in Healthcare. 
EHR User-Centered Design Evaluation Framework. https://www.medicalhumanfactors.net/ehr-vendor-framework/.
---------------------------------------------------------------------------

    An important method for evaluating the usability of health IT 
products is through an analysis of information from users' experiences 
and data in real-world settings. Recent studies\14\ \15\ have used 
audit logs to examine physicians' time spent on specific tasks, and 
some cloud-based health IT systems have the capability to monitor this 
to optimize workflow.\16\ However, given that tasks and workflows may 
vary by specialty, setting, and other factors, it may be difficult to 
compare the results across systems.
---------------------------------------------------------------------------

    \14\ Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, 
Sinsky CA, Gilchrist VJ. Tethered to the EHR: Primary Care Physician 
Workload Assessment Using EHR Event Log Data and Time-Motion 
Observations. Ann Fam Med. 2017 Sep;15(5):419-426.
    \15\ Tai-Seale M, Olson CW, Li J, Chan AS, Morikawa C, Durbin M, 
Wang W, Luft HS. Electronic Health Record Logs Indicate That 
Physicians Split Time Evenly Between Seeing Patients And Desktop 
Medicine. Health Aff (Millwood). 2017 Apr 1;36(4):655-662.A.
    \16\ Robert Wachter. The Digital Doctor. New York, NY: McGraw-
Hill Education; 2015.
---------------------------------------------------------------------------

    With qualitative assessments being a key part of assessing 
usability, subjective user assessments are complementary to 
quantitative measures, such as time to perform tasks. Information about 
the source of reviews, such as if a reviewer has actually used the 
system, their setting, specialty, and background (e.g., as a clinician, 
practice manager, etc.), may affect the value of the reviews to health 
IT acquisition decision makers. As noted in the EHR Compare Report, 
resources exist that provide user reviews, though these may be 
outdated.
    Questions:
     How can the usability results currently available in the 
CHPL best be used to assist in comparisons between certified health IT 
products?
     Describe the availability and feasibility of common 
frameworks or standard scores from established usability assessment 
tools that would allow acquisition decision makers to compare usability 
of systems.
     Discuss the merits and risks of seeking a common set of 
measures for the purpose of real world testing that health IT 
developers could use to compare usability of systems. What specific 
types of data from current users would reflect how well the certified 
health IT product:
    [cir] Supports the cognitive work of clinical users (e.g., displays 
relevant information in useful formats at relevant points in workflow)?
    [cir] Reflects the ability of implementers to make customization 
and

[[Page 42917]]

implementation decisions in a user-centered manner?
     What usability assessment data, if available, are less 
resource intensive than traditional measures (e.g., time motion 
studies)?
     Comment on the feasibility and applicability of usability 
measures created from audit log data. How would health IT acquisition 
decision makers use this information to improve their system 
acquisition, upgrade, and customization decisions to best support end 
users' needs?
    [cir] Who should report audit log data and by what mechanism?
     How feasible would it be to implement usage monitoring 
tools (e.g., for time spent on specific tasks)?
    Interoperability:
    The Cures Act defines interoperability as: ``(A) enables the secure 
exchange of electronic health information with, and use of electronic 
health information from, other health information technology without 
special effort on the part of the user; (B) allows for complete access, 
exchange, and use of all electronically accessible health information 
for authorized use under applicable State or Federal law; and (C) does 
not constitute information blocking.''
    The EHR Compare Report identified product integration as a 
potential means to assess interoperability and proposed federal and 
private sector strategies to address it. The National Quality Forum's 
Measurement Framework to Assess Nationwide Progress Related to 
Interoperable Health Information Exchange to Support the National 
Quality Strategy also specified various domains of interoperability 
that might be useful to measure per the health IT consumers' 
perspective.\17\ Applicable domains include the exchange of electronic 
health information (referring to the availability of electronic health 
information, method of exchange, and quality of data content), and the 
usability of the exchanged electronic health information (referring to 
issues related to relevance, accessibility and comprehensibility of the 
information that is exchanged).
---------------------------------------------------------------------------

    \17\ https://www.qualityforum.org/Publications/2017/09/Interoperability_2016-2017_Final_Report.aspx.
---------------------------------------------------------------------------

    Two existing data sources with many Medicare providers are the 
Inpatient Hospital Promoting Interoperability Program and the Merit-
based Incentive Payment System (MIPS), which include measures related 
to health information exchange and interoperability.\18\ The exchange-
related measures within this domain primarily focus on a summary of 
care record exchange (e.g., send and receipt/acceptance transaction) 
and clinical information reconciliation. The measures would provide 
insights on health IT product performance related to summary of care 
record exchange. For additional information, we encourage reviewers to 
reference the 2019 IPPS Final Rule and the 2019 PFS which includes the 
Quality Payment Program (QPP) NPRM for proposed changes that may impact 
what information is submitted to Medicare.
---------------------------------------------------------------------------

    \18\ Quality Payment Program. https://qpp.cms.gov/mips/advancing-care-information.
---------------------------------------------------------------------------

    Industry reports (such as those listed in Appendix 3 of the EHR 
Compare Report), which typically involve provider surveys, serve as 
another data source to assess interoperability. However, the 
applicability of these reports may be limited to larger providers and 
health systems and may not necessarily reflect the experiences or needs 
of small practices and all settings (e.g., behavioral health). In 
addition, industry reports may not be affordable to all users.
    Questions:
     Please comment on the usefulness of product integration as 
a primary means of assessing interoperability (as proposed in the EHR 
Compare Report).
     What other domains of interoperability (beyond those 
already identified and referenced above) would be useful for 
comparative purposes?
     Of the data sources described in this RFI, which data 
sources would be useful for measuring the interoperability performance 
of certified health IT products?
    [cir] Comment on whether State Medicaid agencies would be able to 
share detailed attestation-level data for the purpose of developing 
reports at a more detailed level, such as by health IT product. If so, 
how would this information be useful to compare performance on 
interoperability across health IT products?
    [cir] How helpful would CMS program data (e.g., Quality Payment 
Program MIPS Promoting Interoperability Category, Inpatient Hospital 
Promoting Interoperability Program, Medicaid Promoting Interoperability 
Programs) related to exchange and interoperability be for comparative 
purposes? What measures should be selected for this purpose? Given that 
some of these data may be reported across providers rather than at the 
individual clinical level, how would this affect reporting of 
performance by health IT product?
     What other data sources and measures could be used to 
compare performance on interoperability across certified health IT 
products?
    Conformance to Certification Testing:
    Health IT that has been submitted by a health IT developer for 
certification and successfully tested and certified is listed in the 
CHPL,\19\ which is an online, openly available resource. This data 
ranges from the user-centered design and transparency disclosures made 
by health IT developers to the certification criteria to which health 
IT has been certified. However, user experiences, product performance, 
and interoperability-oriented metrics from health IT developers and 
healthcare providers are not reported in a consistent way across all 
products certified through the ONC Health IT Certification Program.
---------------------------------------------------------------------------

    \19\ Certified Health IT Product List. https://chpl.healthit.gov/#/search.
---------------------------------------------------------------------------

    ONC-prepared materials to support certification testing, such as 
the 2015 Edition Test Method, are intended to be read and understood 
with the express purpose of evaluating the health IT's functional 
capabilities that have been submitted for certification in a controlled 
environment, but are not determinative of the full scope of the health 
IT's capabilities, such as in a production environment,\20\ 
Nevertheless, testing results for health IT are available on the CHPL, 
and much of it in a structured format that makes the reported test 
results accessible for analysis and comparison.
---------------------------------------------------------------------------

    \20\ 2015 Edition Test Method. https://beta.healthit.gov/topic/certification-ehrs/2015-edition-test-method.
---------------------------------------------------------------------------

    As part of maintaining certification and ensuring ongoing 
conformance to certification testing, ONC-Authorized Certification 
Bodies (ONC-ACB) perform surveillance of health IT products and verify 
that the certified health IT also conforms in the production 
environment (i.e., ``in the field''). This includes reviewing any 
complaints about or potential issues with certified health IT products 
brought to their attention (i.e., reactive surveillance). They may also 
elect to conduct randomized surveillance.\21\ \22\
---------------------------------------------------------------------------

    \21\ Program Guidance #17-02: ONC Exercises Enforcement 
Discretion With Respect to Implementation of Randomized 
Surveillance. Retrieved from https://www.healthit.gov/sites/default/files/ONC_Enforcement_Discretion_Randomized_Surveillance_8-30-17.pdf.
    \22\ Certified Health IT Product List (CHPL) Public User Guide. 
Retrieved from https://www.healthit.gov/sites/default/files/policy/chpl_public_user_guide.pdf.
---------------------------------------------------------------------------

    If a certified health IT product does not demonstrate the 
functionality required by its certification, the certified health IT 
product is considered non-conforming and then listed and detailed as 
non-conforming on the CHPL.\23\ A list

[[Page 42918]]

of banned developers \24\ is maintained on the CHPL in addition to a 
list of decertified health IT \25\ where certification was withdrawn by 
the developer's ONC-ACB, by the developer under surveillance/review, or 
terminated by ONC. The available surveillance information about non-
conformant health IT and developers provides an avenue that can help 
potential consumers evaluate and compare how certified health IT 
performs in real-world settings. In addition, developers must post 
mandatory disclosures on types of additional costs and limitations for 
their certified health IT as part of the ONC Health IT Certification 
Program requirements.
---------------------------------------------------------------------------

    \23\ Products: Corrective Action Status. https://chpl.healthit.gov/#/collections/correctiveAction.
    \24\ Developers Under Certification Ban. https://chpl.healthit.gov/#/collections/developers.
    \25\ Decertified Products. https://chpl.healthit.gov/#/collections/products.
---------------------------------------------------------------------------

    Question:
     What additional information about certified health IT's 
conformance to the certification testing (beyond what is currently 
available on the CHPL) would be useful for comparison purposes? What 
mechanisms or approaches could be considered to obtain such data? What 
barriers might exist for developers and/or end users in reporting on 
such data?
    Other Categories for Consideration:
    The Cures Act lists other possible categories for the EHR Reporting 
Program related to certified health IT product performance, including:
     Enabling the user to order and view the results of 
laboratory tests, imaging tests, and other diagnostic tests;
     Submitting, editing, and retrieving data from registries, 
such as clinician-led clinical data registries;
     Accessing and exchanging information and data from and 
through health information exchanges;
     Accessing and exchanging information and data from medical 
devices;
     Accessing and exchanging information and data held by 
Federal, State, and local agencies and other applicable entities useful 
to a health care provider or other applicable user in the furtherance 
of patient care;
     Accessing and exchanging information from other health 
care providers or applicable users;
     Accessing and exchanging patient generated information;
     Providing the patient or an authorized designee with a 
complete copy of their health information from an electronic health 
record in a computable format; and
     Providing accurate patient information for the correct 
patient, including exchanging such information, and avoiding the 
duplication of patients records.
    Questions:
     How should the above categories be prioritized for 
inclusion/exclusion in the EHR Reporting Program, and why? What other 
criteria would be helpful for comparative purposes to best support end 
users' needs (e.g., to inform health IT acquisition, upgrade, and 
implementation decisions)?
     What data sources could be used to compare performance on 
these categories across certified health IT products?
     Please comment on different types of information, or 
measures, in this area that would be useful to acquisition, upgrade, 
and customization decisions in the ambulatory setting as opposed to 
inpatient settings?
    In addition to the other categories listed in the Cures Act, the 
EHR Compare Report identified gaps in information related to the 
domains of cost transparency, quality metrics, and population health. 
The cost transparency domain includes base, subscription, and 
transaction costs, as well as peer reviews regarding price 
expectations, which could also allow acquisition decision makers to 
make more informed acquisition, upgrade, and customization decisions to 
best support end users' needs.
    Questions:
     Please comment on the usefulness and feasibility of 
including criteria on quality reporting and population health in the 
EHR Reporting Program. What criteria should be considered to assess 
health IT performance in generating quality measures, reporting quality 
measures, and the functions required for supporting population health 
analytics (e.g., bulk data export)?
     What data sources, if any, are available to assess 
certified health IT product capabilities and performance in collecting, 
generating, and reporting on quality measures, and the ability to 
export multiple records for population health analytics? Are these data 
sources publicly available?
     Please comment on other categories, if any, besides those 
listed in this RFI that should be considered to be included in the EHR 
Reporting Program. Why should these be included, and what data sources 
exist to report on performance for the suggested categories?
    Hospitals and Health Systems:
    The focus of this RFI is on the information needs of health IT end 
users in ambulatory and small practice settings, as these groups report 
challenges accessing relevant information at affordable costs to help 
them compare certified health IT. However, ONC is aware that there are 
also gaps in the availability of information that hospitals and health 
systems need.
    Questions:
     Please describe the types of comparative information about 
certified health IT hospitals and health systems currently use (e.g., 
to inform health IT acquisition, upgrade, and customization decisions). 
What are the sources of this information? What information would be 
useful but is currently unavailable?
     What types of comparative information about certified 
health IT, if any, are specifically useful to hospitals and health 
systems, as opposed to ambulatory or small practices? What types of 
information could be collected or reported that would be helpful to 
both hospitals and health systems and to ambulatory and smaller 
providers?
     Please comment on how an EHR Reporting Program could best 
reflect the information needed for hospitals and health systems, 
ambulatory and smaller provider settings, and overlapping information 
in developing summary reports or comparison tools.

V. Collection of Information Requirements

    This document does not impose information collection requirements, 
that is, reporting, recordkeeping or third-party disclosure 
requirements. Consequently, there is no need for review by the Office 
of Management and Budget under the authority of the Paperwork Reduction 
Act of 1995 (44 U.S.C. 3501 et seq.).

VI. Response to Comments

    ONC typically receives a large public response to its published 
Federal Register documents. ONC will consider all comments received by 
the date and time specified in the DATES section of this document, but 
will not be able to acknowledge or respond individually to public 
comments.

    Dated: August 17, 2018.
Donald Rucker,
National Coordinator, Office of the National Coordinator for Health 
Information Technology.

[[Page 42919]]



 Appendix A--Certified Health IT Comparison Tools Identified Through ONC
                             Market Research
------------------------------------------------------------------------
       Comparison tool                      Company website
------------------------------------------------------------------------
4Med+ Marketplace............  www.4medapproved.com/wizard/marketplace.
AmericanEHR..................  www.americanehr.com.
Blackbook....................  www.blackbookrankings.com/healthcare.
California Healthcare          www.chcf.org/publications/2007/10/ehr-
 Foundation.                    selection-toolkit-for-community-health-
                                centers.
CHPL 4.0.....................  www.healthit.gov/chpl.
Consumer Affairs.............  www.consumeraffairs.com/emr-software.
EHR Compare..................  www.ehrcompare.com.
EHR in Practice..............  www.ehrinpractice.com/ehr-product-comparison.html.
Gartner......................  www.gartner.com.
HealthRecord.US..............  www.healthrecord.us.
IDC Health Insights..........  www.idc.com.
KLAS.........................  www.klasresearch.com.
LeadingAge...................  www.leadingage.org/ehr/search.aspx.
NCQA.........................  www.ncqa.org/Programs/Recognition/practices/PatientCenteredMedicalHomePCMH/PCMHPrevalidationProgram/VendorList.aspx.
Software Advice..............  www.softwareadvice.com.
Software Insider.............  www.ehr.softwareinsider.com.
Technology Advice............  www.technologyadvice.com/medical/ehr-emr/smart-advisor.
Texas Medical Association      www.texmed.org/EHRTool.
 (TMA).
------------------------------------------------------------------------

[FR Doc. 2018-18297 Filed 8-23-18; 8:45 am]
 BILLING CODE 4150-45-P