Request for Information Regarding the 21st Century Cures Act Electronic Health Record Reporting Program, 42913-42919 [2018-18297]

Download as PDF Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices permit evaluation of possible sources of conflicts of interest. A nomination package should include the following information for each nominee: (1) A letter of nomination from an employer, a colleague, or a professional organization stating the name, affiliation, and contact information for the nominee, the basis for the nomination (i.e., what specific attributes, perspectives, and/or skills does the individual possess that would benefit the workings of the NACNEP, and the nominee’s field(s) of expertise); (2) A letter of interest from the nominee stating the reasons they would like to serve on the NACNEP; (3) A biographical sketch of the nominee, a copy of his/her curriculum vitae, and his/her contact information (address, daytime telephone number, and email address); and (4) The name, address, daytime telephone number, and email address at which the nominator can be contacted. Nominations will be considered as vacancies occur on the NACNEP. If you submitted a nomination more than four years ago, please resubmit an updated nomination to be considered for council vacancies. HHS strives to ensure that the membership of HHS federal advisory committees are balanced in terms of points of view represented and the committee’s function. The Department encourages nominations of qualified candidates from all groups and locations. Appointment to the NACNEP shall be made without discrimination on the basis of age, race, ethnicity, gender, sexual orientation, disability, and cultural, religious, or socioeconomic status. Amy P. McNulty, Acting Director, Division of Executive Secretariat. [FR Doc. 2018–18344 Filed 8–23–18; 8:45 am] BILLING CODE 4165–15–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Request for Information Regarding the 21st Century Cures Act Electronic Health Record Reporting Program Office of the National Coordinator for Health Information Technology (ONC), HHS. ACTION: Request for information. daltland on DSKBBV9HB2PROD with NOTICES AGENCY: This request for information (RFI) seeks input from the public regarding the Electronic Health Record (EHR) Reporting Program established as Section 4002 of the 21st Century Cures SUMMARY: VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 Act (Cures Act) codified Section 3009A in Title XXX of the Public Health Service Act (PHSA). This RFI is a first step toward implementing the statute. Its responses will be used to inform subsequent discussions among stakeholders and future work toward the development of reporting criteria under the EHR Reporting Program. DATES: To be assured consideration, written or electronic comments must be received at one of the addresses provided below, no later than 5 p.m. on October 17, 2018. ADDRESSES: The public should address written comments on the proposed system of records to https:// www.regulations.gov or to the HHS Office of Security and Strategic Information (OSSI), 200 Independence Avenue SW, Washington, DC 20201. • Federal eRulemaking Portal: Follow the instructions for submitting comments. Attachments should be in Microsoft Word, Microsoft Excel, or Adobe PDF; however, we prefer Microsoft Word. • Regular, Express, or Overnight Mail: Department of Health and Human Services, Office of the National Coordinator for Health Information Technology, Attention: EHR Reporting Program Request for Information, Mary E. Switzer Building, Mail Stop: 7033A, 330 C Street SW, Washington, DC 20201. Please submit one original and two copies. • Hand Delivery or Courier: Office of the National Coordinator for Health Information Technology, Attention: EHR Reporting Program Request for Information, Mary E. Switzer Building, Mail Stop: 7033A, 330 C Street SW, Washington, DC 20201. Please submit one original and two copies. (Because access to the interior of the Mary E. Switzer Building is not readily available to persons without federal government identification, commenters are encouraged to leave their comments in the mail drop slots located in the main lobby of the building.) Enhancing the Public Comment Experience: To facilitate public comment on this RFI, a copy will be made available in Microsoft Word format on ONC’s website (https:// www.healthit.gov). Inspection of Public Comments: All comments received before the close of the comment period will be available for public inspection, including any personally identifiable or confidential business information that is included in a comment. Please do not include anything in your comment submission that you do not wish to share with the general public. Such information PO 00000 Frm 00055 Fmt 4703 Sfmt 4703 42913 includes, but is not limited to: A person’s social security number; date of birth; driver’s license number; state identification number or foreign country equivalent; passport number; financial account number; credit or debit card number; any personal health information; or any business information that could be considered proprietary. We will post all comments that are received before the close of the comment period at https:// www.regulations.gov. Comments received timely will also be available for public inspection, generally beginning approximately 3 weeks after publication of a document at Office of the National Coordinator for Health Information Technology, 330 C Street SW, Room 7033A, Washington, DC 20201. Contact Michael Wittie, listed below, to arrange for inspection. Docket: For access to the docket to read background documents or comments received, go to https:// www.regulations.gov or the Department of Health and Human Services, Office of the National Coordinator for Health Information Technology, Mary E. Switzer Building, Mail Stop: 7033A, 330 C Street SW, Washington, DC 20201 (call ahead to the contact listed below to arrange for inspection). FOR FURTHER INFORMATION CONTACT: Michael Wittie, Office of Policy, Office of the National Coordinator for Health Information Technology, 202–690–7151, Michael.Wittie@hhs.gov or Lauren Richie, Office of Policy, Office of the National Coordinator for Health Information Technology, 202–690–7151, Lauren.Richie@hhs.gov. SUPPLEMENTARY INFORMATION: I. Background The Secretary has delegated authority to the Office of the National Coordinator for Health Information Technology (ONC) to carry out the provisions of sections 4002(a) and 4002(c) of the Cures Act. Section 4002(a) creates PHSA section 3001(c)(5)(D) and instructs the Secretary to ‘‘require, as a condition of certification and maintenance of certification’’ that health IT developers satisfy certain requirements, including submitting ‘‘reporting criteria in accordance with section 3009A(b).’’ Section 4002(c) creates PHSA Section 3009A and requires the Secretary to develop an ‘‘Electronic Health Record Reporting Program’’ (EHR Reporting Program or Program). Section 3009A also calls on the Secretary to lead a public, transparent process to establish the ‘‘reporting criteria’’ associated with the EHR Reporting Program. Section 3009A directs the Secretary to award E:\FR\FM\24AUN1.SGM 24AUN1 42914 Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices grants, contracts, or agreements to independent entities to support the EHR Reporting Program. For the purposes of this RFI and the Program, the term ‘‘certified health IT’’ includes the full range of potential technologies, functions, and systems for which HHS has adopted standards, implementation specifications, and certification criteria under the ONC Health IT Certification Program.1 ONC will engage a contractor to convene stakeholders and use the responses to this RFI to inform stakeholder discussion in order to formally develop these criteria. The Cures Act requires the EHR Reporting Program’s reporting criteria to address the following five categories: Security; interoperability; usability and user-centered design; conformance to certification testing; and other categories, as appropriate to measure the performance of certified EHR technology. The Cures Act also suggests several other categories for consideration, including, but not limited to: Enabling users to order and view results of laboratory tests, imaging tests, and other diagnostic tests; exchanging data with clinical registries; accessing and exchanging data from medical devices, health information exchanges, and other health care providers; accessing and exchanging data held by federal, state, and local agencies. For the purposes of this RFI, we have focused our questions on the five mandatory categories from the Cures Act. However, the public is welcome to comment on any of the additional categories noted by the Cures Act (please consult section 3009A(a)(3)(B)). daltland on DSKBBV9HB2PROD with NOTICES The ONC Health IT Certification Program The ONC Health IT Certification Program provides a process to support certifying health information technology (health IT) to the appropriate standards, implementation specifications, and certification criteria that have been adopted by the Secretary.2 As a result, since 2015, nearly all hospitals and most physicians used health IT certified under the ONC Health IT Certification Program.3 The 2015 Edition certification 1 For further discussion, see the DEFINITIONS FOR CERTIFIED HEALTH IT AND CEHRT section of the 2016 Report on the Feasibility of Mechanisms to Assist Providers in Comparing and Selecting Certified EHR Technology Products (https:// www.healthit.gov/sites/default/files/macraehrpct_ final_4-2016.pdf). 2 Understanding Certified Health IT. https:// www.healthit.gov/sites/default/files/understandingcertified-;health-it-2.pdf. 3 https://dashboard.healthit.gov/apps/healthinformation-technology-data-summaries.php?state= National&cat9=all+data&cat1=ehr+adoption. VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 criteria is the most recent edition of certification criteria adopted by the Secretary for use in the ONC Health IT Certification Program. II. Solicitation of Comments This RFI includes two main sections for public comment: • Cross-cutting: Requests input on priorities on the intersection of health IT product-related reporting criteria and healthcare provider reporting criteria; and • Categories: Requests input on specific focus areas, including the reporting criteria categories required by the Cures Act. In reviewing the RFI questions, commenters should consider existing sources of information about health IT products. Commenters should also consider how reporting criteria for different stakeholders could be constructed based on their differing perspectives, especially, for example, since health IT developers will be required to respond to reporting criteria for their product(s) in order to maintain the product’s certification. To prevent duplication of efforts, commenters should consider what other information is lacking from the existing sources about health IT products and what the reporting criteria under the EHR Reporting Program could uniquely contribute. Overall, we seek input about reporting criteria that will be used to: • Show distinct, measurable differences between products; • Describe the functionalities of health IT products varying by the setting where implemented (e.g., primary versus specialty care); • Provide timely and reliable information in ways not unduly burdensome to users or to small and start-up developers; • Comparatively inform acquisition,4 upgrade, and customization decisions that best support end users’ needs beyond currently available information; and • Support analysis for industry trends with respect to interoperability and other types of user experiences. ONC is especially interested in feedback targeting users in ambulatory and small practice settings, where providers typically do not have substantial time and resources to conduct broad market research. To reduce data collection burden, ONC also seeks input on the availability and applicability of existing data sources 4 In this RFI, all references to acquisition of certified health IT include purchasing, licensing, and other methods of obtaining technology. PO 00000 Frm 00056 Fmt 4703 Sfmt 4703 that could be used to report on this information (e.g., the reporting criteria). Finally, ONC seeks input on the most efficient processes to minimize stakeholder burden for collecting and reporting the information. III. Cross-Cutting Topics Existing Data Sources: In 2016, ONC released the Report to Congress on the Feasibility of Mechanisms to Assist Providers in Comparing and Selecting Certified EHR Technology Products (EHR Compare Report). The report was based on market analysis and insight from subject matter experts including the ONC Certified Technology Comparison Task Force of the Joint Health IT Policy and Health IT Standards Committees. It described mechanisms for improving the health care community’s ability to compare and select certified health IT. The report identified and described existing sources of health IT comparison data, as well as gaps in the information available with possible mechanisms to address those gaps. The sources identified in the report are listed in Appendix A. ONC is interested in stakeholders’ input on currently existing sources of health IT comparison data as well as gaps in such information since the EHR Compare Report release. Questions: • Please identify any sources of health IT comparison information that were not in the EHR Compare Report that would be helpful as potential reporting criteria are considered. In addition, please comment on whether any of the sources of health IT comparison information that were available at the time of the EHR Compare Report have changed notably or are no longer available. • Which, if any, of these sources are particularly relevant or should be considered as they relate to certified health IT for ambulatory and small practice settings? Given the wide range of data that is reported to HHS and other agencies, we seek to avoid duplicate reporting through the EHR Reporting Program. We are interested in stakeholders’ input on information already available from health IT acquisition decision makers and users who report to Federal programs that could be re-used and factored into the EHR Reporting Program. We are particularly interested in any data reported by providers participating in Centers for Medicare & Medicaid Services (CMS) programs since they can be considered verified users of certified health IT. Questions: E:\FR\FM\24AUN1.SGM 24AUN1 daltland on DSKBBV9HB2PROD with NOTICES Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices • What, if any, types of information reported by providers as part of their participation in HHS programs would be useful for the EHR Reporting Program (e.g., to inform health IT acquisition, upgrade, or customization decisions)? • What data reported to State agencies (e.g., Medicaid EHR Incentive Program data), if available nationally, would be useful for the EHR Reporting Program? Data Reported by Health IT Developers versus End-Users: User-reported data can help assess interoperability,5 the usability of information that is exchanged, and the accessibility of that information to end users. There may also be areas where it would be useful to obtain both qualitative end user experiences as well as qualitative information from the developers on the same aspect of a particular EHR Reporting Program criterion, such as interoperability. Such information may provide insights into how well a certified health IT product is performing from both perspectives. However, there may be criteria where developers, as opposed to acquisition decision makers and end users, would serve as the primary source of information. Questions: • What types of reporting criteria should developers of certified health IT report about their certified health IT products: Æ That would be important to use in identifying trends, assessing interoperability and successful exchange of health care information, and supporting assessment of user experiences? Æ That would be valuable to those acquiring health IT in making health IT acquisition, upgrade, or customization decisions that best support end users’ needs? • What types of reporting criteria for health care providers, patients, and other users of certified health IT products would be most useful in making technology acquisition, upgrade, or customization decisions to best support end users’ needs? • What kinds of user-reported information are health IT acquisition decision makers using now; how are they used in comparing systems; and do they remain relevant today? • What types of reporting criteria would be useful to obtain from both 5 Section 4003 of the Cures Act provides that interoperability: (1) Enables secure exchange and use of electronic health information without special effort on the part of the user; (2) allows for complete access, exchange, and use of all electronically accessible health information for authorized use; and (C) does not constitute information blocking. VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 developers and end users to inform health IT comparisons? What about these types of reporting criteria makes them particularly amenable to reporting from both the developer and end user perspective? User-Reported Criteria: The Cures Act calls for collecting EHR Reporting Program reporting criteria information from health care providers, patients, and other users of certified EHR technology, as well as from developers. As addressed in the EHR Compare Report, there are currently private sector resources where users can provide and view reviews of health IT products. However, the resources may be improved upon because they are not comprehensive, reflective of verified users’ views, nor accessible and affordable to all. ONC is interested in input about what user-submitted information would make the EHR Reporting Program a valuable addition to the existing landscape of market research and analysis. ONC is also interested in feedback on what factors might influence end users’ decisions to report more easily. Questions: • How can data be collected without creating or increasing burden on providers? • What recommendations do stakeholders have to improve the timeliness of the data so there are not significant lags between its collection and publication? • Describe the value, if any, in an EHR Reporting Program function that would display reviews from existing sources, or provided a current list with hyperlinks to access them. • Discuss the benefits and limitations of requiring users be verified before submitting reviews. What should be required for such verification? • Which reporting criteria are applicable generally across all providers? What reporting criteria would require customization across different provider types and specialties, including small practices and those in underserved areas? • For what settings (e.g., hospitals, primary care physicians, or specialties) would comparable information on certified health IT be most helpful? If naming several settings, please list in your order of priority. • How helpful are qualitative user reviews (such as ‘star ratings’ or Likert scales) compared to objective reports (e.g., that a system works as expected with quantifiable measures)? Which specific types of information are better reflected in one of these formats or another? PO 00000 Frm 00057 Fmt 4703 Sfmt 4703 42915 • How could HHS encourage clinicians, patients, and other users to share their experiences with certified health IT? • Which particular reporting mechanisms, if any, should be avoided? Health IT Developer-Reported Criteria: The Cures Act requires that health IT developers report information on certified health IT as a condition of certification and maintenance of certification under the ONC Health IT Certification Program. A common set of criteria reported by health IT developers could help acquisition decision makers compare across products to make more informed decisions that best support end users’ needs. Such reporting criteria could also be used to establish a consistent set of metrics to provide a baseline and identify trends over time in key focus areas associated with health IT use and interoperability. However, there may be information that uniform reporting criteria may not adequately reflect, particularly in health IT targeted towards smaller or specialized settings with specific needs. A mixed approach that blends common and optional sets of reporting criteria may better address the needs of providers and developers of varying sizes and settings. Questions: • If you have used the certified health IT product data available on the ONC Certified Health IT Products List (CHPL) to compare products (e.g., to inform acquisition, upgrade, or customization decisions), what information was most helpful and what was missing? If providing a brief list of the information, please prioritize the information from most helpful to least helpful also considering their grouping into categories in Section IV. • Would a common set of criteria reported on by all developers of certified health IT, or a mixed approach blending common and optional sets of criteria, be more effective as we implement the EHR Reporting Program? • What developer-reported criteria are particularly relevant, or not relevant, to health IT users and acquisition decision makers in the ambulatory and small practice settings? • Which criteria topics might be especially burdensome or difficult for a small or new developer to report on? • What types of criteria might introduce bias (e.g., unfair advantage) in favor of larger, established developers or in favor of small or new developers? • In what ways can different health IT deployment architectures be accommodated? For instance, are there certain types of criteria that cloud-based E:\FR\FM\24AUN1.SGM 24AUN1 42916 Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices daltland on DSKBBV9HB2PROD with NOTICES certified health IT developers would be better able to report on versus those who are not cloud-based? How might this affect generating and reporting information on criteria? IV. Categories for the EHR Reporting Program The Cures Act requires the following categories to be addressed when it comes to EHR Reporting Program reporting criteria. Please consult the end of this RFI section for specific questions on the many other reporting criteria categories suggested by the Cures Act. • Security; • Usability and user-centered design; • Interoperability; • Conformance to certification testing; and • Other categories, as appropriate to measure the performance of certified EHR technology. Questions: • What categories of reporting criteria are end users most interested in (e.g., security, usability and user-centered design, interoperability, conformance to certification testing)? Please list by priority. Security: The ONC Health IT Certification Program supports the privacy and security of electronic health information by establishing a detailed set of requirements that health IT developers must meet for their products to be certified to the Privacy and Security certification criteria. Implementation of these capabilities can also help certified health IT users meet certain Health Insurance Portability and Accountability Act (HIPAA) compliance requirements. Questions: • What reporting criteria could provide information on meaningful differences between products in the ease and effectiveness that they enable end users to meet their security and privacy needs? • Describe other useful security and privacy features or functions that a certified health IT product may offer beyond those required by HIPAA and the ONC Health IT Certification Program, such as functions related to requirements under 42 CFR part 2. What information about a certified health IT product’s security and privacy capabilities and performance have acquisition decision makers used to inform decisions about acquisitions, upgrades, or use to best support end users’ needs? How has that information helped inform decision-making? What other information would be useful in comparing certified health IT products on security and privacy (e.g., VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 compatibility with newer security technologies such as biometrics)? Usability and User-Centered Design: Usability centers on the extent to which a system supports a user to efficiently and effectively achieve their desired goals.6 Poor usability of health IT systems can contribute to clinician burden and physician burnout,7 and problems with usability may lead to risks to patient safety and end user error.8 9 Traditionally, usability assessment involves analyses of clinician workflow, including error rates and time spent on specific tasks. Usability assessments use methods that include conducting time-motion studies 10 and other qualitative measures that gather end users’ input about their experiences using the system.11 User-Centered Design (UCD) is a type of development process that can improve system usability. UCD considers users’ needs during each stage of system design and development, and is designed to lead to more usable end products. To have their products certified, health IT developers must attest that they employed a UCD process and report the results of usability testing on certain technical functions. The results—including measures such as time to perform certain tasks, the number of individuals used in the testing, and Likert scale scores that rate usability of the technical functions—are available on the CHPL.12 The UCD process implemented will likely vary based on the health IT functionality certified.13 Thus, it may be difficult to use certification results to assess and 6 ISO 9241–11; https://www.iso.org/obp/ui/ #iso:std:iso:ts:20282:-2:ed-2:v1:en. 7 Friedberg M, et. al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy. RAND Corporation, 2013. https://www.rand.org/ content/dam/rand/pubs/research_reports/RR400/ RR439/RAND_RR439.pdf. 8 NISTIR 7804–1, https://nvlpubs.nist.gov/ nistpubs/ir/2015/NIST.IR.7804-1.pdf. 9 ONC Health IT Playbook. https:// www.healthit.gov/playbook/electronic-healthrecords/#section-1-4. 10 Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders L, Westbrook J, Tutty M, Blike G. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann Intern Med. 2016 Dec 6;165(11):753–760. 11 Friedberg MW, Chen PG, Van Busum KR, Aunon F, Pham C, Caloyeras J, Mattke S, Pitchforth E, Quigley DD, Brook RH, Crosson FJ, Tutty M. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy. Rand Health Q. 2014 Dec 1;3(4):1. 12 Certified Health IT Product List. https:// chpl.healthit.gov/#/search. 13 Medstar National Center for Human Factors in Healthcare. EHR User-Centered Design Evaluation Framework. https://www.medicalhumanfactors.net/ ehr-vendor-framework/. PO 00000 Frm 00058 Fmt 4703 Sfmt 4703 compare effective use of UCD across all certified health IT products. An important method for evaluating the usability of health IT products is through an analysis of information from users’ experiences and data in realworld settings. Recent studies14 15 have used audit logs to examine physicians’ time spent on specific tasks, and some cloud-based health IT systems have the capability to monitor this to optimize workflow.16 However, given that tasks and workflows may vary by specialty, setting, and other factors, it may be difficult to compare the results across systems. With qualitative assessments being a key part of assessing usability, subjective user assessments are complementary to quantitative measures, such as time to perform tasks. Information about the source of reviews, such as if a reviewer has actually used the system, their setting, specialty, and background (e.g., as a clinician, practice manager, etc.), may affect the value of the reviews to health IT acquisition decision makers. As noted in the EHR Compare Report, resources exist that provide user reviews, though these may be outdated. Questions: • How can the usability results currently available in the CHPL best be used to assist in comparisons between certified health IT products? • Describe the availability and feasibility of common frameworks or standard scores from established usability assessment tools that would allow acquisition decision makers to compare usability of systems. • Discuss the merits and risks of seeking a common set of measures for the purpose of real world testing that health IT developers could use to compare usability of systems. What specific types of data from current users would reflect how well the certified health IT product: Æ Supports the cognitive work of clinical users (e.g., displays relevant information in useful formats at relevant points in workflow)? Æ Reflects the ability of implementers to make customization and 14 Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, Sinsky CA, Gilchrist VJ. Tethered to the EHR: Primary Care Physician Workload Assessment Using EHR Event Log Data and TimeMotion Observations. Ann Fam Med. 2017 Sep;15(5):419–426. 15 Tai-Seale M, Olson CW, Li J, Chan AS, Morikawa C, Durbin M, Wang W, Luft HS. Electronic Health Record Logs Indicate That Physicians Split Time Evenly Between Seeing Patients And Desktop Medicine. Health Aff (Millwood). 2017 Apr 1;36(4):655–662.A. 16 Robert Wachter. The Digital Doctor. New York, NY: McGraw-Hill Education; 2015. E:\FR\FM\24AUN1.SGM 24AUN1 daltland on DSKBBV9HB2PROD with NOTICES Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices implementation decisions in a usercentered manner? • What usability assessment data, if available, are less resource intensive than traditional measures (e.g., time motion studies)? • Comment on the feasibility and applicability of usability measures created from audit log data. How would health IT acquisition decision makers use this information to improve their system acquisition, upgrade, and customization decisions to best support end users’ needs? Æ Who should report audit log data and by what mechanism? • How feasible would it be to implement usage monitoring tools (e.g., for time spent on specific tasks)? Interoperability: The Cures Act defines interoperability as: ‘‘(A) enables the secure exchange of electronic health information with, and use of electronic health information from, other health information technology without special effort on the part of the user; (B) allows for complete access, exchange, and use of all electronically accessible health information for authorized use under applicable State or Federal law; and (C) does not constitute information blocking.’’ The EHR Compare Report identified product integration as a potential means to assess interoperability and proposed federal and private sector strategies to address it. The National Quality Forum’s Measurement Framework to Assess Nationwide Progress Related to Interoperable Health Information Exchange to Support the National Quality Strategy also specified various domains of interoperability that might be useful to measure per the health IT consumers’ perspective.17 Applicable domains include the exchange of electronic health information (referring to the availability of electronic health information, method of exchange, and quality of data content), and the usability of the exchanged electronic health information (referring to issues related to relevance, accessibility and comprehensibility of the information that is exchanged). Two existing data sources with many Medicare providers are the Inpatient Hospital Promoting Interoperability Program and the Merit-based Incentive Payment System (MIPS), which include measures related to health information exchange and interoperability.18 The exchange-related measures within this 17 https://www.qualityforum.org/Publications/ 2017/09/Interoperability_2016-2017_Final_ Report.aspx. 18 Quality Payment Program. https://qpp.cms.gov/ mips/advancing-care-information. VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 domain primarily focus on a summary of care record exchange (e.g., send and receipt/acceptance transaction) and clinical information reconciliation. The measures would provide insights on health IT product performance related to summary of care record exchange. For additional information, we encourage reviewers to reference the 2019 IPPS Final Rule and the 2019 PFS which includes the Quality Payment Program (QPP) NPRM for proposed changes that may impact what information is submitted to Medicare. Industry reports (such as those listed in Appendix 3 of the EHR Compare Report), which typically involve provider surveys, serve as another data source to assess interoperability. However, the applicability of these reports may be limited to larger providers and health systems and may not necessarily reflect the experiences or needs of small practices and all settings (e.g., behavioral health). In addition, industry reports may not be affordable to all users. Questions: • Please comment on the usefulness of product integration as a primary means of assessing interoperability (as proposed in the EHR Compare Report). • What other domains of interoperability (beyond those already identified and referenced above) would be useful for comparative purposes? • Of the data sources described in this RFI, which data sources would be useful for measuring the interoperability performance of certified health IT products? Æ Comment on whether State Medicaid agencies would be able to share detailed attestation-level data for the purpose of developing reports at a more detailed level, such as by health IT product. If so, how would this information be useful to compare performance on interoperability across health IT products? Æ How helpful would CMS program data (e.g., Quality Payment Program MIPS Promoting Interoperability Category, Inpatient Hospital Promoting Interoperability Program, Medicaid Promoting Interoperability Programs) related to exchange and interoperability be for comparative purposes? What measures should be selected for this purpose? Given that some of these data may be reported across providers rather than at the individual clinical level, how would this affect reporting of performance by health IT product? • What other data sources and measures could be used to compare performance on interoperability across certified health IT products? Conformance to Certification Testing: PO 00000 Frm 00059 Fmt 4703 Sfmt 4703 42917 Health IT that has been submitted by a health IT developer for certification and successfully tested and certified is listed in the CHPL,19 which is an online, openly available resource. This data ranges from the user-centered design and transparency disclosures made by health IT developers to the certification criteria to which health IT has been certified. However, user experiences, product performance, and interoperability-oriented metrics from health IT developers and healthcare providers are not reported in a consistent way across all products certified through the ONC Health IT Certification Program. ONC-prepared materials to support certification testing, such as the 2015 Edition Test Method, are intended to be read and understood with the express purpose of evaluating the health IT’s functional capabilities that have been submitted for certification in a controlled environment, but are not determinative of the full scope of the health IT’s capabilities, such as in a production environment,20 Nevertheless, testing results for health IT are available on the CHPL, and much of it in a structured format that makes the reported test results accessible for analysis and comparison. As part of maintaining certification and ensuring ongoing conformance to certification testing, ONC-Authorized Certification Bodies (ONC–ACB) perform surveillance of health IT products and verify that the certified health IT also conforms in the production environment (i.e., ‘‘in the field’’). This includes reviewing any complaints about or potential issues with certified health IT products brought to their attention (i.e., reactive surveillance). They may also elect to conduct randomized surveillance.21 22 If a certified health IT product does not demonstrate the functionality required by its certification, the certified health IT product is considered nonconforming and then listed and detailed as non-conforming on the CHPL.23 A list 19 Certified Health IT Product List. https:// chpl.healthit.gov/#/search. 20 2015 Edition Test Method. https:// beta.healthit.gov/topic/certification-ehrs/2015edition-test-method. 21 Program Guidance #17–02: ONC Exercises Enforcement Discretion With Respect to Implementation of Randomized Surveillance. Retrieved from https://www.healthit.gov/sites/ default/files/ONC_Enforcement_Discretion_ Randomized_Surveillance_8-30-17.pdf. 22 Certified Health IT Product List (CHPL) Public User Guide. Retrieved from https:// www.healthit.gov/sites/default/files/policy/chpl_ public_user_guide.pdf. 23 Products: Corrective Action Status. https:// chpl.healthit.gov/#/collections/correctiveAction. E:\FR\FM\24AUN1.SGM 24AUN1 daltland on DSKBBV9HB2PROD with NOTICES 42918 Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices of banned developers 24 is maintained on the CHPL in addition to a list of decertified health IT 25 where certification was withdrawn by the developer’s ONC–ACB, by the developer under surveillance/review, or terminated by ONC. The available surveillance information about nonconformant health IT and developers provides an avenue that can help potential consumers evaluate and compare how certified health IT performs in real-world settings. In addition, developers must post mandatory disclosures on types of additional costs and limitations for their certified health IT as part of the ONC Health IT Certification Program requirements. Question: • What additional information about certified health IT’s conformance to the certification testing (beyond what is currently available on the CHPL) would be useful for comparison purposes? What mechanisms or approaches could be considered to obtain such data? What barriers might exist for developers and/ or end users in reporting on such data? Other Categories for Consideration: The Cures Act lists other possible categories for the EHR Reporting Program related to certified health IT product performance, including: • Enabling the user to order and view the results of laboratory tests, imaging tests, and other diagnostic tests; • Submitting, editing, and retrieving data from registries, such as clinicianled clinical data registries; • Accessing and exchanging information and data from and through health information exchanges; • Accessing and exchanging information and data from medical devices; • Accessing and exchanging information and data held by Federal, State, and local agencies and other applicable entities useful to a health care provider or other applicable user in the furtherance of patient care; • Accessing and exchanging information from other health care providers or applicable users; • Accessing and exchanging patient generated information; • Providing the patient or an authorized designee with a complete copy of their health information from an electronic health record in a computable format; and • Providing accurate patient information for the correct patient, including exchanging such information, and avoiding the duplication of patients records. Questions: • How should the above categories be prioritized for inclusion/exclusion in the EHR Reporting Program, and why? What other criteria would be helpful for comparative purposes to best support end users’ needs (e.g., to inform health IT acquisition, upgrade, and implementation decisions)? • What data sources could be used to compare performance on these categories across certified health IT products? • Please comment on different types of information, or measures, in this area that would be useful to acquisition, upgrade, and customization decisions in the ambulatory setting as opposed to inpatient settings? In addition to the other categories listed in the Cures Act, the EHR Compare Report identified gaps in information related to the domains of cost transparency, quality metrics, and population health. The cost transparency domain includes base, subscription, and transaction costs, as well as peer reviews regarding price expectations, which could also allow acquisition decision makers to make more informed acquisition, upgrade, and customization decisions to best support end users’ needs. Questions: • Please comment on the usefulness and feasibility of including criteria on quality reporting and population health in the EHR Reporting Program. What criteria should be considered to assess health IT performance in generating quality measures, reporting quality measures, and the functions required for supporting population health analytics (e.g., bulk data export)? • What data sources, if any, are available to assess certified health IT product capabilities and performance in collecting, generating, and reporting on quality measures, and the ability to export multiple records for population health analytics? Are these data sources publicly available? • Please comment on other categories, if any, besides those listed in this RFI that should be considered to be included in the EHR Reporting Program. Why should these be included, and what data sources exist to report on performance for the suggested categories? Hospitals and Health Systems: 24 Developers Under Certification Ban. https:// chpl.healthit.gov/#/collections/developers. 25 Decertified Products. https://chpl.healthit.gov/ #/collections/products. VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 PO 00000 Frm 00060 Fmt 4703 Sfmt 4703 The focus of this RFI is on the information needs of health IT end users in ambulatory and small practice settings, as these groups report challenges accessing relevant information at affordable costs to help them compare certified health IT. However, ONC is aware that there are also gaps in the availability of information that hospitals and health systems need. Questions: • Please describe the types of comparative information about certified health IT hospitals and health systems currently use (e.g., to inform health IT acquisition, upgrade, and customization decisions). What are the sources of this information? What information would be useful but is currently unavailable? • What types of comparative information about certified health IT, if any, are specifically useful to hospitals and health systems, as opposed to ambulatory or small practices? What types of information could be collected or reported that would be helpful to both hospitals and health systems and to ambulatory and smaller providers? • Please comment on how an EHR Reporting Program could best reflect the information needed for hospitals and health systems, ambulatory and smaller provider settings, and overlapping information in developing summary reports or comparison tools. V. Collection of Information Requirements This document does not impose information collection requirements, that is, reporting, recordkeeping or third-party disclosure requirements. Consequently, there is no need for review by the Office of Management and Budget under the authority of the Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.). VI. Response to Comments ONC typically receives a large public response to its published Federal Register documents. ONC will consider all comments received by the date and time specified in the DATES section of this document, but will not be able to acknowledge or respond individually to public comments. Dated: August 17, 2018. Donald Rucker, National Coordinator, Office of the National Coordinator for Health Information Technology. E:\FR\FM\24AUN1.SGM 24AUN1 Federal Register / Vol. 83, No. 165 / Friday, August 24, 2018 / Notices 42919 APPENDIX A—CERTIFIED HEALTH IT COMPARISON TOOLS IDENTIFIED THROUGH ONC MARKET RESEARCH Comparison tool Company website 4Med+ Marketplace .............................. AmericanEHR ....................................... Blackbook ............................................. California Healthcare Foundation ......... CHPL 4.0 .............................................. Consumer Affairs .................................. EHR Compare ...................................... EHR in Practice .................................... Gartner ................................................. HealthRecord.US .................................. IDC Health Insights .............................. KLAS .................................................... LeadingAge .......................................... NCQA ................................................... Software Advice ................................... Software Insider ................................... Technology Advice ............................... Texas Medical Association (TMA) ....... www.4medapproved.com/wizard/marketplace. www.americanehr.com. www.blackbookrankings.com/healthcare. www.chcf.org/publications/2007/10/ehr-selection-toolkit-for-community-health-centers. www.healthit.gov/chpl. www.consumeraffairs.com/emr-software. www.ehrcompare.com. www.ehrinpractice.com/ehr-product-comparison.html. www.gartner.com. www.healthrecord.us. www.idc.com. www.klasresearch.com. www.leadingage.org/ehr/search.aspx. www.ncqa.org/Programs/Recognition/practices/PatientCenteredMedicalHomePCMH/ PCMHPrevalidationProgram/VendorList.aspx. www.softwareadvice.com. www.ehr.softwareinsider.com. www.technologyadvice.com/medical/ehr-emr/smart-advisor. www.texmed.org/EHRTool. [FR Doc. 2018–18297 Filed 8–23–18; 8:45 am] BILLING CODE 4150–45–P DEPARTMENT OF HEALTH AND HUMAN SERVICES National Institutes of Health Government-Owned Inventions; Availability for Licensing AGENCY: National Institutes of Health, HHS. ACTION: Notice. The invention listed below is owned by an agency of the U.S. Government and is available for licensing to achieve expeditious commercialization of results of federally-funded research and development. Foreign patent applications are filed on selected inventions to extend market coverage for companies and may also be available for licensing. SUMMARY: daltland on DSKBBV9HB2PROD with NOTICES FOR FURTHER INFORMATION CONTACT: Vince Contreras, Ph.D., 240–669–2823; vince.contreras@nih.gov. Licensing information and copies of the U.S. patent application listed below may be obtained by communicating with the indicated licensing contact at the Technology Transfer and Intellectual Property Office, National Institute of Allergy and Infectious Diseases, 5601 Fishers Lane, Rockville, MD, 20852; tel. 301–496–2644. A signed Confidential Disclosure Agreement will be required to receive copies of unpublished patent applications. SUPPLEMENTARY INFORMATION: Technology description follows. VerDate Sep<11>2014 19:17 Aug 23, 2018 Jkt 244001 Prefusion HPIV F Immunogens and Their Use Description of Technology: Human parainfluenza virus (hPIV) is an RNAbased paramyxovirus that causes respiratory infections in children and adults. There are four serotypes that can result in a myriad of diseases of the respiratory tract including croup, bronchitis, and pneumonia (Mao et al., 2012). hPIV is a leading cause of respiratory tract infection and hospitalization among children under 5, only surpassed by the respiratory syncytial virus (RSV). Currently, there are limited treatment options and no approved vaccines. Recently, studies showed that a large proportion of neutralizing antibodies preferentially recognize exposed epitopes in the prefusion conformation of the RSV F protein, which together with other evidence suggests that creation of stabilized prefusion F protein immunogens might be a universal strategy to develop vaccine candidates for inducing protective immune responses in RSV and other related viruses, such as hPIV. Researchers at the Vaccine Research Center (VRC) of the National Institute of Allergy and Infectious Diseases created immunogenic PIV fusion (F) glycoproteins for types 1,2,3 and 4 (hPIV1, hPIV2, hPIV3 and hPIV4) that have been modified to stabilize the prefusion conformation. These stabilized prefusion F immunogens, especially hPIV3, induced high titer neutralizing responses in mice and rhesus macaques, and should thus serve as promising candidates for the prevention of PIV infection in humans. This technology is available for licensing for commercial development PO 00000 Frm 00061 Fmt 4703 Sfmt 4703 in accordance with 35 U.S.C. 209 and 37 CFR part 404. Potential Commercial Applications: • hPIV vaccines for people of all ages; • Specific focus on the elderly and young children. Competitive Advantages: • Use as a multivalent hPIV vaccine; • Use in combination with influenza or RSV vaccine compositions; • hPIV3 neutralizing titers induced in both mice and rhesus macaques were substantially higher than the highest PIV3 neutralizing titers observed in a cohort of over 100 humans. Development Stage: • In vivo testing (primates and mice). Inventors: Peter Kwong (NIAID), GwoYu Chuang (NIAID), Kai Xu (NIAID),Tongqing Zhou (NIAID), Yaroslav Tsybovsky (Leidos Biomedical Research, Inc), Aliaksandr Druz (NIAID), Antonio Lanzavecchia (Institute for Research in Biomedicine), Davide Corti (Institute for Research in Biomedicine), Guillaume BE StewartJones (NIAID), Baoshan Zhang (NIAID), Yongping Yang (NIAID), Paul Thomas (NIAID), John Mascola (NIAID), Li Ou (NIAID), Wing-pui Kong (NIAID). Intellectual Property: HHS Reference Number E–215–2016 includes U.S. Provisional Patent Application Number 62/412,699 filed 10/25/2016 and PCT Application Number PCT/US2017/ 058322 filed 10/25/2017 (pending). Related Intellectual Property: HHS Reference Number: E–064–2016. Licensing Contact: Vince Contreras, Ph.D., 240–669–2823; vince.contreras@ nih.gov. E:\FR\FM\24AUN1.SGM 24AUN1

Agencies

[Federal Register Volume 83, Number 165 (Friday, August 24, 2018)]
[Notices]
[Pages 42913-42919]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2018-18297]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES


Request for Information Regarding the 21st Century Cures Act 
Electronic Health Record Reporting Program

AGENCY: Office of the National Coordinator for Health Information 
Technology (ONC), HHS.

ACTION: Request for information.

-----------------------------------------------------------------------

SUMMARY: This request for information (RFI) seeks input from the public 
regarding the Electronic Health Record (EHR) Reporting Program 
established as Section 4002 of the 21st Century Cures Act (Cures Act) 
codified Section 3009A in Title XXX of the Public Health Service Act 
(PHSA). This RFI is a first step toward implementing the statute. Its 
responses will be used to inform subsequent discussions among 
stakeholders and future work toward the development of reporting 
criteria under the EHR Reporting Program.

DATES: To be assured consideration, written or electronic comments must 
be received at one of the addresses provided below, no later than 5 
p.m. on October 17, 2018.

ADDRESSES: The public should address written comments on the proposed 
system of records to https://www.regulations.gov or to the HHS Office of 
Security and Strategic Information (OSSI), 200 Independence Avenue SW, 
Washington, DC 20201.
     Federal eRulemaking Portal: Follow the instructions for 
submitting comments. Attachments should be in Microsoft Word, Microsoft 
Excel, or Adobe PDF; however, we prefer Microsoft Word.
     Regular, Express, or Overnight Mail: Department of Health 
and Human Services, Office of the National Coordinator for Health 
Information Technology, Attention: EHR Reporting Program Request for 
Information, Mary E. Switzer Building, Mail Stop: 7033A, 330 C Street 
SW, Washington, DC 20201. Please submit one original and two copies.
     Hand Delivery or Courier: Office of the National 
Coordinator for Health Information Technology, Attention: EHR Reporting 
Program Request for Information, Mary E. Switzer Building, Mail Stop: 
7033A, 330 C Street SW, Washington, DC 20201. Please submit one 
original and two copies. (Because access to the interior of the Mary E. 
Switzer Building is not readily available to persons without federal 
government identification, commenters are encouraged to leave their 
comments in the mail drop slots located in the main lobby of the 
building.)
    Enhancing the Public Comment Experience: To facilitate public 
comment on this RFI, a copy will be made available in Microsoft Word 
format on ONC's website (https://www.healthit.gov).
    Inspection of Public Comments: All comments received before the 
close of the comment period will be available for public inspection, 
including any personally identifiable or confidential business 
information that is included in a comment. Please do not include 
anything in your comment submission that you do not wish to share with 
the general public. Such information includes, but is not limited to: A 
person's social security number; date of birth; driver's license 
number; state identification number or foreign country equivalent; 
passport number; financial account number; credit or debit card number; 
any personal health information; or any business information that could 
be considered proprietary. We will post all comments that are received 
before the close of the comment period at https://www.regulations.gov.
    Comments received timely will also be available for public 
inspection, generally beginning approximately 3 weeks after publication 
of a document at Office of the National Coordinator for Health 
Information Technology, 330 C Street SW, Room 7033A, Washington, DC 
20201. Contact Michael Wittie, listed below, to arrange for inspection.
    Docket: For access to the docket to read background documents or 
comments received, go to https://www.regulations.gov or the Department 
of Health and Human Services, Office of the National Coordinator for 
Health Information Technology, Mary E. Switzer Building, Mail Stop: 
7033A, 330 C Street SW, Washington, DC 20201 (call ahead to the contact 
listed below to arrange for inspection).

FOR FURTHER INFORMATION CONTACT: Michael Wittie, Office of Policy, 
Office of the National Coordinator for Health Information Technology, 
202-690-7151, [email protected] or Lauren Richie, Office of 
Policy, Office of the National Coordinator for Health Information 
Technology, 202-690-7151, [email protected].

SUPPLEMENTARY INFORMATION: 

I. Background

    The Secretary has delegated authority to the Office of the National 
Coordinator for Health Information Technology (ONC) to carry out the 
provisions of sections 4002(a) and 4002(c) of the Cures Act. Section 
4002(a) creates PHSA section 3001(c)(5)(D) and instructs the Secretary 
to ``require, as a condition of certification and maintenance of 
certification'' that health IT developers satisfy certain requirements, 
including submitting ``reporting criteria in accordance with section 
3009A(b).'' Section 4002(c) creates PHSA Section 3009A and requires the 
Secretary to develop an ``Electronic Health Record Reporting Program'' 
(EHR Reporting Program or Program). Section 3009A also calls on the 
Secretary to lead a public, transparent process to establish the 
``reporting criteria'' associated with the EHR Reporting Program. 
Section 3009A directs the Secretary to award

[[Page 42914]]

grants, contracts, or agreements to independent entities to support the 
EHR Reporting Program. For the purposes of this RFI and the Program, 
the term ``certified health IT'' includes the full range of potential 
technologies, functions, and systems for which HHS has adopted 
standards, implementation specifications, and certification criteria 
under the ONC Health IT Certification Program.\1\ ONC will engage a 
contractor to convene stakeholders and use the responses to this RFI to 
inform stakeholder discussion in order to formally develop these 
criteria.
---------------------------------------------------------------------------

    \1\ For further discussion, see the DEFINITIONS FOR CERTIFIED 
HEALTH IT AND CEHRT section of the 2016 Report on the Feasibility of 
Mechanisms to Assist Providers in Comparing and Selecting Certified 
EHR Technology Products (https://www.healthit.gov/sites/default/files/macraehrpct_final_4-2016.pdf).
---------------------------------------------------------------------------

    The Cures Act requires the EHR Reporting Program's reporting 
criteria to address the following five categories: Security; 
interoperability; usability and user-centered design; conformance to 
certification testing; and other categories, as appropriate to measure 
the performance of certified EHR technology. The Cures Act also 
suggests several other categories for consideration, including, but not 
limited to: Enabling users to order and view results of laboratory 
tests, imaging tests, and other diagnostic tests; exchanging data with 
clinical registries; accessing and exchanging data from medical 
devices, health information exchanges, and other health care providers; 
accessing and exchanging data held by federal, state, and local 
agencies.
    For the purposes of this RFI, we have focused our questions on the 
five mandatory categories from the Cures Act. However, the public is 
welcome to comment on any of the additional categories noted by the 
Cures Act (please consult section 3009A(a)(3)(B)).

The ONC Health IT Certification Program

    The ONC Health IT Certification Program provides a process to 
support certifying health information technology (health IT) to the 
appropriate standards, implementation specifications, and certification 
criteria that have been adopted by the Secretary.\2\ As a result, since 
2015, nearly all hospitals and most physicians used health IT certified 
under the ONC Health IT Certification Program.\3\ The 2015 Edition 
certification criteria is the most recent edition of certification 
criteria adopted by the Secretary for use in the ONC Health IT 
Certification Program.
---------------------------------------------------------------------------

    \2\ Understanding Certified Health IT. https://www.healthit.gov/sites/default/files/understanding-certified-;health-it-2.pdf.
    \3\ https://dashboard.healthit.gov/apps/health-information-technology-data-summaries.php?state=National&cat9=all+data&cat1=ehr+adoption.
---------------------------------------------------------------------------

II. Solicitation of Comments

    This RFI includes two main sections for public comment:
     Cross-cutting: Requests input on priorities on the 
intersection of health IT product-related reporting criteria and 
healthcare provider reporting criteria; and
     Categories: Requests input on specific focus areas, 
including the reporting criteria categories required by the Cures Act.
    In reviewing the RFI questions, commenters should consider existing 
sources of information about health IT products. Commenters should also 
consider how reporting criteria for different stakeholders could be 
constructed based on their differing perspectives, especially, for 
example, since health IT developers will be required to respond to 
reporting criteria for their product(s) in order to maintain the 
product's certification. To prevent duplication of efforts, commenters 
should consider what other information is lacking from the existing 
sources about health IT products and what the reporting criteria under 
the EHR Reporting Program could uniquely contribute.
    Overall, we seek input about reporting criteria that will be used 
to:
     Show distinct, measurable differences between products;
     Describe the functionalities of health IT products varying 
by the setting where implemented (e.g., primary versus specialty care);
     Provide timely and reliable information in ways not unduly 
burdensome to users or to small and start-up developers;
     Comparatively inform acquisition,\4\ upgrade, and 
customization decisions that best support end users' needs beyond 
currently available information; and
---------------------------------------------------------------------------

    \4\ In this RFI, all references to acquisition of certified 
health IT include purchasing, licensing, and other methods of 
obtaining technology.
---------------------------------------------------------------------------

     Support analysis for industry trends with respect to 
interoperability and other types of user experiences.
    ONC is especially interested in feedback targeting users in 
ambulatory and small practice settings, where providers typically do 
not have substantial time and resources to conduct broad market 
research. To reduce data collection burden, ONC also seeks input on the 
availability and applicability of existing data sources that could be 
used to report on this information (e.g., the reporting criteria). 
Finally, ONC seeks input on the most efficient processes to minimize 
stakeholder burden for collecting and reporting the information.

III. Cross-Cutting Topics

    Existing Data Sources:
    In 2016, ONC released the Report to Congress on the Feasibility of 
Mechanisms to Assist Providers in Comparing and Selecting Certified EHR 
Technology Products (EHR Compare Report). The report was based on 
market analysis and insight from subject matter experts including the 
ONC Certified Technology Comparison Task Force of the Joint Health IT 
Policy and Health IT Standards Committees. It described mechanisms for 
improving the health care community's ability to compare and select 
certified health IT. The report identified and described existing 
sources of health IT comparison data, as well as gaps in the 
information available with possible mechanisms to address those gaps. 
The sources identified in the report are listed in Appendix A.
    ONC is interested in stakeholders' input on currently existing 
sources of health IT comparison data as well as gaps in such 
information since the EHR Compare Report release.
    Questions:
     Please identify any sources of health IT comparison 
information that were not in the EHR Compare Report that would be 
helpful as potential reporting criteria are considered. In addition, 
please comment on whether any of the sources of health IT comparison 
information that were available at the time of the EHR Compare Report 
have changed notably or are no longer available.
     Which, if any, of these sources are particularly relevant 
or should be considered as they relate to certified health IT for 
ambulatory and small practice settings?
    Given the wide range of data that is reported to HHS and other 
agencies, we seek to avoid duplicate reporting through the EHR 
Reporting Program. We are interested in stakeholders' input on 
information already available from health IT acquisition decision 
makers and users who report to Federal programs that could be re-used 
and factored into the EHR Reporting Program. We are particularly 
interested in any data reported by providers participating in Centers 
for Medicare & Medicaid Services (CMS) programs since they can be 
considered verified users of certified health IT.
    Questions:

[[Page 42915]]

     What, if any, types of information reported by providers 
as part of their participation in HHS programs would be useful for the 
EHR Reporting Program (e.g., to inform health IT acquisition, upgrade, 
or customization decisions)?
     What data reported to State agencies (e.g., Medicaid EHR 
Incentive Program data), if available nationally, would be useful for 
the EHR Reporting Program?
    Data Reported by Health IT Developers versus End-Users:
    User-reported data can help assess interoperability,\5\ the 
usability of information that is exchanged, and the accessibility of 
that information to end users. There may also be areas where it would 
be useful to obtain both qualitative end user experiences as well as 
qualitative information from the developers on the same aspect of a 
particular EHR Reporting Program criterion, such as interoperability. 
Such information may provide insights into how well a certified health 
IT product is performing from both perspectives. However, there may be 
criteria where developers, as opposed to acquisition decision makers 
and end users, would serve as the primary source of information.
---------------------------------------------------------------------------

    \5\ Section 4003 of the Cures Act provides that 
interoperability: (1) Enables secure exchange and use of electronic 
health information without special effort on the part of the user; 
(2) allows for complete access, exchange, and use of all 
electronically accessible health information for authorized use; and 
(C) does not constitute information blocking.
---------------------------------------------------------------------------

    Questions:
     What types of reporting criteria should developers of 
certified health IT report about their certified health IT products:
    [cir] That would be important to use in identifying trends, 
assessing interoperability and successful exchange of health care 
information, and supporting assessment of user experiences?
    [cir] That would be valuable to those acquiring health IT in making 
health IT acquisition, upgrade, or customization decisions that best 
support end users' needs?
     What types of reporting criteria for health care 
providers, patients, and other users of certified health IT products 
would be most useful in making technology acquisition, upgrade, or 
customization decisions to best support end users' needs?
     What kinds of user-reported information are health IT 
acquisition decision makers using now; how are they used in comparing 
systems; and do they remain relevant today?
     What types of reporting criteria would be useful to obtain 
from both developers and end users to inform health IT comparisons? 
What about these types of reporting criteria makes them particularly 
amenable to reporting from both the developer and end user perspective?
    User-Reported Criteria:
    The Cures Act calls for collecting EHR Reporting Program reporting 
criteria information from health care providers, patients, and other 
users of certified EHR technology, as well as from developers. As 
addressed in the EHR Compare Report, there are currently private sector 
resources where users can provide and view reviews of health IT 
products. However, the resources may be improved upon because they are 
not comprehensive, reflective of verified users' views, nor accessible 
and affordable to all.
    ONC is interested in input about what user-submitted information 
would make the EHR Reporting Program a valuable addition to the 
existing landscape of market research and analysis. ONC is also 
interested in feedback on what factors might influence end users' 
decisions to report more easily.
    Questions:
     How can data be collected without creating or increasing 
burden on providers?
     What recommendations do stakeholders have to improve the 
timeliness of the data so there are not significant lags between its 
collection and publication?
     Describe the value, if any, in an EHR Reporting Program 
function that would display reviews from existing sources, or provided 
a current list with hyperlinks to access them.
     Discuss the benefits and limitations of requiring users be 
verified before submitting reviews. What should be required for such 
verification?
     Which reporting criteria are applicable generally across 
all providers? What reporting criteria would require customization 
across different provider types and specialties, including small 
practices and those in underserved areas?
     For what settings (e.g., hospitals, primary care 
physicians, or specialties) would comparable information on certified 
health IT be most helpful? If naming several settings, please list in 
your order of priority.
     How helpful are qualitative user reviews (such as `star 
ratings' or Likert scales) compared to objective reports (e.g., that a 
system works as expected with quantifiable measures)? Which specific 
types of information are better reflected in one of these formats or 
another?
     How could HHS encourage clinicians, patients, and other 
users to share their experiences with certified health IT?
     Which particular reporting mechanisms, if any, should be 
avoided?
    Health IT Developer-Reported Criteria:
    The Cures Act requires that health IT developers report information 
on certified health IT as a condition of certification and maintenance 
of certification under the ONC Health IT Certification Program. A 
common set of criteria reported by health IT developers could help 
acquisition decision makers compare across products to make more 
informed decisions that best support end users' needs. Such reporting 
criteria could also be used to establish a consistent set of metrics to 
provide a baseline and identify trends over time in key focus areas 
associated with health IT use and interoperability.
    However, there may be information that uniform reporting criteria 
may not adequately reflect, particularly in health IT targeted towards 
smaller or specialized settings with specific needs. A mixed approach 
that blends common and optional sets of reporting criteria may better 
address the needs of providers and developers of varying sizes and 
settings.
    Questions:
     If you have used the certified health IT product data 
available on the ONC Certified Health IT Products List (CHPL) to 
compare products (e.g., to inform acquisition, upgrade, or 
customization decisions), what information was most helpful and what 
was missing? If providing a brief list of the information, please 
prioritize the information from most helpful to least helpful also 
considering their grouping into categories in Section IV.
     Would a common set of criteria reported on by all 
developers of certified health IT, or a mixed approach blending common 
and optional sets of criteria, be more effective as we implement the 
EHR Reporting Program?
     What developer-reported criteria are particularly 
relevant, or not relevant, to health IT users and acquisition decision 
makers in the ambulatory and small practice settings?
     Which criteria topics might be especially burdensome or 
difficult for a small or new developer to report on?
     What types of criteria might introduce bias (e.g., unfair 
advantage) in favor of larger, established developers or in favor of 
small or new developers?
     In what ways can different health IT deployment 
architectures be accommodated? For instance, are there certain types of 
criteria that cloud-based

[[Page 42916]]

certified health IT developers would be better able to report on versus 
those who are not cloud-based? How might this affect generating and 
reporting information on criteria?

IV. Categories for the EHR Reporting Program

    The Cures Act requires the following categories to be addressed 
when it comes to EHR Reporting Program reporting criteria. Please 
consult the end of this RFI section for specific questions on the many 
other reporting criteria categories suggested by the Cures Act.
     Security;
     Usability and user-centered design;
     Interoperability;
     Conformance to certification testing; and
     Other categories, as appropriate to measure the 
performance of certified EHR technology.
    Questions:
     What categories of reporting criteria are end users most 
interested in (e.g., security, usability and user-centered design, 
interoperability, conformance to certification testing)? Please list by 
priority.
    Security:
    The ONC Health IT Certification Program supports the privacy and 
security of electronic health information by establishing a detailed 
set of requirements that health IT developers must meet for their 
products to be certified to the Privacy and Security certification 
criteria. Implementation of these capabilities can also help certified 
health IT users meet certain Health Insurance Portability and 
Accountability Act (HIPAA) compliance requirements.
    Questions:
     What reporting criteria could provide information on 
meaningful differences between products in the ease and effectiveness 
that they enable end users to meet their security and privacy needs?
     Describe other useful security and privacy features or 
functions that a certified health IT product may offer beyond those 
required by HIPAA and the ONC Health IT Certification Program, such as 
functions related to requirements under 42 CFR part 2.
    What information about a certified health IT product's security and 
privacy capabilities and performance have acquisition decision makers 
used to inform decisions about acquisitions, upgrades, or use to best 
support end users' needs? How has that information helped inform 
decision-making? What other information would be useful in comparing 
certified health IT products on security and privacy (e.g., 
compatibility with newer security technologies such as biometrics)?
    Usability and User-Centered Design:
    Usability centers on the extent to which a system supports a user 
to efficiently and effectively achieve their desired goals.\6\ Poor 
usability of health IT systems can contribute to clinician burden and 
physician burnout,\7\ and problems with usability may lead to risks to 
patient safety and end user error.\8\ \9\ Traditionally, usability 
assessment involves analyses of clinician workflow, including error 
rates and time spent on specific tasks. Usability assessments use 
methods that include conducting time-motion studies \10\ and other 
qualitative measures that gather end users' input about their 
experiences using the system.\11\
---------------------------------------------------------------------------

    \6\ ISO 9241-11; https://www.iso.org/obp/ui/#iso:std:iso:ts:20282:-2:ed-2:v1:en.
    \7\ Friedberg M, et. al. Factors Affecting Physician 
Professional Satisfaction and Their Implications for Patient Care, 
Health Systems, and Health Policy. RAND Corporation, 2013. https://www.rand.org/content/dam/rand/pubs/research_reports/RR400/RR439/RAND_RR439.pdf.
    \8\ NISTIR 7804-1, https://nvlpubs.nist.gov/nistpubs/ir/2015/NIST.IR.7804-1.pdf.
    \9\ ONC Health IT Playbook. https://www.healthit.gov/playbook/electronic-health-records/#section-1-4.
    \10\ Sinsky C, Colligan L, Li L, Prgomet M, Reynolds S, Goeders 
L, Westbrook J, Tutty M, Blike G. Allocation of Physician Time in 
Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann 
Intern Med. 2016 Dec 6;165(11):753-760.
    \11\ Friedberg MW, Chen PG, Van Busum KR, Aunon F, Pham C, 
Caloyeras J, Mattke S, Pitchforth E, Quigley DD, Brook RH, Crosson 
FJ, Tutty M. Factors Affecting Physician Professional Satisfaction 
and Their Implications for Patient Care, Health Systems, and Health 
Policy. Rand Health Q. 2014 Dec 1;3(4):1.
---------------------------------------------------------------------------

    User-Centered Design (UCD) is a type of development process that 
can improve system usability. UCD considers users' needs during each 
stage of system design and development, and is designed to lead to more 
usable end products. To have their products certified, health IT 
developers must attest that they employed a UCD process and report the 
results of usability testing on certain technical functions. The 
results--including measures such as time to perform certain tasks, the 
number of individuals used in the testing, and Likert scale scores that 
rate usability of the technical functions--are available on the 
CHPL.\12\ The UCD process implemented will likely vary based on the 
health IT functionality certified.\13\ Thus, it may be difficult to use 
certification results to assess and compare effective use of UCD across 
all certified health IT products.
---------------------------------------------------------------------------

    \12\ Certified Health IT Product List. https://chpl.healthit.gov/#/search.
    \13\ Medstar National Center for Human Factors in Healthcare. 
EHR User-Centered Design Evaluation Framework. https://www.medicalhumanfactors.net/ehr-vendor-framework/.
---------------------------------------------------------------------------

    An important method for evaluating the usability of health IT 
products is through an analysis of information from users' experiences 
and data in real-world settings. Recent studies\14\ \15\ have used 
audit logs to examine physicians' time spent on specific tasks, and 
some cloud-based health IT systems have the capability to monitor this 
to optimize workflow.\16\ However, given that tasks and workflows may 
vary by specialty, setting, and other factors, it may be difficult to 
compare the results across systems.
---------------------------------------------------------------------------

    \14\ Arndt BG, Beasley JW, Watkinson MD, Temte JL, Tuan WJ, 
Sinsky CA, Gilchrist VJ. Tethered to the EHR: Primary Care Physician 
Workload Assessment Using EHR Event Log Data and Time-Motion 
Observations. Ann Fam Med. 2017 Sep;15(5):419-426.
    \15\ Tai-Seale M, Olson CW, Li J, Chan AS, Morikawa C, Durbin M, 
Wang W, Luft HS. Electronic Health Record Logs Indicate That 
Physicians Split Time Evenly Between Seeing Patients And Desktop 
Medicine. Health Aff (Millwood). 2017 Apr 1;36(4):655-662.A.
    \16\ Robert Wachter. The Digital Doctor. New York, NY: McGraw-
Hill Education; 2015.
---------------------------------------------------------------------------

    With qualitative assessments being a key part of assessing 
usability, subjective user assessments are complementary to 
quantitative measures, such as time to perform tasks. Information about 
the source of reviews, such as if a reviewer has actually used the 
system, their setting, specialty, and background (e.g., as a clinician, 
practice manager, etc.), may affect the value of the reviews to health 
IT acquisition decision makers. As noted in the EHR Compare Report, 
resources exist that provide user reviews, though these may be 
outdated.
    Questions:
     How can the usability results currently available in the 
CHPL best be used to assist in comparisons between certified health IT 
products?
     Describe the availability and feasibility of common 
frameworks or standard scores from established usability assessment 
tools that would allow acquisition decision makers to compare usability 
of systems.
     Discuss the merits and risks of seeking a common set of 
measures for the purpose of real world testing that health IT 
developers could use to compare usability of systems. What specific 
types of data from current users would reflect how well the certified 
health IT product:
    [cir] Supports the cognitive work of clinical users (e.g., displays 
relevant information in useful formats at relevant points in workflow)?
    [cir] Reflects the ability of implementers to make customization 
and

[[Page 42917]]

implementation decisions in a user-centered manner?
     What usability assessment data, if available, are less 
resource intensive than traditional measures (e.g., time motion 
studies)?
     Comment on the feasibility and applicability of usability 
measures created from audit log data. How would health IT acquisition 
decision makers use this information to improve their system 
acquisition, upgrade, and customization decisions to best support end 
users' needs?
    [cir] Who should report audit log data and by what mechanism?
     How feasible would it be to implement usage monitoring 
tools (e.g., for time spent on specific tasks)?
    Interoperability:
    The Cures Act defines interoperability as: ``(A) enables the secure 
exchange of electronic health information with, and use of electronic 
health information from, other health information technology without 
special effort on the part of the user; (B) allows for complete access, 
exchange, and use of all electronically accessible health information 
for authorized use under applicable State or Federal law; and (C) does 
not constitute information blocking.''
    The EHR Compare Report identified product integration as a 
potential means to assess interoperability and proposed federal and 
private sector strategies to address it. The National Quality Forum's 
Measurement Framework to Assess Nationwide Progress Related to 
Interoperable Health Information Exchange to Support the National 
Quality Strategy also specified various domains of interoperability 
that might be useful to measure per the health IT consumers' 
perspective.\17\ Applicable domains include the exchange of electronic 
health information (referring to the availability of electronic health 
information, method of exchange, and quality of data content), and the 
usability of the exchanged electronic health information (referring to 
issues related to relevance, accessibility and comprehensibility of the 
information that is exchanged).
---------------------------------------------------------------------------

    \17\ https://www.qualityforum.org/Publications/2017/09/Interoperability_2016-2017_Final_Report.aspx.
---------------------------------------------------------------------------

    Two existing data sources with many Medicare providers are the 
Inpatient Hospital Promoting Interoperability Program and the Merit-
based Incentive Payment System (MIPS), which include measures related 
to health information exchange and interoperability.\18\ The exchange-
related measures within this domain primarily focus on a summary of 
care record exchange (e.g., send and receipt/acceptance transaction) 
and clinical information reconciliation. The measures would provide 
insights on health IT product performance related to summary of care 
record exchange. For additional information, we encourage reviewers to 
reference the 2019 IPPS Final Rule and the 2019 PFS which includes the 
Quality Payment Program (QPP) NPRM for proposed changes that may impact 
what information is submitted to Medicare.
---------------------------------------------------------------------------

    \18\ Quality Payment Program. https://qpp.cms.gov/mips/advancing-care-information.
---------------------------------------------------------------------------

    Industry reports (such as those listed in Appendix 3 of the EHR 
Compare Report), which typically involve provider surveys, serve as 
another data source to assess interoperability. However, the 
applicability of these reports may be limited to larger providers and 
health systems and may not necessarily reflect the experiences or needs 
of small practices and all settings (e.g., behavioral health). In 
addition, industry reports may not be affordable to all users.
    Questions:
     Please comment on the usefulness of product integration as 
a primary means of assessing interoperability (as proposed in the EHR 
Compare Report).
     What other domains of interoperability (beyond those 
already identified and referenced above) would be useful for 
comparative purposes?
     Of the data sources described in this RFI, which data 
sources would be useful for measuring the interoperability performance 
of certified health IT products?
    [cir] Comment on whether State Medicaid agencies would be able to 
share detailed attestation-level data for the purpose of developing 
reports at a more detailed level, such as by health IT product. If so, 
how would this information be useful to compare performance on 
interoperability across health IT products?
    [cir] How helpful would CMS program data (e.g., Quality Payment 
Program MIPS Promoting Interoperability Category, Inpatient Hospital 
Promoting Interoperability Program, Medicaid Promoting Interoperability 
Programs) related to exchange and interoperability be for comparative 
purposes? What measures should be selected for this purpose? Given that 
some of these data may be reported across providers rather than at the 
individual clinical level, how would this affect reporting of 
performance by health IT product?
     What other data sources and measures could be used to 
compare performance on interoperability across certified health IT 
products?
    Conformance to Certification Testing:
    Health IT that has been submitted by a health IT developer for 
certification and successfully tested and certified is listed in the 
CHPL,\19\ which is an online, openly available resource. This data 
ranges from the user-centered design and transparency disclosures made 
by health IT developers to the certification criteria to which health 
IT has been certified. However, user experiences, product performance, 
and interoperability-oriented metrics from health IT developers and 
healthcare providers are not reported in a consistent way across all 
products certified through the ONC Health IT Certification Program.
---------------------------------------------------------------------------

    \19\ Certified Health IT Product List. https://chpl.healthit.gov/#/search.
---------------------------------------------------------------------------

    ONC-prepared materials to support certification testing, such as 
the 2015 Edition Test Method, are intended to be read and understood 
with the express purpose of evaluating the health IT's functional 
capabilities that have been submitted for certification in a controlled 
environment, but are not determinative of the full scope of the health 
IT's capabilities, such as in a production environment,\20\ 
Nevertheless, testing results for health IT are available on the CHPL, 
and much of it in a structured format that makes the reported test 
results accessible for analysis and comparison.
---------------------------------------------------------------------------

    \20\ 2015 Edition Test Method. https://beta.healthit.gov/topic/certification-ehrs/2015-edition-test-method.
---------------------------------------------------------------------------

    As part of maintaining certification and ensuring ongoing 
conformance to certification testing, ONC-Authorized Certification 
Bodies (ONC-ACB) perform surveillance of health IT products and verify 
that the certified health IT also conforms in the production 
environment (i.e., ``in the field''). This includes reviewing any 
complaints about or potential issues with certified health IT products 
brought to their attention (i.e., reactive surveillance). They may also 
elect to conduct randomized surveillance.\21\ \22\
---------------------------------------------------------------------------

    \21\ Program Guidance #17-02: ONC Exercises Enforcement 
Discretion With Respect to Implementation of Randomized 
Surveillance. Retrieved from https://www.healthit.gov/sites/default/files/ONC_Enforcement_Discretion_Randomized_Surveillance_8-30-17.pdf.
    \22\ Certified Health IT Product List (CHPL) Public User Guide. 
Retrieved from https://www.healthit.gov/sites/default/files/policy/chpl_public_user_guide.pdf.
---------------------------------------------------------------------------

    If a certified health IT product does not demonstrate the 
functionality required by its certification, the certified health IT 
product is considered non-conforming and then listed and detailed as 
non-conforming on the CHPL.\23\ A list

[[Page 42918]]

of banned developers \24\ is maintained on the CHPL in addition to a 
list of decertified health IT \25\ where certification was withdrawn by 
the developer's ONC-ACB, by the developer under surveillance/review, or 
terminated by ONC. The available surveillance information about non-
conformant health IT and developers provides an avenue that can help 
potential consumers evaluate and compare how certified health IT 
performs in real-world settings. In addition, developers must post 
mandatory disclosures on types of additional costs and limitations for 
their certified health IT as part of the ONC Health IT Certification 
Program requirements.
---------------------------------------------------------------------------

    \23\ Products: Corrective Action Status. https://chpl.healthit.gov/#/collections/correctiveAction.
    \24\ Developers Under Certification Ban. https://chpl.healthit.gov/#/collections/developers.
    \25\ Decertified Products. https://chpl.healthit.gov/#/collections/products.
---------------------------------------------------------------------------

    Question:
     What additional information about certified health IT's 
conformance to the certification testing (beyond what is currently 
available on the CHPL) would be useful for comparison purposes? What 
mechanisms or approaches could be considered to obtain such data? What 
barriers might exist for developers and/or end users in reporting on 
such data?
    Other Categories for Consideration:
    The Cures Act lists other possible categories for the EHR Reporting 
Program related to certified health IT product performance, including:
     Enabling the user to order and view the results of 
laboratory tests, imaging tests, and other diagnostic tests;
     Submitting, editing, and retrieving data from registries, 
such as clinician-led clinical data registries;
     Accessing and exchanging information and data from and 
through health information exchanges;
     Accessing and exchanging information and data from medical 
devices;
     Accessing and exchanging information and data held by 
Federal, State, and local agencies and other applicable entities useful 
to a health care provider or other applicable user in the furtherance 
of patient care;
     Accessing and exchanging information from other health 
care providers or applicable users;
     Accessing and exchanging patient generated information;
     Providing the patient or an authorized designee with a 
complete copy of their health information from an electronic health 
record in a computable format; and
     Providing accurate patient information for the correct 
patient, including exchanging such information, and avoiding the 
duplication of patients records.
    Questions:
     How should the above categories be prioritized for 
inclusion/exclusion in the EHR Reporting Program, and why? What other 
criteria would be helpful for comparative purposes to best support end 
users' needs (e.g., to inform health IT acquisition, upgrade, and 
implementation decisions)?
     What data sources could be used to compare performance on 
these categories across certified health IT products?
     Please comment on different types of information, or 
measures, in this area that would be useful to acquisition, upgrade, 
and customization decisions in the ambulatory setting as opposed to 
inpatient settings?
    In addition to the other categories listed in the Cures Act, the 
EHR Compare Report identified gaps in information related to the 
domains of cost transparency, quality metrics, and population health. 
The cost transparency domain includes base, subscription, and 
transaction costs, as well as peer reviews regarding price 
expectations, which could also allow acquisition decision makers to 
make more informed acquisition, upgrade, and customization decisions to 
best support end users' needs.
    Questions:
     Please comment on the usefulness and feasibility of 
including criteria on quality reporting and population health in the 
EHR Reporting Program. What criteria should be considered to assess 
health IT performance in generating quality measures, reporting quality 
measures, and the functions required for supporting population health 
analytics (e.g., bulk data export)?
     What data sources, if any, are available to assess 
certified health IT product capabilities and performance in collecting, 
generating, and reporting on quality measures, and the ability to 
export multiple records for population health analytics? Are these data 
sources publicly available?
     Please comment on other categories, if any, besides those 
listed in this RFI that should be considered to be included in the EHR 
Reporting Program. Why should these be included, and what data sources 
exist to report on performance for the suggested categories?
    Hospitals and Health Systems:
    The focus of this RFI is on the information needs of health IT end 
users in ambulatory and small practice settings, as these groups report 
challenges accessing relevant information at affordable costs to help 
them compare certified health IT. However, ONC is aware that there are 
also gaps in the availability of information that hospitals and health 
systems need.
    Questions:
     Please describe the types of comparative information about 
certified health IT hospitals and health systems currently use (e.g., 
to inform health IT acquisition, upgrade, and customization decisions). 
What are the sources of this information? What information would be 
useful but is currently unavailable?
     What types of comparative information about certified 
health IT, if any, are specifically useful to hospitals and health 
systems, as opposed to ambulatory or small practices? What types of 
information could be collected or reported that would be helpful to 
both hospitals and health systems and to ambulatory and smaller 
providers?
     Please comment on how an EHR Reporting Program could best 
reflect the information needed for hospitals and health systems, 
ambulatory and smaller provider settings, and overlapping information 
in developing summary reports or comparison tools.

V. Collection of Information Requirements

    This document does not impose information collection requirements, 
that is, reporting, recordkeeping or third-party disclosure 
requirements. Consequently, there is no need for review by the Office 
of Management and Budget under the authority of the Paperwork Reduction 
Act of 1995 (44 U.S.C. 3501 et seq.).

VI. Response to Comments

    ONC typically receives a large public response to its published 
Federal Register documents. ONC will consider all comments received by 
the date and time specified in the DATES section of this document, but 
will not be able to acknowledge or respond individually to public 
comments.

    Dated: August 17, 2018.
Donald Rucker,
National Coordinator, Office of the National Coordinator for Health 
Information Technology.

[[Page 42919]]



 Appendix A--Certified Health IT Comparison Tools Identified Through ONC
                             Market Research
------------------------------------------------------------------------
       Comparison tool                      Company website
------------------------------------------------------------------------
4Med+ Marketplace............  www.4medapproved.com/wizard/marketplace.
AmericanEHR..................  www.americanehr.com.
Blackbook....................  www.blackbookrankings.com/healthcare.
California Healthcare          www.chcf.org/publications/2007/10/ehr-
 Foundation.                    selection-toolkit-for-community-health-
                                centers.
CHPL 4.0.....................  www.healthit.gov/chpl.
Consumer Affairs.............  www.consumeraffairs.com/emr-software.
EHR Compare..................  www.ehrcompare.com.
EHR in Practice..............  www.ehrinpractice.com/ehr-product-comparison.html.
Gartner......................  www.gartner.com.
HealthRecord.US..............  www.healthrecord.us.
IDC Health Insights..........  www.idc.com.
KLAS.........................  www.klasresearch.com.
LeadingAge...................  www.leadingage.org/ehr/search.aspx.
NCQA.........................  www.ncqa.org/Programs/Recognition/practices/PatientCenteredMedicalHomePCMH/PCMHPrevalidationProgram/VendorList.aspx.
Software Advice..............  www.softwareadvice.com.
Software Insider.............  www.ehr.softwareinsider.com.
Technology Advice............  www.technologyadvice.com/medical/ehr-emr/smart-advisor.
Texas Medical Association      www.texmed.org/EHRTool.
 (TMA).
------------------------------------------------------------------------

[FR Doc. 2018-18297 Filed 8-23-18; 8:45 am]
 BILLING CODE 4150-45-P


This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.