Request for Information on the Use of Clinical Algorithms That Have the Potential To Introduce Racial/Ethnic Bias Into Healthcare Delivery, 12948-12949 [2021-04509]

Download as PDF 12948 Federal Register / Vol. 86, No. 42 / Friday, March 5, 2021 / Notices Dated: March 1, 2021. Marquita Cullom, Associate Director. [FR Doc. 2021–04538 Filed 3–4–21; 8:45 am] BILLING CODE 4160–90–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Agency for Healthcare Research and Quality Request for Information on the Use of Clinical Algorithms That Have the Potential To Introduce Racial/Ethnic Bias Into Healthcare Delivery Agency for Healthcare Research and Quality (AHRQ), HHS. ACTION: Notice of Request for Information. AGENCY: The Agency for Healthcare Research and Quality (AHRQ) is seeking information from the public on clinical algorithms that are used or recommended in medical practice and any evidence on clinical algorithms that may introduce bias into clinical decision- making and/or influence access to care, quality of care, or health outcomes for racial and ethnic minorities and those who are socioeconomically disadvantaged. DATES: Comments must be submitted on or before May 4, 2021. The EPC Program will not respond individually to responders but will consider all comments submitted by the deadline. ADDRESSES: Submissions should follow the Submission Instructions below. We prefer that comments be submitted electronically on the submission website. Email submissions may also be sent to: epc@ahrq.gov FOR FURTHER INFORMATION CONTACT: Anjali Jain, Email: Anjali.Jain@ ahrq.hhs.gov. SUMMARY: The Agency for Healthcare Research and Quality (AHRQ) is seeking information from the public on clinical algorithms that are used or recommended in medical practice and any evidence on clinical algorithms that may introduce bias into clinical decision-making and/ or influence access to care, quality of care, or health outcomes for racial and ethnic minorities and those who are socioeconomically disadvantaged. Information received in response to this request will be used to inform an AHRQ Evidence-Based Practice Center Program (EPC) evidence review and may inform other activities commissioned by or in collaboration with AHRQ. Established in 1997, the mission of the jbell on DSKJLSW7X2PROD with NOTICES SUPPLEMENTARY INFORMATION: VerDate Sep<11>2014 20:30 Mar 04, 2021 Jkt 253001 EPC Program (https:// effectivehealthcare.ahrq.gov/about/epc) is to create evidence reviews that improve healthcare by supporting evidence-based decision-making by patients, providers, and policymakers. Evidence reviews summarize and synthesize existing literature and evidence using rigorous methods. AHRQ is conducting this review pursuant to sections 902 and 901(c) of the Public Health Service Act, 42 U.S.C. 299a and 42 U.S.C. 299(c). AHRQ intends to commission an evidence review that will critically appraise the evidence on commonly used algorithms, including whether race/ethnicity is included as an explicit variable, and how algorithms have been developed and validated. The review would examine how race/ethnicity and related variables included in clinical algorithms impact healthcare use, patient outcomes and healthcare disparities. In addition, the review will identify and assess other variables with the potential to introduce bias such as prior utilization. The review will identify and review approaches to clinical algorithm development that avoid the introduction of racial and ethnic bias into clinical decision making and resulting outcomes. For the purposes of this evidence review, clinical algorithms are defined as a set of steps that clinicians use to guide decision-making in preventive services (such as screening), in diagnosis, clinical management, or otherwise assessing or improving a patient’s health. Algorithms are informed by data and research evidence and may include patient-specific factors or characteristics which may be sociodemographic factors such as race/ ethnicity, physiologic factors such as, for example, blood sugar level, or others such as patterns of healthcare utilization. When used appropriately, algorithms can improve disease management and patient health by creating efficiencies in place of individuals having to weigh multiple and complex factors when making a clinical judgement. As a result, the use of clinical algorithms has become widespread in healthcare and includes a heterogeneous set of tools including clinical pathways/guidelines, the establishment of norms and standards that may vary according to patient-specific factors, clinical decision support embedded in electronic health records (EHRs) or within medical devices, pattern recognition software used for diagnosis, and apps and calculators that predict patient risk and prognosis. Some clinical algorithms include information about a patient’s PO 00000 Frm 00049 Fmt 4703 Sfmt 4703 race or ethnicity among its inputs and thus lead clinicians to decision-making that varies by race/ethnicity, including decisions about how best to diagnose and manage individual patients. The purpose of this evidence review is to understand which algorithms are currently used in different clinical settings; the type and extent of their validation; their potential for bias with impact on access, quality, and outcomes of care; awareness among clinicians of these issues; and strategies for developing and testing clinical algorithms to assure that they are free of bias in order to inform the scope of a future evidence review. We are interested in understanding which algorithms are currently in use in clinical practice including those related to the use of clinical preventive services. How many include race/ ethnicity and other factors that could lead to bias within the algorithm? We are interested in all algorithms including clinical pathways/guidelines, norms and standards (including laboratory values) that vary according to patient-specific factors such as race/ ethnicity and related variables, clinical decision support embedded in EHRs, pattern recognition software, and apps and calculators for patient risk and prognosis. We are interested both in algorithms developed through traditional methods and through new and ongoing methods including machine learning and artificial intelligence. AHRQ seeks information • From healthcare providers who use clinical algorithms to screen, diagnose, triage, treat or otherwise care for patients • From laboratorians or technicians who use algorithms to interpret lab or radiology data • From researchers and clinical decision support developers who develop algorithms used in healthcare for patients • From clinical professional societies or other groups who develop clinical algorithms for healthcare • From payers who use clinical algorithms to guide payment decisions for care for patients • From healthcare delivery organizations who use clinical algorithms to determine healthcare practices and policies for patients • From device developers who incorporate algorithms into device software to interpret data and set standards • From patients whose healthcare and healthcare decisions may be informed by clinical algorithms E:\FR\FM\05MRN1.SGM 05MRN1 jbell on DSKJLSW7X2PROD with NOTICES Federal Register / Vol. 86, No. 42 / Friday, March 5, 2021 / Notices Specific questions of interest to the AHRQ include, but are not limited to, the following: 1. What clinical algorithms are used in clinical practice, hospitals, health systems, payment systems, or other instances? What is the estimated impact of these algorithms in size and characteristics of population affected, quality of care, clinical outcomes, quality of life and health disparities? 2. Do the algorithms in question 1 include race/ethnicity as a variable and, if so, how was race and ethnicity defined (including from whose perspective and whether there is a designation for mixed race or multiracial individuals)? 3. Do the algorithms in question 1 include measures of social determinants of health (SDOH) and, if so, how were these defined? Are these independently or collectively examined for their potential contribution to healthcare disparities and biases in care? 4. For the algorithms in question 1, what evidence, data quality and types (such as claims/utilization data, clinical data, social determinants of health), and data sources were used in their development and validation? What is the sample size of the datasets used for development and validation? What is the representation of Black, Indigenous, and People of Color (BIPOC) and what is the power to detect between-group differences? What methods were used to validate the algorithms and measure health outcomes associated with the use of the algorithms? 5. For the algorithms in question 1, what approaches are used in updating these algorithms? 6. Which clinical algorithms have evidence that they contribute to healthcare disparities, including decreasing access to care, quality of care or worsening health outcomes for BIPOC? What are the priority populations or conditions for assessing whether algorithms increase racial/ ethnic disparities? What are the mechanisms by which use of algorithms contribute to poor care for BIPOC? 7. To what extent are users of algorithms including clinicians, health systems, and health plans aware of the inclusion of race/ethnicity or other variables that could introduce bias in these algorithms and the implications for clinical decision making? What evidence is available about the degree to which the use of clinical algorithms contributes to bias in care delivery and resulting disparities in health outcomes? To what extent are patients aware of the inclusion of race/ethnicity or other variables that can result in bias in algorithms that influence their care? Do VerDate Sep<11>2014 20:30 Mar 04, 2021 Jkt 253001 providers or health systems communicate this information with patients in ways that can be understood? 8. What are approaches to identifying sources of bias and/or correcting or developing new algorithms that may be free of bias? What evidence, data quality and types (such as claims/utilization data, clinical data, information on social determinants of health), and data sources and sample size are used in their development and validation? What is the impact of these new approaches and algorithms on outcomes? 9. What challenges have arisen or can arise by designing algorithms developed using traditional biomedical or physiologic factors (such as blood glucose) yet include race/ethnicity as a proxy for other factors such as specific biomarkers, genetic information, etc.? What strategies can be used to address these challenges? 10. What are existing and developing standards (national and international) about how clinical algorithms should be developed, validated, and updated in a way to avoid bias? Are you aware of guidance on the inclusion or race/ ethnicity, related variables such as SDOH, prior utilization, or other variables to minimize the risk of bias? 11. To what extent are users of clinical algorithms educated about how algorithms are developed or may influence their decision-making? What educational curricula and training is available for clinicians that addresses bias in clinical algorithms? AHRQ is interested in all of the questions listed above, but respondents are welcome to address as many or as few as they choose and to address additional areas of interest not listed. This RFI is for planning purposes only and should not be construed as a policy, solicitation for applications, or as an obligation on the part of the Government to provide support for any ideas identified in response to it. AHRQ will use the information submitted in response to this RFI at its discretion and will not provide comments to any responder’s submission. However, responses to the RFI may be reflected in future solicitation(s) or policies. The information provided will be analyzed and may appear in reports. Respondents will not be identified in any published reports. Respondents are advised that the Government is under no obligation to acknowledge receipt of the information received or provide feedback to respondents with respect to any information submitted. No proprietary, classified, confidential, or sensitive information should be included in your response. The contents PO 00000 Frm 00050 Fmt 4703 Sfmt 4703 12949 of all submissions will be made available to the public upon request. Materials submitted must be publicly available or can be made public. Dated: March 1, 2021. Marquita Cullom, Associate Director. [FR Doc. 2021–04509 Filed 3–4–21; 8:45 am] BILLING CODE 4160–90–P DEPARTMENT OF HEALTH AND HUMAN SERVICES Centers for Disease Control and Prevention [Docket No. CDC–2020–0011] Draft Infection Control in Healthcare Personnel: Epidemiology and Control of Selected Infections Transmitted Among Healthcare Personnel and Patients: Diphtheria, Group A Streptococcus, Meningococcal Disease, and Pertussis Sections; ReOpening of Comment Period Centers for Disease Control and Prevention (CDC), Department of Health and Human Services (DHHS). ACTION: Notice with comment. AGENCY: The Centers for Disease Control and Prevention (CDC), in the Department of Health and Human Services (DHHS), announces the reopening of a docket to obtain a public comment on the DRAFT Infection Control in Healthcare Personnel: Epidemiology and Control of Selected Infections Transmitted Among Healthcare Personnel and Patients: Diphtheria, Group A Streptococcus, Meningococcal Disease, and Pertussis Sections (‘‘Draft Guideline’’). DATES: Written comments must be received on or before May 4, 2021. ADDRESSES: You may submit comments, identified by Docket No. CDC–2020– 0011, by any of the following methods: • Federal eRulemaking Portal: http:// www.regulations.gov. Follow the instructions for submitting comments. • Mail: Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, Attn: Docket No. CDC– 2020–0011, Infection Prevention and Control Guidelines, 1600 Clifton Rd. NE, Mailstop H16–2, Atlanta, Georgia, 30329. Instructions: All submissions received must include the agency name and Docket Number. All relevant comments received will be posted without change to http://regulations.gov, including any personal information provided. For SUMMARY: E:\FR\FM\05MRN1.SGM 05MRN1

Agencies

[Federal Register Volume 86, Number 42 (Friday, March 5, 2021)]
[Notices]
[Pages 12948-12949]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2021-04509]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Agency for Healthcare Research and Quality


Request for Information on the Use of Clinical Algorithms That 
Have the Potential To Introduce Racial/Ethnic Bias Into Healthcare 
Delivery

AGENCY: Agency for Healthcare Research and Quality (AHRQ), HHS.

ACTION: Notice of Request for Information.

-----------------------------------------------------------------------

SUMMARY: The Agency for Healthcare Research and Quality (AHRQ) is 
seeking information from the public on clinical algorithms that are 
used or recommended in medical practice and any evidence on clinical 
algorithms that may introduce bias into clinical decision- making and/
or influence access to care, quality of care, or health outcomes for 
racial and ethnic minorities and those who are socioeconomically 
disadvantaged.

DATES: Comments must be submitted on or before May 4, 2021. The EPC 
Program will not respond individually to responders but will consider 
all comments submitted by the deadline.

ADDRESSES: Submissions should follow the Submission Instructions below. 
We prefer that comments be submitted electronically on the submission 
website. Email submissions may also be sent to: [email protected]

FOR FURTHER INFORMATION CONTACT: Anjali Jain, Email: 
[email protected].

SUPPLEMENTARY INFORMATION: The Agency for Healthcare Research and 
Quality (AHRQ) is seeking information from the public on clinical 
algorithms that are used or recommended in medical practice and any 
evidence on clinical algorithms that may introduce bias into clinical 
decision-making and/or influence access to care, quality of care, or 
health outcomes for racial and ethnic minorities and those who are 
socioeconomically disadvantaged.
    Information received in response to this request will be used to 
inform an AHRQ Evidence-Based Practice Center Program (EPC) evidence 
review and may inform other activities commissioned by or in 
collaboration with AHRQ. Established in 1997, the mission of the EPC 
Program (https://effectivehealthcare.ahrq.gov/about/epc) is to create 
evidence reviews that improve healthcare by supporting evidence-based 
decision-making by patients, providers, and policymakers. Evidence 
reviews summarize and synthesize existing literature and evidence using 
rigorous methods. AHRQ is conducting this review pursuant to sections 
902 and 901(c) of the Public Health Service Act, 42 U.S.C. 299a and 42 
U.S.C. 299(c).
    AHRQ intends to commission an evidence review that will critically 
appraise the evidence on commonly used algorithms, including whether 
race/ethnicity is included as an explicit variable, and how algorithms 
have been developed and validated. The review would examine how race/
ethnicity and related variables included in clinical algorithms impact 
healthcare use, patient outcomes and healthcare disparities. In 
addition, the review will identify and assess other variables with the 
potential to introduce bias such as prior utilization. The review will 
identify and review approaches to clinical algorithm development that 
avoid the introduction of racial and ethnic bias into clinical decision 
making and resulting outcomes.
    For the purposes of this evidence review, clinical algorithms are 
defined as a set of steps that clinicians use to guide decision-making 
in preventive services (such as screening), in diagnosis, clinical 
management, or otherwise assessing or improving a patient's health. 
Algorithms are informed by data and research evidence and may include 
patient-specific factors or characteristics which may be 
sociodemographic factors such as race/ethnicity, physiologic factors 
such as, for example, blood sugar level, or others such as patterns of 
healthcare utilization.
    When used appropriately, algorithms can improve disease management 
and patient health by creating efficiencies in place of individuals 
having to weigh multiple and complex factors when making a clinical 
judgement. As a result, the use of clinical algorithms has become 
widespread in healthcare and includes a heterogeneous set of tools 
including clinical pathways/guidelines, the establishment of norms and 
standards that may vary according to patient-specific factors, clinical 
decision support embedded in electronic health records (EHRs) or within 
medical devices, pattern recognition software used for diagnosis, and 
apps and calculators that predict patient risk and prognosis. Some 
clinical algorithms include information about a patient's race or 
ethnicity among its inputs and thus lead clinicians to decision-making 
that varies by race/ethnicity, including decisions about how best to 
diagnose and manage individual patients.
    The purpose of this evidence review is to understand which 
algorithms are currently used in different clinical settings; the type 
and extent of their validation; their potential for bias with impact on 
access, quality, and outcomes of care; awareness among clinicians of 
these issues; and strategies for developing and testing clinical 
algorithms to assure that they are free of bias in order to inform the 
scope of a future evidence review. We are interested in understanding 
which algorithms are currently in use in clinical practice including 
those related to the use of clinical preventive services. How many 
include race/ethnicity and other factors that could lead to bias within 
the algorithm? We are interested in all algorithms including clinical 
pathways/guidelines, norms and standards (including laboratory values) 
that vary according to patient-specific factors such as race/ethnicity 
and related variables, clinical decision support embedded in EHRs, 
pattern recognition software, and apps and calculators for patient risk 
and prognosis. We are interested both in algorithms developed through 
traditional methods and through new and ongoing methods including 
machine learning and artificial intelligence. AHRQ seeks information
     From healthcare providers who use clinical algorithms to 
screen, diagnose, triage, treat or otherwise care for patients
     From laboratorians or technicians who use algorithms to 
interpret lab or radiology data
     From researchers and clinical decision support developers 
who develop algorithms used in healthcare for patients
     From clinical professional societies or other groups who 
develop clinical algorithms for healthcare
     From payers who use clinical algorithms to guide payment 
decisions for care for patients
     From healthcare delivery organizations who use clinical 
algorithms to determine healthcare practices and policies for patients
     From device developers who incorporate algorithms into 
device software to interpret data and set standards
     From patients whose healthcare and healthcare decisions 
may be informed by clinical algorithms

[[Page 12949]]

    Specific questions of interest to the AHRQ include, but are not 
limited to, the following:
    1. What clinical algorithms are used in clinical practice, 
hospitals, health systems, payment systems, or other instances? What is 
the estimated impact of these algorithms in size and characteristics of 
population affected, quality of care, clinical outcomes, quality of 
life and health disparities?
    2. Do the algorithms in question 1 include race/ethnicity as a 
variable and, if so, how was race and ethnicity defined (including from 
whose perspective and whether there is a designation for mixed race or 
multiracial individuals)?
    3. Do the algorithms in question 1 include measures of social 
determinants of health (SDOH) and, if so, how were these defined? Are 
these independently or collectively examined for their potential 
contribution to healthcare disparities and biases in care?
    4. For the algorithms in question 1, what evidence, data quality 
and types (such as claims/utilization data, clinical data, social 
determinants of health), and data sources were used in their 
development and validation? What is the sample size of the datasets 
used for development and validation? What is the representation of 
Black, Indigenous, and People of Color (BIPOC) and what is the power to 
detect between-group differences? What methods were used to validate 
the algorithms and measure health outcomes associated with the use of 
the algorithms?
    5. For the algorithms in question 1, what approaches are used in 
updating these algorithms?
    6. Which clinical algorithms have evidence that they contribute to 
healthcare disparities, including decreasing access to care, quality of 
care or worsening health outcomes for BIPOC? What are the priority 
populations or conditions for assessing whether algorithms increase 
racial/ethnic disparities? What are the mechanisms by which use of 
algorithms contribute to poor care for BIPOC?
    7. To what extent are users of algorithms including clinicians, 
health systems, and health plans aware of the inclusion of race/
ethnicity or other variables that could introduce bias in these 
algorithms and the implications for clinical decision making? What 
evidence is available about the degree to which the use of clinical 
algorithms contributes to bias in care delivery and resulting 
disparities in health outcomes? To what extent are patients aware of 
the inclusion of race/ethnicity or other variables that can result in 
bias in algorithms that influence their care? Do providers or health 
systems communicate this information with patients in ways that can be 
understood?
    8. What are approaches to identifying sources of bias and/or 
correcting or developing new algorithms that may be free of bias? What 
evidence, data quality and types (such as claims/utilization data, 
clinical data, information on social determinants of health), and data 
sources and sample size are used in their development and validation? 
What is the impact of these new approaches and algorithms on outcomes?
    9. What challenges have arisen or can arise by designing algorithms 
developed using traditional biomedical or physiologic factors (such as 
blood glucose) yet include race/ethnicity as a proxy for other factors 
such as specific biomarkers, genetic information, etc.? What strategies 
can be used to address these challenges?
    10. What are existing and developing standards (national and 
international) about how clinical algorithms should be developed, 
validated, and updated in a way to avoid bias? Are you aware of 
guidance on the inclusion or race/ethnicity, related variables such as 
SDOH, prior utilization, or other variables to minimize the risk of 
bias?
    11. To what extent are users of clinical algorithms educated about 
how algorithms are developed or may influence their decision-making? 
What educational curricula and training is available for clinicians 
that addresses bias in clinical algorithms?
    AHRQ is interested in all of the questions listed above, but 
respondents are welcome to address as many or as few as they choose and 
to address additional areas of interest not listed.
    This RFI is for planning purposes only and should not be construed 
as a policy, solicitation for applications, or as an obligation on the 
part of the Government to provide support for any ideas identified in 
response to it. AHRQ will use the information submitted in response to 
this RFI at its discretion and will not provide comments to any 
responder's submission. However, responses to the RFI may be reflected 
in future solicitation(s) or policies. The information provided will be 
analyzed and may appear in reports. Respondents will not be identified 
in any published reports. Respondents are advised that the Government 
is under no obligation to acknowledge receipt of the information 
received or provide feedback to respondents with respect to any 
information submitted. No proprietary, classified, confidential, or 
sensitive information should be included in your response. The contents 
of all submissions will be made available to the public upon request. 
Materials submitted must be publicly available or can be made public.

    Dated: March 1, 2021.
Marquita Cullom,
Associate Director.
[FR Doc. 2021-04509 Filed 3-4-21; 8:45 am]
BILLING CODE 4160-90-P