Request for Information: Responsible Procurement of Artificial Intelligence in Government, 22196-22197 [2024-06547]

Download as PDF 22196 Federal Register / Vol. 89, No. 62 / Friday, March 29, 2024 / Notices categories alone, and then combines all other respondents reporting multiple race and/or ethnicity categories into an aggregated Multiracial and/or Multiethnic category. This approach will often obscure the specific racial and ethnic diversity of the population (e.g., over half of the population who identify as American Indian or Alaska Native and Native Hawaiian or Pacific Islander may be assigned to the Multiracial and/ or Multiethnic group). Therefore, Federal agencies should use this approach in conjunction with another approach (like Approaches 1 or 2) to comply with the requirement to report as much information on race and ethnicity as possible, including data for respondents who reported more than one race and/or ethnicity category. The percentages in this approach will sum to 100 percent because the response categories are mutually exclusive. The following illustrates the tabulation categories used for this approach: • American Indian or Alaska Native alone • Asian alone • Black or African American alone • Hispanic or Latino alone • Middle Eastern or North African alone • Native Hawaiian or Pacific Islander alone • White alone • Multiracial and/or Multiethnic With respect to tabulation and presentation, regardless of approach, the seven minimum race and ethnicity categories shall be treated co-equally except if a program or collection effort focuses on a specific racial or ethnic group, and as approved by OIRA. When tabulating and presenting data, agencies must use a consistent approach across all categories within a single table. If categories must be combined in order to reach sample size thresholds for reporting, those combinations should be labeled with the list of combined categories rather than with ‘‘other.’’ 5. Use of the Standards for Record Keeping and Reporting khammond on DSKJM1Z7X2PROD with NOTICES a. Statistical Reporting These standards shall be used for all Federally sponsored statistical data collections that include data on race and ethnicity. Any variation must be specifically authorized by OIRA through the PRA information collection approval process. In those cases where the data collection is not subject to the information collection clearance process, a direct request for a variance must be made to OIRA. VerDate Sep<11>2014 16:49 Mar 28, 2024 Jkt 262001 b. General Program Administrative and Grant Reporting public through their websites at the time of submission to OMB. These standards shall be used for all Federal administrative reporting or record keeping requirements that include data on race and ethnicity. Agencies that cannot follow these standards must request a variance from OIRA. Variances will be considered if the agency can demonstrate that it is not reasonable for the primary reporter to determine race and ethnicity in terms of the specified minimum categories, or that the specific program is directed to only one or a limited number of races and ethnicities. [FR Doc. 2024–06469 Filed 3–28–24; 8:45 am] c. Civil Rights and Other Compliance Reporting These standards must be used by all Federal agencies for civil rights and other compliance reporting from the public and private sectors and all levels of government. Any variation requiring less detailed data or data which cannot be aggregated into the minimum categories must be specifically approved by OIRA. 6. Effective Date The provisions of these standards are effective March 28, 2024 for all new record keeping or reporting requirements that include race and ethnicity data. All existing record keeping or reporting requirements should be made consistent with these standards through a non-substantive change request as soon as possible, or at the time they are submitted for extension or revision to OIRA under the PRA, but not later than March 28, 2029. Within 18 months of publication of these standards, the Chief Financial Officers Act Agencies and the U.S. Equal Employment Opportunity Commission 47 must submit to OMB, through their agency Statistical Officials and in coordination with their agency’s Chief Data Officer, Evaluation Officer,48 Senior Agency Officials for Privacy, and other agency officials as appropriate, an Action Plan on Race and Ethnicity Data describing how they intend to bring their agency collections and publications into compliance with these standards by March 28, 2029. Agencies must make these plans available to the 47 The U.S. Equal Employment Opportunity Commission does not currently have a Statistical Official and should submit their Action Plan through their Chief Data Officer. 48 These three agency officials make up the Data Governance Bodies established under OMB M–19– 23, Phase 1 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Learning Agendas, Personnel, and Planning Guidance (July 10, 2019), available at https://www.whitehouse.gov/ wp-content/uploads/2019/07/m-19-23.pdf. PO 00000 Frm 00081 Fmt 4703 Sfmt 4703 BILLING CODE 3110–01–P OFFICE OF MANAGEMENT AND BUDGET Request for Information: Responsible Procurement of Artificial Intelligence in Government Office of Management and Budget. ACTION: Request for information: responsible procurement of artificial intelligence in government. AGENCY: This request for information on the responsible procurement of artificial intelligence is being issued concurrently with the release of the OMB Memorandum titled Advancing Governance, Innovation, and Risk Management for Agency Use of Artificial Intelligence (the ‘‘AI Mmemo’’). Executive Order 14110, Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence, directed OMB within 180 days of the issuance of the AI M-memo to develop an initial means to ensure that agency contracts for the acquisition of AI systems and services align with the guidance provided in the AI M-memo and advance the other aims identified in the Advancing American AI Act (‘‘AI Act’’). DATES: Responses to this request for information will be accepted for consideration until April 29, 2024. ADDRESSES: Responses must be submitted electronically through regulations.gov. Mailed paper submissions will not be accepted, and electronic submissions received after the deadline may not be considered. Instructions: Federal eRulemaking Portal: Go to www.regulations.gov to submit your comments electronically. Information on how to use Regulations.gov, including instructions for accessing agency documents, submitting comments, and viewing the docket, is available on the site under ‘‘FAQ’’ (https://www.regulations.gov/ faq). Privacy Act Statement: OMB is issuing this request for information (RFI) pursuant to Executive Order 14110.1 Submission of comments in response to this RFI is voluntary. Comments may be used to inform sound decision-making on topics related to SUMMARY: 1 E.O. 14110, Executive Order on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. E:\FR\FM\29MRN1.SGM 29MRN1 khammond on DSKJM1Z7X2PROD with NOTICES Federal Register / Vol. 89, No. 62 / Friday, March 29, 2024 / Notices this RFI. Please note that submissions received in response to this notice may be posted in the Federal eRulemaking Portal at www.regulations.gov or otherwise released in their entirety, including any personal and business confidential information provided. Do not include in your submissions any information of a confidential nature, such as personal or proprietary information, or any information you would not like to be made publicly available. Comments and commenter information are maintained under the OMB Public Input System of Records, OMB/INPUT/01. The system of records notice accessible at 88 FR 20913 (https://www.federalregister.gov/ documents/2023/04/07/2023-07452/ privacy-act-of-1974-system-of-records) includes a list of routine uses associated with the collection of this information. Comments containing references, studies, research, and other empirical data that are not widely published should include electronic links to the referenced materials, if they are available online. Please note that the U.S. Government will not pay for response preparation, or for the use of any information contained in the response. A response to this RFI will not be viewed as a binding commitment to develop or pursue the project or ideas discussed. FOR FURTHER INFORMATION CONTACT: Please direct questions regarding this Notice to Samantha Hubner at OFCIO_ AI@OMB.eop.gov with ‘‘AI Procurement RFI’’ in the subject line, or by phone at 202–395–0379. SUPPLEMENTARY INFORMATION: Consistent with Section 7224(d)(1) of the AI Act, this ‘‘initial means’’ (see SUMMARY section) will at a minimum: —Address protection of privacy, civil rights, and civil liberties; —Address the ownership and security of data and other information created, used, processed, stored, maintained, disseminated, disclosed, or disposed of by a contractor or subcontractor on behalf of the Federal Government; —Include considerations for securing the training data, algorithms, and other components of any artificial intelligence system against misuse, unauthorized alteration, degradation, or rendering inoperable; and —Address any other issue or concern determined to be relevant by the Director to ensure appropriate use and protection of privacy and Government data and other information.2 2 AI Act, Section 7224 (d) https://uscode. house.gov/view.xhtml?req=(title:40%20section: 11301%20edition:prelim). VerDate Sep<11>2014 16:49 Mar 28, 2024 Jkt 262001 The Administration has undertaken numerous efforts to advance responsible AI innovation and secure protections for people’s rights and safety. OMB has issued this RFI to help inform its development of an initial means to ensure the responsible procurement of AI by Federal agencies. OMB is specifically asking for information on the questions posed below. However, this list is not intended to limit the scope of topics that may be addressed by submissions. Commenters are invited to provide feedback on any topic believed to have implications for the procurement of AI by Federal agencies. When responding to one or more of the questions below, please note in the text of your response the number of the question to which you are responding. Commenters should include a page number on each page of their submissions. Commenters are not required to respond to all questions, but OMB asks that comments be limited to no more than eight pages in length. Strengthening the AI Marketplace 1. How may standard practices and strategies of Federal procurement, such as Statements of Objectives, Quality Assurance Surveillance Plans, modular contracts, use of contract incentives, and teaming agreements,3 as well as innovative procurement practices, such as those in the Periodic Table of Acquisition Innovations,4 be best used to reflect emerging practices in AI procurement? Are there additional materials or resources that OMB could provide to vendors or agencies to improve alignment between agency missions and technical requirements? 2. How can OMB promote robust competition, attract new entrants, including small businesses, into the Federal marketplace, and avoid vendor lock-in across specific elements of the technology sector, including data collectors and labelers, model developers, infrastructure providers, and AI service providers? Are there ways OMB can address practices that limit competition, such as inappropriate tying, egress fees, and self-preferencing? 3. Should the Federal Government standardize assessments for the benefits and trade-offs between in-house AI development, contracted AI development, licensing of AI-enabled software, and use of AI-enabled services? If so, how? 4. How might metrics be developed and communicated to enable 3 October 2023, Report on Recommendations on Procurement from the National Artificial Intelligence Advisor Committee (NAIAC). 4 https://acquisitiongateway.gov/periodic-table. PO 00000 Frm 00082 Fmt 4703 Sfmt 4703 22197 performance-based procurement of AI? What questions should agencies be asking vendors to determine whether AI is already being used in performancebased services contracts? Managing the Performance and Risks of AI 5. What access to documentation, data, code, models, software, and other technical components might vendors provide to agencies to demonstrate compliance with the requirements established in the AI M-memo? What contract language would best effectuate this access, and is this best envisioned as a standard clause, or requirementsspecific elements in a statement of work? 6. Which elements of testing, evaluation, and impact assessments are best conducted by the vendor, and which responsibilities should remain with the agencies? 7. What if any terms should agencies include in contracts to protect the Federal Government’s rights and access to its data, while maintaining protection of a vendor’s intellectual property? 8. What if any terms, including terms governing information-sharing among agencies, vendors, and the public, should be included in contracts for AI systems or services to implement the AI M-memo’s provisions regarding notice and appeal (sections 5(c)(v)(D) and (E))? 9. How might agencies structure their procurements to reduce the risk that an AI system or service they acquire may produce harmful or illegal content, such as fraudulent or deceptive content, or content that includes child sex abuse material or non-consensual intimate imagery? 10. How might OMB ensure that agencies procure AI systems or services in a way that advances equitable outcomes and mitigates risks to privacy, civil rights, and civil liberties? David A. Myklegard, Deputy Federal Chief Information Officer. Christine J. Harada, Senior Advisor, Office of Federal Procurement Policy, Performing, by delegation, the duties of the Administrator for Federal Procurement Policy. [FR Doc. 2024–06547 Filed 3–28–24; 8:45 am] BILLING CODE 3110–01–P MILLENNIUM CHALLENGE CORPORATION [MCC FR 24–02] Notice of Open Meeting Millennium Challenge Corporation. AGENCY: E:\FR\FM\29MRN1.SGM 29MRN1

Agencies

Management and Budget Office

[Federal Register Volume 89, Number 62 (Friday, March 29, 2024)]
[Notices]
[Pages 22196-22197]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2024-06547]


-----------------------------------------------------------------------

OFFICE OF MANAGEMENT AND BUDGET


Request for Information: Responsible Procurement of Artificial 
Intelligence in Government

AGENCY: Office of Management and Budget.

ACTION: Request for information: responsible procurement of artificial 
intelligence in government.

-----------------------------------------------------------------------

SUMMARY: This request for information on the responsible procurement of 
artificial intelligence is being issued concurrently with the release 
of the OMB Memorandum titled Advancing Governance, Innovation, and Risk 
Management for Agency Use of Artificial Intelligence (the ``AI M-
memo''). Executive Order 14110, Safe, Secure, and Trustworthy 
Development and Use of Artificial Intelligence, directed OMB within 180 
days of the issuance of the AI M-memo to develop an initial means to 
ensure that agency contracts for the acquisition of AI systems and 
services align with the guidance provided in the AI M-memo and advance 
the other aims identified in the Advancing American AI Act (``AI 
Act'').

DATES: Responses to this request for information will be accepted for 
consideration until April 29, 2024.

ADDRESSES: Responses must be submitted electronically through 
regulations.gov. Mailed paper submissions will not be accepted, and 
electronic submissions received after the deadline may not be 
considered.
    Instructions: Federal eRulemaking Portal: Go to www.regulations.gov 
to submit your comments electronically. Information on how to use 
Regulations.gov, including instructions for accessing agency documents, 
submitting comments, and viewing the docket, is available on the site 
under ``FAQ'' (https://www.regulations.gov/faq).
    Privacy Act Statement: OMB is issuing this request for information 
(RFI) pursuant to Executive Order 14110.\1\ Submission of comments in 
response to this RFI is voluntary. Comments may be used to inform sound 
decision-making on topics related to

[[Page 22197]]

this RFI. Please note that submissions received in response to this 
notice may be posted in the Federal eRulemaking Portal at 
www.regulations.gov or otherwise released in their entirety, including 
any personal and business confidential information provided. Do not 
include in your submissions any information of a confidential nature, 
such as personal or proprietary information, or any information you 
would not like to be made publicly available. Comments and commenter 
information are maintained under the OMB Public Input System of 
Records, OMB/INPUT/01. The system of records notice accessible at 88 FR 
20913 (https://www.federalregister.gov/documents/2023/04/07/2023-07452/privacy-act-of-1974-system-of-records) includes a list of routine uses 
associated with the collection of this information.
---------------------------------------------------------------------------

    \1\ E.O. 14110, Executive Order on Safe, Secure, and Trustworthy 
Development and Use of Artificial Intelligence.
---------------------------------------------------------------------------

    Comments containing references, studies, research, and other 
empirical data that are not widely published should include electronic 
links to the referenced materials, if they are available online.
    Please note that the U.S. Government will not pay for response 
preparation, or for the use of any information contained in the 
response. A response to this RFI will not be viewed as a binding 
commitment to develop or pursue the project or ideas discussed.

FOR FURTHER INFORMATION CONTACT: Please direct questions regarding this 
Notice to Samantha Hubner at [email protected] with ``AI Procurement 
RFI'' in the subject line, or by phone at 202-395-0379.

SUPPLEMENTARY INFORMATION: Consistent with Section 7224(d)(1) of the AI 
Act, this ``initial means'' (see SUMMARY section) will at a minimum:

--Address protection of privacy, civil rights, and civil liberties;
--Address the ownership and security of data and other information 
created, used, processed, stored, maintained, disseminated, disclosed, 
or disposed of by a contractor or subcontractor on behalf of the 
Federal Government;
--Include considerations for securing the training data, algorithms, 
and other components of any artificial intelligence system against 
misuse, unauthorized alteration, degradation, or rendering inoperable; 
and
--Address any other issue or concern determined to be relevant by the 
Director to ensure appropriate use and protection of privacy and 
Government data and other information.\2\
---------------------------------------------------------------------------

    \2\ AI Act, Section 7224 (d) https://uscode.house.gov/view.xhtml?req=(title:40%20section:11301%20edition:prelim).

    The Administration has undertaken numerous efforts to advance 
responsible AI innovation and secure protections for people's rights 
and safety.
    OMB has issued this RFI to help inform its development of an 
initial means to ensure the responsible procurement of AI by Federal 
agencies. OMB is specifically asking for information on the questions 
posed below. However, this list is not intended to limit the scope of 
topics that may be addressed by submissions. Commenters are invited to 
provide feedback on any topic believed to have implications for the 
procurement of AI by Federal agencies.
    When responding to one or more of the questions below, please note 
in the text of your response the number of the question to which you 
are responding. Commenters should include a page number on each page of 
their submissions. Commenters are not required to respond to all 
questions, but OMB asks that comments be limited to no more than eight 
pages in length.

Strengthening the AI Marketplace

    1. How may standard practices and strategies of Federal 
procurement, such as Statements of Objectives, Quality Assurance 
Surveillance Plans, modular contracts, use of contract incentives, and 
teaming agreements,\3\ as well as innovative procurement practices, 
such as those in the Periodic Table of Acquisition Innovations,\4\ be 
best used to reflect emerging practices in AI procurement? Are there 
additional materials or resources that OMB could provide to vendors or 
agencies to improve alignment between agency missions and technical 
requirements?
---------------------------------------------------------------------------

    \3\ October 2023, Report on Recommendations on Procurement from 
the National Artificial Intelligence Advisor Committee (NAIAC).
    \4\ https://acquisitiongateway.gov/periodic-table.
---------------------------------------------------------------------------

    2. How can OMB promote robust competition, attract new entrants, 
including small businesses, into the Federal marketplace, and avoid 
vendor lock-in across specific elements of the technology sector, 
including data collectors and labelers, model developers, 
infrastructure providers, and AI service providers? Are there ways OMB 
can address practices that limit competition, such as inappropriate 
tying, egress fees, and self-preferencing?
    3. Should the Federal Government standardize assessments for the 
benefits and trade-offs between in-house AI development, contracted AI 
development, licensing of AI-enabled software, and use of AI-enabled 
services? If so, how?
    4. How might metrics be developed and communicated to enable 
performance-based procurement of AI? What questions should agencies be 
asking vendors to determine whether AI is already being used in 
performance-based services contracts?

Managing the Performance and Risks of AI

    5. What access to documentation, data, code, models, software, and 
other technical components might vendors provide to agencies to 
demonstrate compliance with the requirements established in the AI M-
memo? What contract language would best effectuate this access, and is 
this best envisioned as a standard clause, or requirements-specific 
elements in a statement of work?
    6. Which elements of testing, evaluation, and impact assessments 
are best conducted by the vendor, and which responsibilities should 
remain with the agencies?
    7. What if any terms should agencies include in contracts to 
protect the Federal Government's rights and access to its data, while 
maintaining protection of a vendor's intellectual property?
    8. What if any terms, including terms governing information-sharing 
among agencies, vendors, and the public, should be included in 
contracts for AI systems or services to implement the AI M-memo's 
provisions regarding notice and appeal (sections 5(c)(v)(D) and (E))?
    9. How might agencies structure their procurements to reduce the 
risk that an AI system or service they acquire may produce harmful or 
illegal content, such as fraudulent or deceptive content, or content 
that includes child sex abuse material or non-consensual intimate 
imagery?
    10. How might OMB ensure that agencies procure AI systems or 
services in a way that advances equitable outcomes and mitigates risks 
to privacy, civil rights, and civil liberties?

David A. Myklegard,
Deputy Federal Chief Information Officer.
Christine J. Harada,
Senior Advisor, Office of Federal Procurement Policy, Performing, by 
delegation, the duties of the Administrator for Federal Procurement 
Policy.
[FR Doc. 2024-06547 Filed 3-28-24; 8:45 am]
BILLING CODE 3110-01-P

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.