Statistical Policy Directive No. 2 Addendum: Standards and Guidelines for Cognitive Interviews, 29107-29109 [2016-10958]

Download as PDF Vol. 81 Tuesday, No. 90 May 10, 2016 Part IV Office of Management and Budget asabaliauskas on DSK3SPTVN1PROD with NOTICES Statistical Policy Directive No. 2 Addendum: Standards and Guidelines for Cognitive Interviews; Notice VerDate Sep<11>2014 20:00 May 09, 2016 Jkt 238001 PO 00000 Frm 00001 Fmt 4717 Sfmt 4717 E:\FR\FM\10MYN2.SGM 10MYN2 29108 Federal Register / Vol. 81, No. 90 / Tuesday, May 10, 2016 / Notices Statistical Policy Directive No. 2 Addendum: Standards and Guidelines for Cognitive Interviews Office of Management and Budget, Executive Office of the President. AGENCY: Notice of solicitation of comments. ACTION: Under the Budget and Accounting Procedures Act of 1950 (31 U.S.C. 1104(d)) and the Paperwork Reduction Act of 1995 (44 U.S.C. 3504(e)), the Office of Management and Budget (OMB) issues for comment a proposed addendum to Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys (71 FR 55522, September 22, 2006). This addendum reflects the ongoing commitment of the Federal statistical system to ensure robust application across the Government of advances in survey methods. In its role as coordinator of the Federal statistical system under the Paperwork Reduction Act, OMB, among other responsibilities, is required to ensure the efficiency and effectiveness of the system as well as the relevance, accuracy, objectivity, and confidentiality of information collected for statistical purposes. OMB is also charged with developing and overseeing the implementation of Governmentwide principles, policies, standards, and guidelines concerning the development, presentation, and dissemination of statistical information. Accordingly, OMB requests comments on the recommendations that it received from the Federal Committee on Statistical Methodology (FCSM) Subcommittee on Question Evaluation Methodology for an addendum to OMB Statistical Policy Directive No. 2, Standards and Guidelines for Statistical Surveys. The addendum, Standards and Guidelines for Cognitive Interviews is intended to ensure that the results of statistical surveys sponsored by the Federal Government are as reliable and useful as possible while minimizing respondent burden. The addendum may be accessed at www.omb.gov/inforeg/Directive_No. 2_Addendum. Comments are also requested regarding suggestions for other aspects of statistical methodology to be considered for inclusion in future addenda to this directive. Additional discussion of the proposed addendum may be found in the SUPPLEMENTARY INFORMATION section below. asabaliauskas on DSK3SPTVN1PROD with NOTICES SUMMARY: VerDate Sep<11>2014 20:00 May 09, 2016 Jkt 238001 Comments and recommendations on the proposed addendum detailed in this notice must be in writing. To ensure consideration of comments, they must be received no later than June 24, 2016. Please be aware of delays in mail processing at Federal facilities due to security screenings. Respondents are encouraged to send comments electronically via email, FAX, or http://www.regulations.gov (discussed in ADDRESSES below). DATES: OFFICE OF MANAGEMENT AND BUDGET Please send any comments or questions about this directive to: Katherine K. Wallman, Chief Statistician, Office of Management and Budget, 1800 G St., 9th Floor, Washington, DC 20503, telephone number: (202) 395–3093, FAX number: (202) 395–7245. You may also send comments or questions via email to Directive_No._2_Addendum@ omb.eop.gov or to http:// www.regulations.gov—a Federal EGovernment Web site that allows the public to find, review, and submit comments on documents that agencies have published in the Federal Register and that are open for comment. Simply type ‘‘Directive No. 2 Addendum’’ (in quotes) in the Comment or Submission search box, click Go, and follow the instructions for submitting comments. Comments submitted in response to this notice may be made available to the public through relevant Web sites. For this reason, please do not include in your comments information of a confidential nature, such as sensitive personal information or proprietary information. If you send an email comment, your email address will be automatically captured and included as part of the comment that is placed in the public docket. Please note that responses to this public comment request containing any routine notice about the confidentiality of the communication will be treated as public comments that may be made available to the public notwithstanding the inclusion of the routine notice. Because of delays in the receipt of regular mail related to security screening, respondents are encouraged to use electronic communications. Electronic Availability: This document is available on the Internet on the OMB Web site at www.omb.gov/ inforeg/Directive_No._2_Addendum. ADDRESSES: FOR FURTHER INFORMATION CONTACT: Jennifer Park, 1800 G St., 9th Floor, Washington, DC 20503, email address: jpark@omb.eop.gov with subject Directive No. 2 Addendum, telephone number: (202) 395–9046, FAX number: (202) 395–7245. PO 00000 Frm 00002 Fmt 4701 Sfmt 4703 The Nation relies on the flow of credible statistics to support the decisions of governments, businesses, individuals, households, and other organizations. Federal surveys collect much of the information available about the United States’ economy, population, natural resources, environment, and public and private institutions. It is essential that these surveys collect information in a clear, straight-forward manner so as to maximize the accuracy of data while minimizing respondent burden. Background: Consistent with the Information Quality Act (Pub. L. 106– 554, Division C, title V, Sec. 515, Dec. 21, 2000; 114 Stat. 2763A–153 to 2763A–154) and in accordance with Statistical Policy Directive No. 1: Fundamental Responsibilities of Federal Statistical Agencies and Recognized Statistical Units (79 FR 71610, Dec. 2, 2014), it is the responsibility of Federal agencies engaging in statistical work to continuously review statistical methodologies and implement improvements as they are identified so as to better ensure the relevance, accuracy, and objectivity of the statistical products our Nation uses to monitor and assess performance, progress, and needs. Further, these responsibilities must be achieved in a manner that protects the confidentiality of information acquired solely for statistical purposes. Among other requirements, Federal guidelines, such as those provided in Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys, recommend robust testing of survey questions before implementation. Cognitive interviewing is a key method used to pretest survey questions and questionnaires. This method investigates whether respondents understand survey questions according to their intended design and whether respondents can provide accurate answers based on that intent. Cognitive interviews determine respondent interpretations and detail the phenomena considered by respondents in forming their answer. Findings from cognitive interviews can indicate whether a survey question captures the intended construct; cognitive interview findings can also identify difficulties that respondents experience when formulating responses. Ultimately, rigorous cognitive interviews support the efficient production of useful statistics since the findings from cognitive interviews can be used to help minimize costs and burden while ensuring the accuracy of the information collected. SUPPLEMENTARY INFORMATION: E:\FR\FM\10MYN2.SGM 10MYN2 Federal Register / Vol. 81, No. 90 / Tuesday, May 10, 2016 / Notices asabaliauskas on DSK3SPTVN1PROD with NOTICES Although cognitive interviews are broadly used across a variety of Federal agencies engaged in statistical activities, to date no specific Federal guidance has established the manner in which this particular method should be conducted. As such, the term ‘‘cognitive interview,’’ when applied to Federal statistical surveys could refer to any of an array of related research techniques that differ in their appropriate use for statistical surveys. The purpose of this addendum is to establish rigorous standards for the use of cognitive interviews in Federal information collections. These standards would allow data collectors and users alike to better evaluate the quality of survey questions and the resulting survey statistics. The addendum would apply to all Federal agencies subject to the Paperwork Reduction Act of 1995. Agencies not subject to the PRA would also benefit from this addendum, and are therefore encouraged to apply this guidance as well. Development and Review: Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys was issued in 2006. It remains a robust and comprehensive source of guidance to Federal agencies engaged in statistical acitivites. Specifically, Statisical Policy Directive No. 2 was issued to ensure implementation of improved survey methods across Federal agencies. To achieve this, periodic updates and VerDate Sep<11>2014 20:00 May 09, 2016 Jkt 238001 addenda are required to ensure that this Directive remains most useful to Federal agencies. Accordingly, in 2014 OMB requested members of the Federal Interagency Council on Statistical Policy (ICSP) to nominate representatives for a new subcommittee formed under the aegis of the Federal Committee on Statistical Methodology (FCSM). The Question Evaluation Methodology Subcommittee was asked to identify best practices for conducting cognitive interviews so as to improve the resulting data quality. Subcommittee members reviewed relevant scientific literature, the Paperwork Reduction Act, the Information Quality Act, Governmentwide Information Quality Guidelines (Information Quality Guidelines) (67 FR 8453, Jan. 3, 2002), Statistical Policy Directive No. 1 and Directive No. 2, and other information quality standards currently used by Federal statistical agencies. Through this careful process, the subcommittee reached a consensus in identifying seven standards for cognitive interviews. The subcommittee provided draft standards and guidelines for review by the FCSM and the ICSP in 2015. The subcommittee addressed the comments it received at each stage and provided its final recommendations to the FCSM in 2015. OMB proposes the issuance of these recommendations as an addendum to Statistical Policy PO 00000 Frm 00003 Fmt 4701 Sfmt 9990 29109 Directive No. 2: Standards and Guidelines for Cognitive Interviews. Issues for Comment: With this notice, OMB requests comments on the proposed addendum, which can be accessed at www.omb.gov/inforeg/ Directive_No. 2_Addendum. OMB seeks comments from all interested parties on all aspects of this proposed addendum. In particular, OMB seeks comment on the merit of the proposed standards and guidelines both about technical terms, such as statistical definitions, and as statistical policy. These standards and guidelines for cognitive interviews should reflect best practices for Federal agencies and their contractors in conducting statistical surveys as well as sound policy for the Federal statistical system. OMB seeks comment on whether some provisions of this proposal should be modified or deleted to meet these goals. Finally, OMB seeks comment from affected agencies on the expected benefits and burdens of the proposed addendum. Suggestions regarding other aspects of statistical methodology that should be considered as future addenda to Directive No. 2 are also encouraged. Howard A. Shelanski, Administrator, Office of Information and Regulatory Affairs. [FR Doc. 2016–10958 Filed 5–9–16; 8:45 am] BILLING CODE P E:\FR\FM\10MYN2.SGM 10MYN2

Agencies

[Federal Register Volume 81, Number 90 (Tuesday, May 10, 2016)]
[Notices]
[Pages 29107-29109]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2016-10958]



[[Page 29107]]

Vol. 81

Tuesday,

No. 90

May 10, 2016

Part IV





Office of Management and Budget





-----------------------------------------------------------------------





 Statistical Policy Directive No. 2 Addendum: Standards and Guidelines 
for Cognitive Interviews; Notice

Federal Register / Vol. 81 , No. 90 / Tuesday, May 10, 2016 / 
Notices

[[Page 29108]]


-----------------------------------------------------------------------

OFFICE OF MANAGEMENT AND BUDGET


Statistical Policy Directive No. 2 Addendum: Standards and 
Guidelines for Cognitive Interviews

AGENCY: Office of Management and Budget, Executive Office of the 
President.

ACTION: Notice of solicitation of comments.

-----------------------------------------------------------------------

SUMMARY: Under the Budget and Accounting Procedures Act of 1950 (31 
U.S.C. 1104(d)) and the Paperwork Reduction Act of 1995 (44 U.S.C. 
3504(e)), the Office of Management and Budget (OMB) issues for comment 
a proposed addendum to Statistical Policy Directive No. 2: Standards 
and Guidelines for Statistical Surveys (71 FR 55522, September 22, 
2006). This addendum reflects the ongoing commitment of the Federal 
statistical system to ensure robust application across the Government 
of advances in survey methods.
    In its role as coordinator of the Federal statistical system under 
the Paperwork Reduction Act, OMB, among other responsibilities, is 
required to ensure the efficiency and effectiveness of the system as 
well as the relevance, accuracy, objectivity, and confidentiality of 
information collected for statistical purposes. OMB is also charged 
with developing and overseeing the implementation of Government-wide 
principles, policies, standards, and guidelines concerning the 
development, presentation, and dissemination of statistical 
information. Accordingly, OMB requests comments on the recommendations 
that it received from the Federal Committee on Statistical Methodology 
(FCSM) Subcommittee on Question Evaluation Methodology for an addendum 
to OMB Statistical Policy Directive No. 2, Standards and Guidelines for 
Statistical Surveys. The addendum, Standards and Guidelines for 
Cognitive Interviews is intended to ensure that the results of 
statistical surveys sponsored by the Federal Government are as reliable 
and useful as possible while minimizing respondent burden. The addendum 
may be accessed at www.omb.gov/inforeg/Directive_No. 2_Addendum. 
Comments are also requested regarding suggestions for other aspects of 
statistical methodology to be considered for inclusion in future 
addenda to this directive.
    Additional discussion of the proposed addendum may be found in the 
SUPPLEMENTARY INFORMATION section below.

DATES: Comments and recommendations on the proposed addendum detailed 
in this notice must be in writing. To ensure consideration of comments, 
they must be received no later than June 24, 2016. Please be aware of 
delays in mail processing at Federal facilities due to security 
screenings. Respondents are encouraged to send comments electronically 
via email, FAX, or http://www.regulations.gov (discussed in ADDRESSES 
below).

ADDRESSES: Please send any comments or questions about this directive 
to: Katherine K. Wallman, Chief Statistician, Office of Management and 
Budget, 1800 G St., 9th Floor, Washington, DC 20503, telephone number: 
(202) 395-3093, FAX number: (202) 395-7245. You may also send comments 
or questions via email to Directive_No._2_Addendum@omb.eop.gov or to 
http://www.regulations.gov--a Federal E-Government Web site that allows 
the public to find, review, and submit comments on documents that 
agencies have published in the Federal Register and that are open for 
comment. Simply type ``Directive No. 2 Addendum'' (in quotes) in the 
Comment or Submission search box, click Go, and follow the instructions 
for submitting comments.
    Comments submitted in response to this notice may be made available 
to the public through relevant Web sites. For this reason, please do 
not include in your comments information of a confidential nature, such 
as sensitive personal information or proprietary information. If you 
send an email comment, your email address will be automatically 
captured and included as part of the comment that is placed in the 
public docket. Please note that responses to this public comment 
request containing any routine notice about the confidentiality of the 
communication will be treated as public comments that may be made 
available to the public notwithstanding the inclusion of the routine 
notice. Because of delays in the receipt of regular mail related to 
security screening, respondents are encouraged to use electronic 
communications.
    Electronic Availability: This document is available on the Internet 
on the OMB Web site at www.omb.gov/inforeg/Directive_No._2_Addendum.

FOR FURTHER INFORMATION CONTACT: Jennifer Park, 1800 G St., 9th Floor, 
Washington, DC 20503, email address: jpark@omb.eop.gov with subject 
Directive No. 2 Addendum, telephone number: (202) 395-9046, FAX number: 
(202) 395-7245.

SUPPLEMENTARY INFORMATION: The Nation relies on the flow of credible 
statistics to support the decisions of governments, businesses, 
individuals, households, and other organizations. Federal surveys 
collect much of the information available about the United States' 
economy, population, natural resources, environment, and public and 
private institutions. It is essential that these surveys collect 
information in a clear, straight-forward manner so as to maximize the 
accuracy of data while minimizing respondent burden.
    Background: Consistent with the Information Quality Act (Pub. L. 
106-554, Division C, title V, Sec. 515, Dec. 21, 2000; 114 Stat. 2763A-
153 to 2763A-154) and in accordance with Statistical Policy Directive 
No. 1: Fundamental Responsibilities of Federal Statistical Agencies and 
Recognized Statistical Units (79 FR 71610, Dec. 2, 2014), it is the 
responsibility of Federal agencies engaging in statistical work to 
continuously review statistical methodologies and implement 
improvements as they are identified so as to better ensure the 
relevance, accuracy, and objectivity of the statistical products our 
Nation uses to monitor and assess performance, progress, and needs. 
Further, these responsibilities must be achieved in a manner that 
protects the confidentiality of information acquired solely for 
statistical purposes. Among other requirements, Federal guidelines, 
such as those provided in Statistical Policy Directive No. 2: Standards 
and Guidelines for Statistical Surveys, recommend robust testing of 
survey questions before implementation.
    Cognitive interviewing is a key method used to pretest survey 
questions and questionnaires. This method investigates whether 
respondents understand survey questions according to their intended 
design and whether respondents can provide accurate answers based on 
that intent. Cognitive interviews determine respondent interpretations 
and detail the phenomena considered by respondents in forming their 
answer. Findings from cognitive interviews can indicate whether a 
survey question captures the intended construct; cognitive interview 
findings can also identify difficulties that respondents experience 
when formulating responses. Ultimately, rigorous cognitive interviews 
support the efficient production of useful statistics since the 
findings from cognitive interviews can be used to help minimize costs 
and burden while ensuring the accuracy of the information collected.

[[Page 29109]]

    Although cognitive interviews are broadly used across a variety of 
Federal agencies engaged in statistical activities, to date no specific 
Federal guidance has established the manner in which this particular 
method should be conducted. As such, the term ``cognitive interview,'' 
when applied to Federal statistical surveys could refer to any of an 
array of related research techniques that differ in their appropriate 
use for statistical surveys. The purpose of this addendum is to 
establish rigorous standards for the use of cognitive interviews in 
Federal information collections. These standards would allow data 
collectors and users alike to better evaluate the quality of survey 
questions and the resulting survey statistics. The addendum would apply 
to all Federal agencies subject to the Paperwork Reduction Act of 1995. 
Agencies not subject to the PRA would also benefit from this addendum, 
and are therefore encouraged to apply this guidance as well.
    Development and Review: Statistical Policy Directive No. 2: 
Standards and Guidelines for Statistical Surveys was issued in 2006. It 
remains a robust and comprehensive source of guidance to Federal 
agencies engaged in statistical acitivites. Specifically, Statisical 
Policy Directive No. 2 was issued to ensure implementation of improved 
survey methods across Federal agencies. To achieve this, periodic 
updates and addenda are required to ensure that this Directive remains 
most useful to Federal agencies.
    Accordingly, in 2014 OMB requested members of the Federal 
Interagency Council on Statistical Policy (ICSP) to nominate 
representatives for a new subcommittee formed under the aegis of the 
Federal Committee on Statistical Methodology (FCSM). The Question 
Evaluation Methodology Subcommittee was asked to identify best 
practices for conducting cognitive interviews so as to improve the 
resulting data quality. Subcommittee members reviewed relevant 
scientific literature, the Paperwork Reduction Act, the Information 
Quality Act, Government-wide Information Quality Guidelines 
(Information Quality Guidelines) (67 FR 8453, Jan. 3, 2002), 
Statistical Policy Directive No. 1 and Directive No. 2, and other 
information quality standards currently used by Federal statistical 
agencies. Through this careful process, the subcommittee reached a 
consensus in identifying seven standards for cognitive interviews. The 
subcommittee provided draft standards and guidelines for review by the 
FCSM and the ICSP in 2015. The subcommittee addressed the comments it 
received at each stage and provided its final recommendations to the 
FCSM in 2015. OMB proposes the issuance of these recommendations as an 
addendum to Statistical Policy Directive No. 2: Standards and 
Guidelines for Cognitive Interviews.
    Issues for Comment: With this notice, OMB requests comments on the 
proposed addendum, which can be accessed at www.omb.gov/inforeg/Directive_No. 2_Addendum. OMB seeks comments from all interested 
parties on all aspects of this proposed addendum. In particular, OMB 
seeks comment on the merit of the proposed standards and guidelines 
both about technical terms, such as statistical definitions, and as 
statistical policy. These standards and guidelines for cognitive 
interviews should reflect best practices for Federal agencies and their 
contractors in conducting statistical surveys as well as sound policy 
for the Federal statistical system. OMB seeks comment on whether some 
provisions of this proposal should be modified or deleted to meet these 
goals. Finally, OMB seeks comment from affected agencies on the 
expected benefits and burdens of the proposed addendum. Suggestions 
regarding other aspects of statistical methodology that should be 
considered as future addenda to Directive No. 2 are also encouraged.

Howard A. Shelanski,
Administrator, Office of Information and Regulatory Affairs.
[FR Doc. 2016-10958 Filed 5-9-16; 8:45 am]
 BILLING CODE P