Agency Information Collection Activities; Submission to the Office of Management and Budget (OMB) for Review and Approval; Comment Request; Generic Clearance for Internet Panel Pretesting and Qualitative Survey Methods Testing, 71529-71531 [2023-22896]

Agencies

• Census Bureau
• DEPARTMENT OF COMMERCE

[Federal Register Volume 88, Number 199 (Tuesday, October 17, 2023)]
[Notices]
[Pages 71529-71531]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2023-22896]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF COMMERCE

Census Bureau


Agency Information Collection Activities; Submission to the 
Office of Management and Budget (OMB) for Review and Approval; Comment 
Request; Generic Clearance for Internet Panel Pretesting and 
Qualitative Survey Methods Testing

AGENCY: Census Bureau, Department of Commerce.

ACTION: Notice of information collection, request for comment.

-----------------------------------------------------------------------

SUMMARY: The Department of Commerce, in accordance with the Paperwork 
Reduction Act (PRA) of 1995, invites the general public and other 
Federal agencies to comment on proposed, and continuing information 
collections, which helps us assess the impact of our information 
collection requirements and minimize the public's reporting burden. The 
purpose of this notice is to allow for 60 days of public comment on the 
proposed extension of the Generic Clearance for Internet Panel 
Pretesting and Qualitative Survey Methods Testing, prior to the 
submission of the information collection request (ICR) to OMB for 
approval.

DATES: To ensure consideration, comments regarding this proposed

[[Page 71530]]

information collection must be received on or before December 18, 2023.

ADDRESSES: Interested persons are invited to submit written comments by 
email to [email protected]. Please reference OMB Control Number 0607-
0978 in the subject line of your comments. You may also submit 
comments, identified by Docket Number USBC-2023-0002, to the Federal e-
Rulemaking Portal: https://www.regulations.gov. All comments received 
are part of the public record. No comments will be posted to https://www.regulations.gov for public viewing until after the comment period 
has closed. Comments will generally be posted without change. All 
Personally Identifiable Information (for example, name and address) 
voluntarily submitted by the commenter may be publicly accessible. Do 
not submit Confidential Business Information or otherwise sensitive or 
protected information. You may submit attachments to electronic 
comments in Microsoft Word, Excel, or Adobe PDF file formats.

FOR FURTHER INFORMATION CONTACT: Requests for additional information or 
specific questions related to collection activities should be directed 
to Aleia Clark Fobia, U.S. Census Bureau, 4600 Silver Hill Road, Center 
for Behavioral Science Methods, Washington, DC 20233 or by calling 202-
893-4091.

SUPPLEMENTARY INFORMATION:

I. Abstract

    The Census Bureau is committed to conducting research in a cost-
efficient manner. The U.S. Census Bureau plans to request an extension 
of the current OMB approval to conduct a series of medium-scale 
internet-based tests, as a cost-efficient method of testing questions 
and contact strategies over the internet through different types of 
samples. Using internet panel pretesting, we can answer some research 
questions more thoroughly than in the small-scale testing, but less 
expensively than in the large-scale field test.
    This research program will be used by the Census Bureau and survey 
sponsors to test alternative contact methods, including emails and text 
messages, improve online questionnaires and procedures, reduce 
respondent burden, and ultimately increase the quality of data 
collected in the censuses and surveys. We will use the clearance to 
conduct experimental pretesting of decennial and demographic census and 
survey questionnaires prior to fielding them as well as communications 
and/or marketing strategies and data dissemination tools for the Census 
Bureau. The primary method of identifying measurement problems with the 
questionnaire or survey procedure is split panel tests. This will 
encompass both methodological and subject matter research questions 
that can be tested on a medium-scale internet panel.
    This research program will also be used by the Census Bureau for 
remote usability testing of electronic interfaces and to perform other 
qualitative analyses such as respondent debriefings. An advantage of 
using remote, medium-scale testing is that participants can test 
products at their convenience using their own equipment, as opposed to 
using Census Bureau-supplied computers. A diverse participant pool 
(geographically, demographically, or economically) is another 
advantage. Remote usability testing would use click through rates and 
other paradata, accuracy and satisfaction scores, and written 
qualitative comments to determine optimal interface designs and to 
obtain feedback from respondents.
    The public is currently offered an opportunity to participate in 
this research remotely, by signing up for an online research panel. If 
a person opts in, the Census Bureau will occasionally email (or text, 
if applicable) the person an invitation to complete a survey for one of 
our research projects. Invited respondents will be told the topic of 
the survey, and how long it will take to complete it. Under this 
clearance, we will also conduct similar-scale and similarly designed 
research using other email lists to validate preliminary findings and 
expand the research.

II. Method of Collection

    Split sample experiments. This involves testing alternative 
versions of questionnaires, invitations to questionnaires (e.g., emails 
or text messages), or websites, at least some of which have been 
designed to address problems identified in draft versions or versions 
from previous waves. The use of multiple questionnaires, invitations, 
or websites, randomly assigned to permit statistical comparisons, is 
the critical component here; data collection will be via the internet. 
Comparison of revised questionnaires (or invitations) against a control 
version, preferably, or against each other facilitates statistical 
evaluation of the performance of alternative versions of the 
questionnaire (or invitation or website).
    The number of versions tested and the number of cases per version 
will depend on the objectives of the test. We cannot specify with 
certainty a minimum panel size, although we would expect that no 
questionnaire versions would be administered to less than fifty 
respondents.
    Split sample tests that incorporate methodological questionnaire 
design experiments will have a larger maximum sample size (up to 
several hundred cases per panel) than other pretest methods. This will 
enable the detection of statistically significant differences and 
facilitate methodological experiments that can extend questionnaire 
design knowledge more generally for use in a variety of Census Bureau 
data collection instruments.
    Usability Interviews: This method involves getting respondent input 
to aid in the development of automated questionnaires and websites and 
associated materials. The objective is to identify problems that keep 
respondents from completing automated questionnaires accurately and 
efficiently with minimal burden, or that prevent respondents from 
successfully navigating websites and finding the information they seek. 
Remote usability testing may be conducted under this clearance, whereby 
a user would receive an invitation to use a website or survey, then 
answer targeted questions about that experience.
    Qualitative Interviews: This method involves one-on-one (or 
sometimes group) interviews in which the respondent is typically asked 
questions about survey content areas, survey questions or the survey 
process. A number of different techniques may be involved, including 
cognitive interviews and focus groups. The objective is to identify 
problems of ambiguity or misunderstanding, or other difficulties 
respondents may have answering survey questions in order to improve the 
information ultimately collected in large scale surveys and censuses.
    Data collection for this project is authorized under the 
authorizing legislation for the questionnaire being tested. This may be 
Title 13, Sections 131, 141, 161, 181, 182, 193, and 301 for Census 
Bureau-sponsored surveys, and Title 13, Section 8(b), and Title 15 for 
surveys sponsored by other Federal agencies. We do not now know what 
other titles will be referenced, since we do not know what survey 
questionnaires will be pretested during the course of the clearance.
    Literature on and considerations about the use of internet samples 
for this type of work have been thoroughly covered by a Task Force 
commissioned by the American Association for Public Opinion Research 
and are well documented there (Baker, et al., 2013).
    The information collected in this program of developing and testing

[[Page 71531]]

questionnaires will be used by staff from the Census Bureau and 
sponsoring agencies to evaluate and improve the quality of the data in 
the surveys and censuses that are ultimately conducted. Because the 
questionnaires being tested under this clearance are still in the 
process of development, the data that result from these collections are 
not considered official statistics of the Census Bureau or other 
Federal agencies. Data will be included in research reports prepared 
for sponsors inside and outside of the Census Bureau. The results may 
also be prepared for presentations related to survey methodology at 
professional meetings or publications in professional journals.

III. Data

    OMB Control Number: 0607-0978.
    Form Number(s): TBD.
    Type of Review: Regular submission, Request for an Extension, 
without Change, of a Currently Approved Collection.
    Affected Public: Individuals or households.
    Estimated Number of Respondents: 67,600.
    Estimated Time per Response: 15 minutes.
    Estimated Total Annual Burden Hours: 16,900.
    Estimated Total Annual Cost to Public: $0. (This is not the cost of 
respondents' time, but the indirect costs respondents may incur for 
such things as purchases of specialized software or hardware needed to 
report, or expenditures for accounting or records maintenance services 
required specifically by the collection.)
    Respondent's Obligation: Voluntary.
    Legal Authority: Data collection for this project is authorized 
under the authorizing legislation for the questionnaire being tested. 
This may be Title 13, Sections 131, 141, 161, 181, 182, 193, and 301 
for Census Bureau-sponsored surveys, and Title 13 and 15 for surveys 
sponsored by other Federal agencies. We do not now know what other 
titles will be referenced, since we do not know what survey 
questionnaires will be pretested during the course of the clearance.

IV. Request for Comments

    We are soliciting public comments to permit the Department/Bureau 
to: (a) Evaluate whether the proposed information collection is 
necessary for the proper functions of the Department, including whether 
the information will have practical utility; (b) Evaluate the accuracy 
of our estimate of the time and cost burden for this proposed 
collection, including the validity of the methodology and assumptions 
used; (c) Evaluate ways to enhance the quality, utility, and clarity of 
the information to be collected; and (d) Minimize the reporting burden 
on those who are to respond, including the use of automated collection 
techniques or other forms of information technology.
    Comments that you submit in response to this notice are a matter of 
public record. We will include, or summarize, each comment in our 
request to OMB to approve this ICR. Before including your address, 
phone number, email address, or other personal identifying information 
in your comment, you should be aware that your entire comment--
including your personal identifying information--may be made publicly 
available at any time. While you may ask us in your comment to withhold 
your personal identifying information from public review, we cannot 
guarantee that we will be able to do so.

Sheleen Dumas,
Department PRA Clearance Officer, Office of the Under Secretary for 
Economic Affairs, Commerce Department.
[FR Doc. 2023-22896 Filed 10-16-23; 8:45 am]
BILLING CODE 3510-07-P