Notice of Random Assignment Study To Evaluate Workforce Investment Act Adult and Dislocated Worker Programs; Final Notice, 5850-5852 [2012-2521]

Agencies

• Employment and Training Administration
• DEPARTMENT OF LABOR

[Federal Register Volume 77, Number 24 (Monday, February 6, 2012)]
[Notices]
[Pages 5850-5852]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2012-2521]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF LABOR

Employment and Training Administration


Notice of Random Assignment Study To Evaluate Workforce 
Investment Act Adult and Dislocated Worker Programs; Final Notice

AGENCY: Employment and Training Administration (ETA), Labor.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Employment and Training Administration (ETA) of the 
Department of Labor (DOL or the Department) will conduct an evaluation 
to provide rigorous, nationally-representative estimates of the net 
impacts of intensive services and training provided under the Workforce 
Investment Act (WIA) Adult and Dislocated Worker Programs. The 
Department has determined that it is in the public interest to use a 
random assignment impact methodology for the study. This methodology 
will provide ETA with estimates of the net impacts of WIA intensive 
services and training that are offered during the evaluation study 
period. On July 21, 2011 (76 FR 43729-43731), the Department solicited 
comments concerning the Department's plan to use random assignment 
methodology in carrying out the study. This notice is to provide the 
Department's response to the comments received.

FOR FURTHER INFORMATION CONTACT: Eileen Pederson, U.S. Department of 
Labor, Employment and Training Administration, Office of Policy 
Development and Research, 200 Constitution Avenue NW., Frances Perkins 
Bldg., Room N-5641, Washington, DC, 20210. Telephone: (202) 693-3647 
(this is not a toll-free number) or email: pederson.eileen@dol.gov. 
Individuals with hearing or speech impairments may access the telephone 
number above via TTY by calling the toll-free Federal Information Relay 
Service at 1-(877) 889-5627 (TTY/TDD).

SUPPLEMENTARY INFORMATION:

I. Background

    On July 21, 2011, the Department announced its plans to conduct an 
evaluation of the net impacts of intensive services and training 
provided under WIA (Pub. L. 105-220) Adult and Dislocated Worker 
Programs. To obtain rigorous, nationally representative estimates of 
WIA's effectiveness for adults and dislocated workers, the Department 
determined that it would use random assignment impact methodology for 
the evaluation.
    The design of the study was described as follows: The evaluation 
will be done in approximately 30 randomly selected LWIAs. WIA 
applicants in the selected LWIAs who are eligible for intensive 
services would be randomly assigned to one of three groups. The three 
research groups to which they would be assigned are: (1) The full-WIA 
group--adults and dislocated workers in this group can receive any WIA 
services and training for which they are eligible, (2) the core-and-
intensive group--adults and dislocated workers in this group can 
receive any WIA services for which they are eligible but no training, 
and (3) the core-only group--adults and dislocated workers in this 
group can receive only WIA core services but no intensive services or 
training.
    In the LWIAs selected for the study, all applicants for intensive 
services and/or training will be asked to participate in the study 
during the 12-18 month study enrollment period. They will be informed 
of the evaluation, provided an opportunity to ask questions or seek 
clarification of their role and responsibilities should they agree to 
participate, and then required to give their consent to participate. 
Applicants who do not consent to participate in the study will not be 
randomly assigned to one of the study groups but will be allowed to 
receive core services only. The participant enrollment period will 
range between 12 and 18 months in each LWIA.
    To protect the rights and welfare of WIA program applicants who 
agree to participate in the evaluation, the evaluation team, led by 
researchers from Mathematica and its subcontractor MDRC, submitted the 
WIA Adult and Dislocated Worker Programs evaluation design to MDRC's 
Institutional Review Board (IRB) for concurrence. An IRB is a committee 
specifically responsible for protecting the rights and welfare of 
humans involved in biomedical and behavioral research. On June 17, 
2010, MDRC's IRB determined this study to be of no more than minimal 
risk and approved it.
    The Department requested comments concerning its intent to carry 
out the random assignment study described above. The Department asked 
for comments focused on whether there is a methodology that would yield 
as credible and reliable impacts of the WIA programs as random 
assignment, but avoids adverse effect on the study participants. The 
Department also welcomed comments that suggest ways to more effectively 
minimize any adverse impact on the study participants who participate 
in the study described above.

II. Discussion of Comments Received

    The Department received comments from four sources in response to 
the notice. The comments were received from two workforce departments, 
one advocacy group, and one private citizen. The Department's responses 
to the comments are provided below.
    Comment: Two commenters asked about how other sources of funding 
for services would be accounted for in the study. One of these 
commenters asked whether the core-only group would have access to other 
partner services and, if so, the commenter suggested that the study 
take it into account through the follow-up survey. The other commenter 
was concerned that the study would not capture the nature of

[[Page 5851]]

the ``WIA system'' in which individuals are referred to partner 
services.
    Response: Training and employment services can be funded by many 
sources other than WIA (such as via Pell grants or State grants) and, 
as a commenter pointed out, staff from the One-Stop Career Centers may 
refer customers to services funded by these other sources. Accordingly, 
we will estimate the impact of WIA-funded services in communities as 
they exist, which includes services from other providers, rather than 
estimate the impact of WIA-funded services in a vacuum. Hence, all 
customers in the study (including members of the core-only group) will 
have access to services provided by other community service providers. 
After the 12-18 month study enrollment period concludes, we will 
conduct two follow-up surveys of study participants, one at 15 months 
after enrollment and one at 30 months after. The surveys will collect 
detailed information on services received by study participants. The 
LWIA counselors will be trained not to refer anyone to services to 
which they would not have referred them in the absence of the study. 
The study research questions address the impact of the additional 
receipt of WIA-funded intensive services and training above the other 
services in the community that customers may access. Through our 
analysis of the study, we will document the WIA system that is in place 
in each of the participating LWIAs. Through on-site visits, researchers 
will document the availability of non-WIA training and employment 
services provided by partner agencies.
    Comment: One commenter suggested that because of variation in how 
LWIAs label services as core, intensive, or training, the study should 
focus on the impact of specific services with similar intensity rather 
than the relative effectiveness of level of service as authorized by 
WIA.
    Response: As the commenter correctly points out, the activities 
labeled as core, intensive, and training vary by LWIA. However, 
consistent with the commenter's suggestion, for purposes of this study, 
during the analysis of outcomes and qualitative data, we will develop a 
standard definition of services so that the definition of ``core 
services,'' ``intensive services,'' and ``training'' are similar across 
all study sites, irrespective of how each LWIA individually defines its 
services.
    Comment: One commenter asked how the study would estimate the 
effectiveness of training when customers are randomly assigned to a 
group that is eligible for training in general--which can vary 
tremendously across and within LWIAs--rather than a specific type of 
training. Two commenters noted the importance of taking into account 
the fact that different study participants in the full-WIA group may 
receive different forms of training for different industries and 
occupations.
    Response: Customers will be randomly assigned to three groups: (1) 
The core-only group, in which customers can only receive core services; 
(2) the core-and-intensive services group, in which customers can 
receive core or intensive services, but not training; and (3) the full-
WIA group, in which customers can receive core, intensive, and training 
services. We will estimate the impact of training by comparing 
customers in the full-WIA group with customers in the intensive-
services group. We can account statistically for customers in the full-
WIA group who do not receive training. To do so, we will use (1) 
information from the study registration forms on counselors' 
projections made prior to random assignment on the likelihood customers 
will receive WIA training services and (2) standard instrumental 
variables methods that assume zero impacts for those in the full-WIA 
group who do not receive WIA intensive or training services.
    We agree with the commenters that it is important to take into 
account different types of training that may be received by 
participants. Our follow-up survey will collect detailed information 
about the type of training received, including the occupation targeted 
by the training, the length of the training, any credentials received 
as a result of the training, and the type of training provider. Through 
qualitative data collection, the researchers also will document the 
types of training available through the local workforce investment area 
and larger community.
    Comment: One commenter asked whether the study enrollment period 
would be long enough to capture the impacts of training.
    Response: The enrollment period refers to the period during which 
customers will be enrolled into the study and randomly assigned. 
Participants in the study will be surveyed twice, once at 15 months 
after random assignment and once at 30 months after random assignment. 
The 30-month follow-up period is long enough to capture the impacts of 
training, including quite long-term training.
    Comment: One commenter was concerned about 50 percent of the 
participants being placed in a control group. The commenter was also 
concerned about the time and effort taken by LWIA staff to determine 
eligibility for training and then after that, the time a customer is 
denied access to WIA-funded training because of the study.
    Response: Only about six percent of all study participants will be 
placed in one of the two restrictive-service groups (that is, the core- 
or intensive-services groups). All study participants will be allowed 
to receive core services or other services available in the community. 
This percentage limits services to the smallest number of customers 
while still meeting requirements for a valid, rigorous evaluation of 
the impact of intensive services and training. Random assignment of 
customers will be determined right after they have been determined 
eligible for intensive services, and the time it takes for customers to 
be determined eligible for training will take place well after they 
know the results of random assignment. Customers randomly assigned to 
one of the restrictive service groups will be eligible to reapply for 
all WIA intensive services and training 15 months after random 
assignment into the study.
    Comment: The Department received three comments that questioned the 
timing of the evaluation at a time of high unemployment. These 
commenters had two concerns: (1) that it is wrong to deny customers 
services in time of great need, and (2) that services are less 
effective when there is high unemployment.
    Response: While demand for intensive services and training 
increases as unemployment increases, there has not been an increase in 
funding for these services. Hence, because of funding limitations, not 
all customers who apply for these services can access them even in the 
absence of the study. During the study enrollment period, random 
assignment is also a fair way to allocate the resources that are 
available. The LWIAs participating in the study will be able to serve 
the same number of people who were able to access services and training 
in the absence of the study. Rather than the One-Stop allocating 
resources on a first-come, first-served basis, random assignment will 
give everyone who enrolls during the entire study enrollment period an 
equal chance of receiving intensive services and training.
    Whether intensive services and training are more or less effective 
when unemployment is high is unknown. Arguments can be made that 
training will be more effective in these periods because it provides 
workers with skills for which there is demand, or connections with 
employers that the

[[Page 5852]]

average person may not have; arguments can also be made that training 
is ineffective in periods of high unemployment because there are no 
jobs in the areas in which customers are trained. However, a study of 
the impacts of training in Europe finds that the magnitude of the 
impacts is higher in periods of high unemployment (Lechner and Wunsch, 
2006, IZA discussion paper number 2355). The Department believes that 
the public workforce investment system must prove its worth under all 
economic conditions, including during times of economic challenges and 
high unemployment, since Federal funding for these programs is not--
currently--predicated on the country's or area's employment situation.
    Comment: Two comments mentioned the additional work the study will 
require of local staff. One commenter suggested that participating 
sites should receive monetary compensation for participating in the 
study.
    Response: We recognize that the study requires additional work of 
local staff in the selected LWIAs and, therefore, participating LWIAs 
are receiving compensation for extra costs incurred due to the study. 
In addition, evaluation staff will work with staff in the selected 
LWIAs to minimize the effect that study procedures may have on each 
area's ongoing procedures.
    Comment: One commenter suggested that there should be adjustments 
for performance measures for participating sites.
    Response: The restricted service groups are so small that the 
evaluation is unlikely to affect performance measures for participating 
sites.
    The Department appreciates the comments received in response to the 
request for public comment. All the comments gave useful information 
and provided suggestions which we had already incorporated into the 
study's design. The responses provided above outline the specific 
aspects of the evaluation methodology that address each comment.
    Conclusion: Accordingly, the Department has determined that it is 
in the public interest to use a random assignment methodology for the 
study since this methodology will provide the most reliable estimates 
of the net impacts of WIA intensive services and training.

    Signed: at Washington, DC, this 25th day of January 2012.
Jane Oates,
Assistant Secretary for Employment and Training.
[FR Doc. 2012-2521 Filed 2-3-12; 8:45 am]
BILLING CODE 4510-FN-P