Family Self-Sufficiency Performance Measurement System (“Composite Score”), 57493-57500 [2018-24949]

Agencies

• DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

[Federal Register Volume 83, Number 221 (Thursday, November 15, 2018)]
[Notices]
[Pages 57493-57500]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2018-24949]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

[Docket No. FR-6046-N-02]


Family Self-Sufficiency Performance Measurement System 
(``Composite Score'')

AGENCY: Office of Public and Indian Housing, HUD.

ACTION: Notice of new performance measurement system (``Composite 
Score'') for the Family Self-Sufficiency Program.

-----------------------------------------------------------------------

SUMMARY: This notice describes and responds to comments on a 
performance measurement system that HUD plans to implement for Public 
Housing Agencies (PHAs) that receive HUD Family Self-Sufficiency (FSS) 
program coordinator grants. The desired effect of this notice is to 
notify the public regarding the criteria for evaluating FSS programs.

DATES: Applicability Date: December 17, 2018.

FOR FURTHER INFORMATION CONTACT: Questions on this notice may be 
addressed to [email protected] or by contacting Anice Chenault at 502-618-
8163 (email strongly preferred).
    Electronic Data Availability. This Federal Register notice and a 
spreadsheet containing scores using the methodology for FSS programs 
funded in any of the last three years will be available electronically 
from the HUD FSS web page: https://www.hud.gov/program_offices/public_indian_housing/programs/hcv/fss. Federal Register notices also 
are available electronically at https://www.federalregister.gov/, the 
U.S. Government Printing Office website.

SUPPLEMENTARY INFORMATION: 

I. Background

    On December 12, 2017, HUD published a notice in the Federal 
Register (FR-6046-N-01, 82 FR 58434) (2017 Notice) describing and 
requesting comment on a performance measurement system that HUD plans 
to implement for public housing agencies (PHAs) that receive HUD Family 
Self Sufficiency (FSS) program coordinator grants. Through this notice, 
HUD is implementing the FSS performance measurement system, as proposed 
in the 2017 Notice. Additionally, in response to public comments, HUD 
is revising the methodology it uses to compute FSS Performance Scores 
under the new system; these revisions are described below, in section 
III of this notice. Henceforth, HUD will use the new system to evaluate 
the performance of PHAs receiving HUD program coordinator funding in a 
strictly advisory manner. Beginning with Fiscal Year (FY) 2019 
appropriations, HUD intends to use the performance measurement system 
in the determination of FSS funding awards. The complete, updated 
methodology can be found on HUD's website at: https://www.hud.gov/program_offices/public_indian_housing/programs/hcv/fss.
    Under section 23(i) of the Housing Act of 1937 (42 U.S.C. 
1437u(i)), HUD is required to establish criteria to evaluate eligible 
entities' implementation of local FSS programs. HUD has developed this 
new FSS performance measurement system to provide HUD, Congress, public 
housing agencies (PHAs), and other eligible entities with information 
on the performance of individual FSS programs. The information will 
help grantees determine how their programs compare to others across the 
country in efforts to help participants to successfully graduate from 
the program and make progress toward economic security. The information 
will also help HUD understand the extent to which FSS program 
performance--individually and collectively--improves or declines over 
time.
    Initially, HUD plans to use the performance measures to identify 
high performing and low performing FSS programs, which could inform its 
understanding of best practices and its delivery of technical 
assistance. Toward these goals, at least once per year, HUD will 
analyze data collected through the Public Housing Information Center 
(PIC) to calculate FSS performance scores for each FSS program that 
received an FSS coordinator grant in one or more of the past three 
fiscal year NOFA competitions. Beginning in Fiscal Year 2019, HUD plans 
to consider the FSS performance score of an FSS program in determining 
FSS funding awards.
    HUD developed the approach described in this Notice based in part 
on feedback received on an earlier performance measurement approach 
proposed in the FY 2014 FSS Notice of Funding Availability (NOFA). In 
the FY 2014 NOFA, HUD proposed, and asked for feedback on, evaluating 
FSS programs based on the share of FSS participants that experience an 
increase in earned income (also known as ``earnings growth'') over a 
specified time period. Some commenters raised concerns that this 
approach did not adequately account for differences in local economic 
conditions and differences in the approaches of local FSS programs. 
While some FSS programs encourage participants to increase their 
earnings immediately, others encourage FSS participants to build skills 
and credentials first and then seek higher paying jobs. The FSS 
performance measurement system proposed in the December 2017 Notice was 
developed to address these issues, as well as many others, and to allow 
for a more nuanced evaluation of the performance of local FSS programs.
    A PHA's FSS performance score will be calculated based on three 
measures, weighted as follows:
    A. Earnings Performance Measure (50 percent);
    B. FSS Graduation Rate (30 percent);
    C. Participation Rate (20 percent).
    HUD has selected these measures because they are important 
indicators of program performance and are verifiable using the data HUD 
collects through the PIC data system. No outside or additional 
reporting will be required, which ensures that the system will not 
increase the reporting burden of PHAs. No new Paperwork Reduction Act 
(PRA) Information Collection will be required for the scoring, as 
proposed.
    The Earnings Performance Measure represents the difference between 
the

[[Page 57494]]

earnings growth of FSS participants and the earnings growth of similar 
non-FSS households assisted by the PHA within a specified time frame. 
This approach, along with a statistical adjustment described below, 
helps to control for variations in local economic conditions. The 
program was envisioned and designed for the purpose of increasing 
employment and earnings for its participants. Therefore, the 
performance score assigns the Earnings Performance Measure a high 
weight.
    HUD has assigned the next highest weight to the Graduation Rate 
indicator--which represents the rate of FSS participants who 
successfully ``graduate'' from the program--to encourage PHAs to work 
closely with individual FSS participants to increase graduation rates. 
To graduate from FSS, a participant must be employed, be independent of 
cash welfare assistance for at least one year, and achieve the other 
goals set forth in the participant's contract of participation.
    Finally, the FSS performance score looks at the local program's 
Participation Rate, which reflects the extent to which a PHA exceeds 
the minimum number of households that HUD requires the PHA to serve as 
a condition of receiving an FSS grant. PHAs with higher Participation 
Rates are serving more households than required, which is a desired 
output, provided the PHAs are serving those households effectively. 
Because the Earnings Performance Measure is weighted more heavily than 
the Participation Rate, however, PHAs should be careful not to execute 
more Contracts of Participation than they can serve effectively, 
because doing so would likely reduce their scores on the Earnings 
Performance Measure. Together, the Earnings Performance Measure, 
Graduation Rate, and Participation Rate are expected to provide a 
balanced measurement of the performance of an individual FSS program.
    As indicated in the 2017 Notice soliciting public comment, HUD does 
not intend to use this performance measurement system for Tribes/
Tribally Designated Housing Entities (TDHEs), who do not report into 
Public and Indian Housing Information Center (PIC), or for PHAs with a 
Moving to Work (MTW) designation, as they report differently into PIC, 
using Form HUD-50058-MTW. However, HUD is presently exploring a change 
to the reporting processes for MTW agencies, in order to include them 
in the FSS performance scoring process. Nor does HUD intend, after 
considering public comment, to use this performance measurement system 
for unfunded PHAs, and PHAs and private owners that serve Project-based 
Rental Assistance (PBRA) residents at this time.\1\ The Agency will 
continue to explore options for modifying the scoring system for those 
sub-groups.
---------------------------------------------------------------------------

    \1\ Section 306 of the Economic Growth, Regulatory Relief, and 
Consumer Protection Act (Pub. L. 115-174, Approved May 24, 2018) 
amended the United States Housing Act of 1937. Among various 
provisions, this law extended FSS program eligibility to tenants of 
certain privately-owned properties subsidized with project-based 
rental assistance (PBRA).
---------------------------------------------------------------------------

II. HUD's Responses to Public Comments

    HUD received 68 unique public comments on the planned measures, 
which are summarized below along with HUD's responses. HUD's responses 
to comments are organized into five categories: (A) Overall Comments; 
(B) Comments on Earnings Performance Measure; (C) Comments on FSS 
Graduation Rate Measure; (D) Comments on Participation Rate Measure; 
and (E) Comments on Weighting of the Measures. At the conclusion of 
this Notice, in Section III., Final Thresholds, HUD provides the final 
FSS performance measurement system thresholds that it intends to adopt 
to calculate FSS performance scores.

A. Overall Comments

    1. Comment: Data Quality. Many commenters raised concerns about the 
quality of data from the PIC system used to calculate the FSS 
performance scores, particularly with regard to data entered prior to 
HUD's 2016 guidance. Some requested that PHAs be allowed to examine and 
correct all data used for calculating their measures prior to HUD 
calculating the FSS performance measures. Others suggested that this 
might not be possible or that there would not be resources to correct 
the data.
    HUD Response: Data Quality. On May 6, 2016, HUD issued PIH Notice 
2016-08 to help PHAs understand how to submit timely and accurate PIC 
data regarding FSS, along with a series of webinars to help PHAs apply 
the guidance to improve their PIC data quality for both current and 
past participants. Further, HUD has emphasized the importance of PHAs 
submitting accurate PIC data for many years. HUD believes it is 
reasonable to rely on existing PIC data in calculating FSS performance 
scores.
    It is important to note that each time the FSS performance scores 
are calculated, HUD will retrieve a new data report from the PIC 
system. This ensures that if a PHA has made changes to improve the 
accuracy of its reporting on any metric, for current or past 
participants, all of these changes will be reflected in its performance 
score.
    2. Comment: Limitations on Included Measures. Many commenters 
expressed the view that the measures in the planned performance 
measurement system do not address the variations in participants' 
goals. Some participants or programs may have interim goals related to 
addressing barriers to work (e.g., treating psychiatric illness or 
barriers, accessing medical care, securing childcare, or completing 
training, or education), which would not immediately result in higher 
earnings, even if participants make important progress. Several 
commenters suggested that participation in/provision of services or 
progress toward Individual Training and Services Plan (ITSP) goals 
should be included as a measure. Some suggested that changes in 
educational attainment also be included as a measure.
    Several commenters also stated that inputs and outputs should be 
included in the measures, such as the work associated with serving 
participants, meeting with participants, connecting participants to 
services, making referrals, etc. Some indicated that, without these 
measures, they are not given adequate ``credit'' for serving high-needs 
participants or that they may be penalized for participant performance 
issues that are beyond their control (through the earnings and FSS 
graduation measures).
    HUD Response: Limitations on Included Measures. HUD agrees that 
there is tremendous variety in the ITSP goals of individual FSS 
participants, which go beyond the statutorily mandated goals of 
employment and being welfare-free. It is precisely this variety, 
however, that makes these goals extremely difficult to factor into a 
performance measurement system. Since each ITSP is set up individually, 
it would be both impracticable and unwise to standardize ITSP goals 
across all programs. While HUD could potentially measure the share of 
ITSP goals achieved for each participant, this would not represent a 
direct comparison across local FSS programs if some programs set goals 
that were easy to attain while others set more difficult targets. This 
approach could also create an incentive for PHAs to change how they are 
defining individuals' goals to increase their FSS performance scores, 
without necessarily improving outcomes for participants. Finally, HUD 
does not currently collect data on the

[[Page 57495]]

goals set nor the share of ITSP goals that participants attain, so the 
inclusion of ITSP goal data in a performance measurement system for FSS 
would require additional reporting by PHAs, which would add to their 
administrative burden.
    HUD recognizes the importance and value of setting a range of goals 
for participants, including goals other than employment. Over time, 
however, HUD believes the achievement of these goals will support the 
ultimate goal of the program, which is increased earnings, which will 
then be captured in the performance measurement system. This is one of 
the benefits of having five (or more) years to work with participants. 
The long duration of the FSS program provides PHAs an opportunity to 
work with participants on a range of issues--including education, 
training, work readiness, etc.--that will, over time, contribute to 
earnings gains that can be measured and reflected in the FSS 
performance measurement system. The earnings and FSS graduation rate 
measures accommodate this long time-frame, examining data for FSS 
participants that entered the program as far back as 7.5 to 8 years 
ago, respectively.
    3. Comment: Homeownership. A few commenters expressed concern that 
the measures do not support homeownership goals for FSS participants 
and stated that progress toward homeownership should be included as a 
measure in the performance measurement system.
    HUD Response: Homeownership. HUD commends PHAs that work with 
participants on homeownership and recognizes that the achievement of 
homeownership is an important outcome for many FSS participants. At the 
same time, it is clear that homeownership is a more realistic goal in 
some parts of the U.S. than others, due to variations in the local 
economy. This makes it difficult and inequitable to use homeownership 
as a performance measure in comparing FSS programs on a national basis.
    4. Comment: Reliance on Past Performance Data. Some commenters 
opined that it is unfair to base an assessment of FSS performance on 
data from prior periods during which FSS coordinators were unaware of 
the performance measures and could not change their programs 
accordingly.
    HUD Response: Reliance on Past Performance Data. The performance 
measurement system recognizes that it takes considerable time for an 
individual FSS participant to make material progress in increasing his 
or her earnings and to graduate from the program. This requires 
measurements that span years, rather than months. To implement such a 
system prospectively, without relying on data from prior periods, would 
require HUD to wait many years before having valid measures of FSS 
program performance. Such a delay would undermine HUD's ability to 
achieve the key purposes of the FSS performance measurement system. In 
order to ensure that FSS funds are spent responsibly and that FSS 
participants have access to high-quality programs, HUD needs the 
ability to recognize the achievements of high-performing FSS programs 
and identify struggling FSS programs in need of improvement.
    The goals of improving earnings and helping FSS participants 
graduate successfully from the program should not come as a surprise to 
PHAs administering FSS programs. These goals have been clear since the 
program's inception and NOFAs have been announcing HUD's intent to use 
increased earnings as an evaluation metric since FY 2014. The 
participation rate also should not come as a surprise to PHAs, as HUD 
has historically based funding decisions on the number of FSS families 
served by PHAs. HUD's interest in PHAs serving more families (so long 
as they can do so without undermining earnings growth and FSS 
graduation rates), as reflected in the participation rate, is a factor 
that PHAs can influence going forward by adjusting their caseloads.
    5. Comment: Real-Time Data. Some commenters requested a way to 
monitor their programs' progress with respect to the measures 
periodically or in real time.
    HUD Response: Real-Time Data. HUD plans to provide updated scores 
at least once each year so PHAs can track their progress. In addition, 
PHAs can calculate their own participation rates and FSS graduation 
rates at any time.
    6. Comment: Small PHAs/Small FSS Programs. Several commenters 
raised the concern that the measures could disadvantage small PHAs or 
small FSS programs because volatility in the data would be more likely 
and factors beyond the FSS program's control could drive results.
    HUD Response: Small PHAs/Small FSS Programs. HUD recognizes that 
there may be greater volatility in the data for small FSS programs, 
which could be affected by the outcomes for one or more participants 
with unusual characteristics or experiences. Accordingly, in assigning 
earnings scores, HUD has built in protection for small FSS programs by 
using a test of statistical significance that makes it more difficult 
for smaller FSS programs than larger programs to receive a zero (0) 
score on the earnings measure. See the Dec. 12, 2017 Federal Register 
Notice (at page 82 FR 58437) for more details on the statistical test.
    HUD has also examined the FSS performance composite scores of PHAs 
to determine if small programs systematically receive lower composite 
scores and determined that, there is not a strong relationship between 
program size and composite FSS performance score. In fact, the decile 
of PHAs with the second smallest FSS programs (10th through 19th 
percentile) had the second highest median composite scores of any 
decile (the highest was the group of PHAs in the 70th through the 79th 
percentile in size). PHAs with the very smallest FSS programs (0 to 9th 
percentile) did have the lowest median composite score, but the next 
lowest score was recorded by PHAs in the 80th to 89th percentile in 
size. This is an indication that there is not a strong relationship 
between program size and composite FSS performance score. However, HUD 
may continue to monitor scores to determine if there are any patterns 
that might help with the targeting of technical assistance efforts or 
the interpretation of performance data.
    7. Comment: Joint Applicants. One commenter suggested that it would 
be more appropriate to pool joint applicant data for all measures, not 
just for participation.
    HUD Response: Joint Applicants. HUD agrees, and is changing the 
methodology accordingly.
    8. Comment: Initial Funding Period. Some commenters thought that 
FSS programs should not be assessed during their initial 12-month 
funding period or directly after receiving additional funding for the 
first time.
    HUD Response: Initial Funding Period. HUD agrees with the need to 
be careful in interpreting the FSS performance scores of newly funded 
FSS programs and will take this into account in determining how to use 
the scores. However, HUD believes it is important to measure the 
performance of all FSS programs that receive HUD coordinator funding so 
that programs have a way of tracking their performance over time. Also, 
since HUD has not funded new applicants in several years, all PHAs 
currently being scored have had programs funded since at least FY2012.
    9. Comment: Minimum Standards. A few commenters said that HUD 
should consider setting minimum standards for performance rather than 
rating FSS programs on a curve.

[[Page 57496]]

    HUD Response: Minimum Standards. FSS programs will not be graded on 
a curve, but rather based on whether or not they exceed the specific 
fixed standards (or thresholds) adopted in the final FSS performance 
measures. While HUD used percentiles of the distribution to determine 
the initial thresholds for each score, those thresholds have now been 
fixed. This means that over time, a PHA's scores may move up or down, 
based on where the PHA's earnings, FSS graduation, and participation 
measures fall relative to the thresholds. In other words, a PHA's 
performance will determine in which performance category the PHA falls, 
since there is not a set number of ``high'' or ``low'' performers.
    10. Comment: Zero Housing Assistance Payments (HAP). Some 
commenters suggested that attainment of a zero HAP amount (either at 
FSS graduation or in general) should be added as a performance measure.
    HUD Response: Zero Housing Assistance Payments (HAP). The ability 
of an FSS participant to reach a level of earnings at which his or her 
HAP amount drops to zero will depend to a significant degree on the 
local labor market and the level of the voucher payment standard, which 
is a function of the rental housing market as well as a PHA's policies. 
Since FSS participants in some markets have a much greater likelihood 
of achieving zero HAP than others, this measure does not provide a 
useful basis for comparing the performance of PHAs in different labor 
and housing markets.
    11. Comment: Unfunded PHAs, MTW PHAs, and PHAs that serve PBRA 
residents. HUD requested comments on the treatment of these types of 
PHAs and received many thoughtful comments on the development of 
performance measures for such PHAs.
    HUD Response: Unfunded PHAs, MTW PHAs, and PHAs that serve PBRA 
residents. HUD appreciates all the thoughtful comments received on 
these subjects and will be considering these comments as HUD works to 
determine how best to evaluate the performance of these programs.
    12. Comment: Portability. Some commenters were concerned about 
which PHA gets ``credit'' for FSS participants who port out of their 
PHA or into their PHA, although there was no consensus on how this 
should be addressed.
    HUD Response: Portability. If a family ports, for the Participation 
Measure, each PHA (the receiving and the initial PHA) will benefit from 
the family's FSS enrollment. For the earnings and FSS graduation 
measures, the composite score will count the family as a participant in 
the FSS program at the PHA who currently administers the FSS contract 
and thus has final influence on the family's outcomes.

B. Comments on Earnings Performance Measure

    1. Comment: Complexity of Earnings Performance Measure. Several 
commenters expressed a concern that the measures (especially the 
earnings measure) are too complicated or confusing. They indicated that 
PHAs will not understand them and will not be able to track their own 
progress. A few asked for information on which comparison households 
are included for their PHA so that they can track progress and correct 
data for those comparison households if needed. A few commenters 
expressed confusion about how comparison households are chosen and who 
chooses them.
    HUD Response: Complexity of Earnings Performance Measure. HUD 
acknowledges that the methodology for computing the earnings 
performance score is somewhat complex, but believes the complexity is 
justified as a means of adjusting for variations in local economic 
conditions and approaches (e.g., human capital development or ``work 
first'' or some combination) at different PHAs. Fortunately, however, 
the measure produces a single clear data point--the earnings 
performance measure--that PHAs can use to track their progress over 
time. To the extent that FSS programs are successful in helping 
participants to increase their earnings--whether in the short-term or 
in the long-term--they should be able to achieve a strong earnings 
performance score. For information on how the measure works and how 
comparison households are selected, see the December 12, 2017 Federal 
Register Notice (at pages 82 FR 58435-37) and comments below.
    2. Comment: Elderly Individuals and Persons with Disabilities. A 
few commenters suggested that excluding households headed by elderly 
persons or persons with disabilities from the earnings performance 
measure would discourage FSS programs from serving these households.
    HUD Response: Elderly Individuals and Persons with Disabilities. 
This comment provides a good opportunity to clarify that the 
methodology is designed to achieve the opposite effect. Although 
program regulations require FSS programs to serve any resident who 
desires to participate and is able to ``seek and maintain employment,'' 
see 24 CFR 984.303(b)(4), some FSS programs may be concerned that 
serving elderly persons and persons with disabilities would lower their 
earnings performance score because this population may be less likely 
to experience large earnings gains than other individuals. The 
methodology excludes households headed by elderly persons or persons 
with disabilities from the earnings performance measure, which ensures 
that PHAs can serve these households without worrying about the 
possibility that this might reduce their earnings performance score. 
All households served through FSS (regardless of age category or 
disability status) will be counted in the participation and FSS 
graduation measures.
    3. Comment: Changes in Elderly or Disability Status. One commenter 
asked how HUD will account for FSS participants who age out of the non-
elderly category while enrolled in FSS and those that acquire a 
disability while participating in the program. Will they be included or 
excluded from the analysis used to calculate the earnings performance 
measure?
    HUD Response: Changes in Elderly or Disability Status. Given the 
strong interest in and capacity for work of many adults in the 60 to 65 
age range, HUD believes it is appropriate to retain in the earnings 
analysis FSS participants who begin their FSS tenure below the age of 
62 but achieve that age during their participation. On the other hand, 
HUD agrees that a person whose status changes to ``disabled'' during 
the course of participation in FSS should be excluded from the earnings 
analysis in order to be consistent with the inclusion of data for other 
persons with disabilities in the earnings analysis. The methodology for 
calculating the earnings performance measure has thus been changed to 
exclude people who are or become disabled while participating in FSS 
from the analysis.
    4. Comment: Selecting Comparison Households. Many commenters 
expressed concern that the variables used to select comparison 
households were not sufficient to account for important life 
circumstances that may affect the potential for employment and 
increased earnings. The most common variables they recommended be 
included were: Language, education level, childcare availability, 
family composition (including children of all ages and workable adults 
or presence of a household member with a disability), mental health, 
and additional information about household composition. Some commenters 
also noted that FSS participants are different than non-FSS 
participants in terms of motivation, resources, or barriers to

[[Page 57497]]

employment, though there was disagreement among commenters on whether 
FSS participants are more likely to have high barriers or low barriers.
    HUD Response: Selecting Comparison Households. As described in the 
December 12, 2017 Federal Register Notice, in selecting comparison 
households for purposes of calculating the earnings performance 
measure, HUD considered the following household characteristics: 
Earnings as of the time of the FSS household's entry into FSS, age of 
head of household, length of time in the voucher or public housing 
program, number of adults in the household and number of children under 
age 5. While some of the additional factors recommended by commenters 
are not available in the PIC dataset used to compute the FSS 
performance measures, several are, including: presence of children of 
any age and presence of a household member with a disability.
    In response to this comment, HUD has considered whether the 
increased precision of adding additional comparison factors would 
outweigh the dilution of the weight of the existing factors and lead to 
an insufficient number of comparison households. Further analysis has 
determined that number of children under 18 is better than presence of 
children under age 5 in predicting whether a household would join FSS 
and therefore is a better factor in choosing comparison households. HUD 
will therefore remove presence of children under age 5 from the factors 
used to match comparison households and instead include number of 
children under 18.
    After further analysis, it has been determined that the presence of 
a child with a disability and presence of a non-head of household adult 
with a disability are not substantial factors predicting a household's 
choice to participate in FSS, but each of these factors is associated 
with a large and significant difference in a household's future 
earnings change. As a result, HUD will include both factors in 
selecting comparison households.
    5. Comment: Location of Comparison Households. A few commenters 
stated that households selected as comparisons for purposes of the 
earnings performance measure should be matched by similar census tract, 
neighborhood, or other measure of geography, to account for local 
variations in opportunity.
    HUD Response: Location of Comparison Households. HUD agrees that it 
would be preferable to select comparison households from the same 
geography as the FSS participants to which they are being compared but 
notes that this may be impossible to achieve at a very small level of 
geography, such as census tract or ZIP code, due to an insufficient 
number of comparison households, especially at small PHAs. Moreover, 
households in neighboring census tracts or ZIP codes are likely to 
still be in the same labor market, and thus can still be effective 
comparators.
    In PHAs that serve a very large geographical area, such as 
statewide PHAs, however, this point may not hold true since the 
economic conditions may be very different in different parts of the 
state. Accordingly, HUD plans to modify the protocol to require, under 
certain circumstances, that comparison households be in the same county 
and PHA as the FSS participants to which they are being compared. HUD 
will apply this protocol to all state PHAs and to non-State PHAs 
serving three or more counties where at least 10 percent of the PHA's 
housing choice voucher (HCV) or public housing households are leased in 
each of those counties. To ensure this approach does not unduly dilute 
the ability to find comparable households, HUD will require that FSS 
participants be matched to comparison households in the same county 
only in counties where there are at least four times as many non-FSS 
households as FSS households being served by the PHA.
    6. Comment: Shifts in Enrollment. Many commenters were concerned 
that the performance measures would encourage PHAs to recruit or enroll 
participants with a high probability of increases in earnings or 
chances of FSS graduation. This comment arose most often for the 
earnings measure, though commenters differed on whether this would lead 
to recruiting minimally employed participants so that they had room to 
grow or participants who are already somewhat financially successful 
and have high potential to increase salaries without much intervention. 
A few commenters raised the concern that FSS programs will stop serving 
participants with substantial barriers who are riskier for the earnings 
and FSS graduation measures and require more intensive intervention.
    HUD Response: Shifts in Enrollment. HUD appreciates these concerns 
and would remind PHAs of the requirement to open the program equally to 
all residents and administer the program for the residents who sign up 
for it, without trying to adjust enrollment to gain a higher score. As 
the commenters note, earnings gains among both unemployed participants 
and already employed participants can help boost a program's earnings 
performance score. It is also important to note that by regulation, FSS 
programs may screen families for interest and motivation to participate 
in the FSS program, but such programs are only permitted to screen for 
permissible motivational screening factors, i.e., those which solely 
measure the family's interest and motivation to participate in the FSS 
program. They may not exclude interested households based on other, 
prohibited characteristics.\2\
---------------------------------------------------------------------------

    \2\ 24 CFR 984.203(c).
---------------------------------------------------------------------------

    7. Comment: Variations in Economic Conditions. Some commenters 
raised the concern that the earnings measure advantages communities 
with higher wages and stronger employment opportunities (primarily 
urban areas) and disadvantages communities with lower wages and weaker 
employment opportunities (primarily rural and suburban areas).
    HUD Response: Variations in Economic Conditions. Because the 
earnings performance score is calculated based on the difference 
between the earnings growth of FSS participants and comparison 
households at the same PHA, it already controls to some extent for 
difference in economic conditions. Presumably, the comparison 
households at a PHA in a stronger economic market will experience 
greater earnings growth than the comparison households at a PHA in a 
weaker economic market, setting up a higher bar for FSS programs to 
exceed in the stronger market.
    Based on these comments, however, HUD has conducted additional 
analysis to determine if there are some residual effects of strong 
economic conditions that are not accounted for in this methodology and 
therefore a need to account for it in assigning earnings performance 
scores. This analysis found that there is in fact still a relationship 
between the earnings performance measures and county median income. 
Accordingly, HUD has decided to apply an adjustment factor to the 
earnings performance measure to account for the residual effect of 
local economic conditions.
    To compute this adjustment factor, HUD first used a linear 
regression model to examine the relationship between the earnings 
growth of comparison households within a PHA and the average county 
median income of those households. On average, earnings growth of 
comparison households was higher in counties with high median incomes, 
and lower in counties with low median incomes. HUD developed an 
adjustment factor that eliminated this relationship and then applied 
this

[[Page 57498]]

adjustment factor to the earnings performance measure for each PHA, 
resulting in an adjusted earnings performance measure.
    Using these adjusted earnings performance measures, HUD has 
recalculated the thresholds for awarding a 10, 7.5, or 0 earnings 
performance score by focusing on the 80th, 60th, and 20th percentile, 
respectively, of the distribution of adjusted measures. In selecting 
the revised thresholds, HUD has analyzed the distribution of scores 
across all funded PHAs, rather than the narrower universe described in 
the December 12, 2017 Federal Register Notice at 82 FR 58437 (the 
earlier notice included only PHAs whose earnings performance measures 
have a significant likelihood of being different from $0, per a 
statistical test). This makes the methodology more consistent with how 
HUD is calculating thresholds for the FSS graduation rate.
    8. Comment: Interim Earnings. Many commenters expressed the view 
that the results of interim reexaminations of income should be included 
in analyzing earnings growth because they capture seasonal income, and 
the most recent progress toward higher earnings. Several were also 
concerned that if participants reach a level of earnings where they no 
longer receive any HAP, this increase in earnings may only be captured 
by interim reexaminations and FSS exit reports.
    HUD Response: Interim Earnings. As noted in the December 12, 2017 
Federal Register Notice, HUD did not consider the earnings reported 
through interim reexaminations of income in the analysis of earnings 
gains because some PHAs conduct such reexaminations when income 
increases between annual reexaminations and others do not. Excluding 
these interim results thus facilitates a direct comparison of local FSS 
programs. Further, participants' incomes are not reexamined at the time 
of exit from FSS. While excluding interim reexaminations will mean 
missing certain earnings changes, such as when a family's earnings 
increase to the point where they are paying zero HAP, HUD has 
determined that their inclusion would make it difficult to compare 
results across PHAs, an essential element of the performance 
measurement system.
    9. Comment: Other Comments on the Earnings Measure. Most commenters 
agreed that averages were more appropriate than medians for the 
earnings measure. A few commenters stated that new employment and/or 
employment retention should be included as part of the earnings measure 
or in addition to the earnings measure. A few commenters suggested that 
escrow accumulation be included as part of or in addition to the 
earnings measure.
    HUD Response: Other Comments on the Earnings Measure. As noted in 
the December 12, 2017 Federal Register Notice (at page 82 FR 58438-39), 
HUD chose to focus on average earnings growth rather than median 
earnings growth to ensure that PHAs received credit for the major, 
transformative earnings gains experienced by some FSS participants, 
even if this experience was not typical of the whole population of FSS 
participants. HUD appreciates that most commenters agreed with this 
approach. However, HUD disagrees with adding new employment, employment 
retention, and escrow accumulation as additional measures or as part of 
the earnings measure. Households that experience new employment and 
escrow accumulation are likely to also experience increased earnings, 
since these measures are strongly related. Accordingly, the inclusion 
of these measures as additional measures would provide even heavier 
weight to earnings than is already the case, which HUD does not believe 
to be appropriate. HUD also notes that data on ``new employment'' is 
not currently collected (though HUD could make inferences about this 
from the PIC data) and that this measure could disadvantage PHAs that 
serve a population that generally enters FSS employed. Escrow is driven 
largely by earnings gains, though it is also affected by the loss of 
welfare assistance or other non-earnings income and thus is less 
precise than earnings in measuring earnings growth. Escrow accumulation 
also does not take into account earnings gains for households above 50 
percent of Area Medium Income (AMI), which is taken into consideration 
by the earnings measure currently in place. Additionally, until HUD has 
published a regulation or notice that implements Section 102 of Housing 
Opportunity Through Modernization Act of 2016 (HOTMA), residents who 
are subject to the Earned Income Disregard will have their escrow 
affected by that policy (in that escrow will not grow while income is 
disregarded for rent calculation purposes). While the current measure 
does not directly measure employment retention, it does factor it in 
since an FSS participant who retains his or her job while a comparison 
household does not will experience greater gains in earnings (zero) 
than the comparison household (a negative number), boosting the PHAs' 
average earnings performance score.

C. Comments on FSS Graduation Rate Measure

    1. Comment: FSS Graduation Rate. Many commenters were concerned 
that the inclusion of an FSS graduation rate measure would encourage 
PHAs to graduate families quickly instead of encouraging families to 
set ambitious employment goals in addition to the necessary 
requirements of maintaining entry level employment and being free of 
welfare cash assistance for twelve (12) months. Others noted that PHAs 
define/operationalize some of the FSS graduation standards differently 
from one another, so this measure would not be consistent across PHAs. 
A few commenters said that the FSS graduation measure penalizes 
programs for terminating non-compliant participants.
    HUD Response: FSS Graduation Rate. FSS graduation is an important 
milestone in the FSS program. FSS graduation marks the point at which 
FSS participants attain both their individual goals and the required 
program goals of employment and independence from welfare cash 
assistance. It also is the prerequisite for participants to receive the 
final disbursement from their escrow accounts.
    Together, the Earnings Performance Measure, Graduation Rate, and 
Participation Rate provide a balanced measurement of the performance of 
an individual FSS program. Because the Earnings Performance Measure is 
weighted more heavily than the Graduation Rate, PHAs should balance the 
need to graduate participants with setting ambitious employment goals 
so participants can maximize their earnings growth while in the 
program. In addition, while PHAs have the discretion to terminate the 
FSS participation of non-compliant participants, HUD would encourage 
PHAs to first work with participants to determine if their challenges 
can be addressed so participants can successfully complete the FSS 
program. Additional guidance can be found in the FSS Promising 
Practices Guidebook.

D. Comments on Participation Rate Measure

    1. Comment: Top Participation Scores. Many commenters expressed the 
view that having the top scores for participation substantially higher 
than the minimum a PHA is expected to serve with HUD funding is unfair 
and encourages PHAs to enroll more people than they can effectively 
serve. A few saw it as an unfunded mandate.

[[Page 57499]]

    HUD Response: Top Participation Scores. All PHAs that serve the 
minimum number of participants expected based on the level of HUD 
coordinator funding will receive at least a 5 as a participation score. 
If a PHA can attain strong earnings and FSS graduation results while 
exceeding this minimum, however, HUD wishes to encourage them to do so 
as this helps to maximize the number of families benefitting from the 
FSS program. This is the reason for assigning higher participation 
scores to PHAs that achieve higher participation levels. Since earnings 
is weighted much more heavily than participation, however, HUD 
emphasizes that PHAs should only increase their caseloads if and to the 
extent they can do so without undermining their earnings and FSS 
graduation results.
    HUD examined FSS performance data to determine if there is a 
correlation between a PHA's participation rate and its earnings and FSS 
graduation rate, paying particular attention to the participation rate 
threshold for obtaining a score of 10 points (80th percentile). This 
analysis did not find a strong relationship between participation rate 
and earnings performance measure. In fact, PHAs with participation 
rates between the 80th and 90th percentile had the highest average 
earnings performance measure of any decile and a median earnings 
performance measure that was typical for the sample as a whole, 
confirming that the threshold for obtaining a score of 10 points is not 
one that leads to lower earnings performance scores.
    In terms of FSS graduation rates, the median FSS graduation rate 
was fairly similar for most deciles of participation rate, except for 
the very highest and lowest deciles, which both had lower FSS 
graduation rates than the other deciles. However, the threshold for 
qualifying for 10 points on the participation rate is set at the 80th 
percentile and not the 90th percentile (the starting point for the 
highest decile) and PHAs with participation rates between the 80th and 
90th percentile had median and average FSS graduation rates that were 
typical for the sample as a whole, confirming that this threshold does 
not inherently lead to sub-par performance.
    Based on this analysis, HUD has determined that it is appropriate 
to encourage PHAs to adopt higher participation rates, so long as they 
can do so without compromising their earnings performance and FSS 
graduation rates. However, HUD has decided to change the final scoring 
so as to reward incremental improvements in participation rates, rather 
than only participation rates that exceed one of two specific 
thresholds. Accordingly, HUD will assign PHAs with participation rates 
above .95 a score of 5, 6, 7, 8, 9 or 10, depending on their 
participation rate, as specified in Section III of this notice. A score 
of 10 will be awarded for a participation rate at or above 2.0, which 
is close to the 80th percentile level HUD previously identified.
    2. Comment: Participation Rate and PHA Size. A few commenters said 
that the participation rate measure disadvantages either large PHAs/
programs or small ones. For small programs in small PHAs, there may be 
less opportunity to recruit participants and smaller economies of scale 
for the coordinator. For large programs, increases in the number of 
participants enrolled would have to be very large in order to increase 
the participation score.
    HUD Response: Participation Rate and PHA Size. The commenters are 
split about whether the participation rate calculation benefits smaller 
or larger PHAs. HUD believes this reflects the reality that all PHAs 
(regardless of size) have the potential to obtain either a high or a 
low participation rate, depending on how they manage their FSS program. 
This is confirmed by the fact that, in the initial spreadsheet of PHA 
scores, PHAs of all sizes are well represented at each of the 
participation score levels. While all PHAs must comply with the minimum 
enrollment requirements associated with the receipt of HUD coordinator 
funding, each PHA should make a determination of how many families they 
can serve effectively above this minimum based on their staff capacity, 
the intensity of participants' needs, and other resources available at 
the PHA and from partner organizations. HUD encourages PHAs to serve as 
many households as they can, so long as they do not exceed the level 
they can effectively support. Additionally, as explained above, there 
is no clear correlation between a PHA's size and the overall composite 
score.

E. Comments on Weighting of the Measures

    1. Comment: Weighting. Several commenters felt that the weights are 
appropriate and did not comment further. Many commenters expressed the 
view that the earnings measure is weighted too highly. Commenters who 
suggested this were often concerned that the earnings measure would not 
show progress for FSS participants in longer-running education or 
training programs and so, did not account for variations in participant 
goals. Some commenters felt that FSS graduation and participation 
should have the same weight, regardless of the weight of the earnings 
measure. One reason given for this is that participation is essential 
for FSS graduation. Another was that weighting FSS graduation rate too 
highly compared to participation would encourage PHAs to graduate 
families before they had met ambitious goals.
    HUD Response: Weighting. HUD appreciates the range of views 
expressed on this matter. After considering the comments, HUD plans to 
retain the weighting specified in the December 12, 2017 Federal 
Register Notice. Earnings represent by far the most powerful and 
objective measure available to HUD. While there are many goals to which 
FSS participants aspire, the achievement of most of these should lead 
to higher earnings which can then be measured through the earnings 
performance measure. Accordingly, HUD believes that a weight of 50 
percent is appropriate.
    While there is a case for weighting FSS graduation rate and 
participation rate equally, HUD believes weights of 30 percent for the 
FSS graduation rate and 20 percent of the participation rate are 
appropriate. As noted above, FSS graduation is an important milestone 
for the FSS program and HUD would like to see PHAs raise FSS graduation 
rates. HUD would also like to see PHAs serve more families if and to 
the extent they can do so without jeopardizing their achievement of 
strong earnings and FSS graduation rates. Weighting FSS graduation rate 
more heavily than participation rate is consistent with HUD's goal of 
not creating incentives for PHAs to raise caseloads beyond the point 
where families can be served effectively.

III. Final Thresholds

A. Summary of Adjustments to FSS Performance Score Methodology

    After considering all of the public comments, HUD is adopting the 
proposed FSS performance measurement system, with the adjustments noted 
above, which will henceforth be used by HUD to evaluate the performance 
of PHAs receiving HUD program coordinator funding. These adjustments 
are summarized in the table below:

[[Page 57500]]



                           Changes to Methodology for Computing FSS Performance Scores
----------------------------------------------------------------------------------------------------------------
 
----------------------------------------------------------------------------------------------------------------
Overall.............................................   Where a family ports, each PHA (the receiving and
                                                       the initial PHA) will benefit from the family's FSS
                                                       enrollment as it relates to the PHA's participation
                                                       measure. For the earnings and FSS graduation measures,
                                                       HUD will include the family for the PHA who currently
                                                       administers the FSS contract.
                                                       HUD will treat joint applicants as a single PHA
                                                       for purposes of computing all three components of the FSS
                                                       performance score.
Earnings Performance Score..........................   In calculating the earnings performance score,
                                                       HUD will exclude FSS participants who become classified
                                                       as disabled at any point during their participation.
                                                       HUD will include within the earnings measure FSS
                                                       participants that begin the FSS contract below age 62,
                                                       even if they reach or exceed the age of 62 during their
                                                       Contract of Participation.
                                                       In selecting comparison households, HUD will
                                                       match FSS families with comparison families based on the
                                                       number of children under the age of 18, rather than the
                                                       presence of child under age 5. HUD will also match FSS
                                                       families with comparison families based on presence of a
                                                       child with a disability and presence of a non-head of
                                                       household adult with a disability.
                                                       Under certain circumstances, HUD will require
                                                       that comparison households be in the same county and PHA
                                                       as the FSS participants to which they are being compared.
                                                       HUD will apply this protocol to all state PHAs and to any
                                                       additional PHAs where three or more counties are each
                                                       home to at least 10 percent of households receiving
                                                       housing assistance from the PHA (through HCV or public
                                                       housing). To ensure this approach does not unduly dilute
                                                       the ability to find comparable households, HUD will
                                                       require that FSS participants be matched to comparison
                                                       households in the same county only in counties where
                                                       there are at least four times as many non-FSS households
                                                       as FSS households being served by the PHA.
                                                       HUD will apply an adjustment factor to the
                                                       earnings performance measure to account for variations in
                                                       local economic conditions.
----------------------------------------------------------------------------------------------------------------

    After making these adjustments to the methodology, HUD has 
recalculated the thresholds for translating the FSS performance 
measures into individual component scores and the final composite score 
and notes the final thresholds below.

B. Updated Thresholds for FSS Performance Scores

    The following are the updated thresholds HUD will use to compute an 
FSS Performance Score for each PHA. See the December 12, 2017 Federal 
Register Notice and the updated complete methodology, which can be 
found on HUD's website at https://www.hud.gov/program_offices/public_indian_housing/programs/hcv/fss, for more information on each of 
the two steps in this process.
1. Step One: Assigning Scores to Each of the Three Measures
    In Step One, HUD will assign a score of 0 to 10 to each PHA's FSS 
program for each of the three measures. Scores will be assigned using 
the thresholds and procedures described below. The ranges for awarding 
points between two values include those values as well as all 
intermediary values.
    a. Earnings Performance Measure (50 percent of final score):
     10 points: Earnings performance measure of $8,700 or 
higher.
     7.5 points: Earnings performance measure between $6,950 
and $8,699.99.
     0 points: Earnings performance measure below $4,050 and a 
p-value of <.10 on a statistical test measuring the likelihood that a 
PHA's earnings performance measure is significantly lower than the 
median measure of $6,302 (see December 12, 2017 Federal Register Notice 
at page 82 FR 58437 for an explanation of this statistical test).
     5 points: All PHAs that do not qualify for a 10, 7.5, or a 
0.
    b. FSS Graduation Rate (30 percent of final score):
     10 points: FSS graduation rate of 38 percent or higher.
     7.5 points: FSS graduation rate between 28 percent and 
37.9 percent.
     0 points: FSS graduation rate below 10 percent.
     5 points: All PHAs that do not qualify for a 10, 7.5, or a 
0
    c. Participation Rate (20 percent of final score):
     10 points: Participation rate of 2.0 or higher.
     9 points: Participation rate between 1.8 and 1.99.
     8 points: Participation rate between 1.6 and 1.79.
     7 points: Participation rate between 1.4 and 1.59.
     6 points: Participation rate between 1.2 and 1.39.
     5 points: Participation rate between .96 and 1.19.
     0 points: Participation rate of .95 or lower.
2. Step Two: Developing the Final FSS Performance Score and Grade
    In Step Two, after computing individual scores for each of the 
three measures, HUD will aggregate each PHA's scores using the weights 
noted above to develop a final FSS Performance Score from 0 to 10. 
Based on this score, HUD will assign the following ranking to the PHA's 
performance:
     Category 1: FSS Performance score of 8.0 or higher.
     Category 2: FSS Performance score between 4.26 and 7.99.
     Category 3: FSS Performance score between 3.26 and 4.25.
     Category 4: FSS Performance score of 3.25 or lower.

IV. Environmental Impact

    This notice does not direct, provide for assistance or loan and 
mortgage insurance for, or otherwise govern or regulate, real property 
acquisition, disposition, leasing, rehabilitation, alteration, 
demolition, or new construction, or establish, revise or provide for 
standards for construction or construction materials, manufactured 
housing, or occupancy. Accordingly, under 24 CFR 50.19(c)(1), this 
notice is categorically excluded from environmental review under the 
National Environmental Policy Act of 1969 (42 U.S.C. 4321).

    Dated: November 7, 2018.
Dominique Blom,
General Deputy Assistant Secretary, Public and Indian Housing.
[FR Doc. 2018-24949 Filed 11-14-18; 8:45 am]
 BILLING CODE 4210-67-P