Proposed Model Performance Measures for State Traffic Records Systems, 20438-20448 [2011-8738]

Agencies

• National Highway Traffic Safety Administration
• DEPARTMENT OF TRANSPORTATION

[Federal Register Volume 76, Number 70 (Tuesday, April 12, 2011)]
[Notices]
[Pages 20438-20448]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2011-8738]


-----------------------------------------------------------------------

DEPARTMENT OF TRANSPORTATION

National Highway Traffic Safety Administration

[Docket No. NHTSA-2011-0044]


Proposed Model Performance Measures for State Traffic Records 
Systems

AGENCY: National Highway Traffic Safety Administration (NHTSA), 
Department of Transportation (DOT).

ACTION: Notice

-----------------------------------------------------------------------

SUMMARY: This notice announces the publication of Model Performance 
Measures for State Traffic Records Systems DOT HS 811 44, which 
proposes model performance measures for State traffic record systems to 
monitor the development and implementation of traffic record data

[[Page 20439]]

systems, strategic plans, and data-improvement grants. These model 
performance measures are voluntary and are to help States monitor and 
improve the quality of the data in their traffic record systems

DATES: Written comments may be submitted to this agency and must be 
received no later than June 13, 2011.

ADDRESSES: You may submit comments identified by DOT Docket ID number 
NHTSA-2011-0044 by any of the following methods:
     Electronic Submissions: Go to https://www.regulations.gov. 
Follow the online instructions for submitting comments.
     Fax: 202-366-2746.
     Mail: Docket Management Facility, M-30 U.S. Department of 
Transportation, West Building, Ground Floor, Room W12-140, 1200 New 
Jersey Ave., SE., Washington, DC 20590.
     Hand Delivery or Courier: Docket Management Facility, M-30 
U.S. Department of Transportation, West Building, Ground Floor, Room 
W12-140, 1200 New Jersey Ave., SE., Washington, DC 20590, between 9 
a.m. and 5 p.m. Eastern time, Monday through Friday, except Federal 
holidays.
    Regardless of how you submit your comments, you should identify the 
Docket number of this document.
    Instructions: For detailed instructions on submitting comments and 
additional information, see https://www.regulations.gov. Note that all 
comments received will be posted without change to https://www.regulations.gov, including any personal information provided. 
Please read the ``Privacy Act'' heading below.
    Privacy Act: Anyone is able to search the electronic form of all 
contents received into any of our dockets by the name of the individual 
submitting the comment (or signing the comment, if submitted on behalf 
of an association, business, labor union, etc.). You may review the 
complete User Notice and Privacy Notice for Regulations.gov at https://www.regulations.gov/search/footer/privacyanduse.jsp.
    Docket: For access to the docket to read background documents or 
comments received, go to https://www.regulations.gov at any time or to 
West Building Ground Floor, Room W12-140, 1200 New Jersey Avenue, SE., 
Washington, DC between 9 a.m. and 5 p.m., Eastern Time, Monday through 
Friday, except Federal holidays.

FOR FURTHER INFORMATION CONTACT: For programmatic issues: Luke Johnson, 
Office of Traffic Records and Analysis, NPO-423, National Highway 
Traffic Safety Administration, 400 Seventh Street, SW., Washington, DC 
20590. Telephone (202) 366-1722. For legal issues: Roland Baumann, 
Office of Chief Counsel, NCC-113, National Highway Traffic Safety 
Administration, 400 Seventh Street, SW., Washington, DC 20590. 
Telephone (202) 366-5260.

SUPPLEMENTARY INFORMATION: The National Highway Traffic Safety 
Administration (NHTSA) has identified 61 model performance measures for 
the six core State traffic records data systems: Crash, vehicle, 
driver, roadway, citation/adjudication, and EMS/injury surveillance. 
These model performance measures address the six performance 
attributes: Timeliness, accuracy, completeness, uniformity, 
integration, and accessibility. States can use these measures to 
develop and track performance goals in their Traffic Records Strategic 
Plans, Traffic Records Assessments, and Highway Safety Plans; establish 
data-quality improvement measures for specific traffic records 
projects; and support data improvement goals in the Strategic Highway 
Safety Plan. The full text of the report Model Performance Measures for 
State Traffic Records Systems DOT HS 811 44, is available at https://www.nhtsa.gov/.

Key Features of the Model Performance Measures

    Use is voluntary: States should use the measures for those data 
system performance attributes they wish to monitor or improve. If the 
suggested measures are not deemed appropriate, States are free to 
modify them or develop their own.
    The measures are flexible: The measures are models. States can 
modify a measure to meet a specific need as long as its overall intent 
remains the same.
    The measures do not set numerical performance goals: They describe 
what to measure and suggest how it should be measured but are not 
intended to establish a numerical performance goal. Each State should 
set its own performance goals.
    The measures provide a template or structure States can populate 
with specific details: For example, the States must decide what data 
files to use and what data elements are critical. States should take 
advantage of these decision-making opportunities to focus on their most 
important performance features.
    The measures are not exhaustive: The measures attempt to capture 
one or two key performance features of each data system performance 
attribute. States may wish to use additional or alternative measures to 
address specific performance issues.
    The measures are not intended to be used to compare States: Their 
purpose is to help each State improve its own performance. Each State 
selects the measures it uses, establishes its own definitions of key 
terms, and may modify the measures to fit its circumstances. Since the 
measures will vary considerably from State to State, it is unlikely 
that they could be used for any meaningful comparisons between States. 
NHTSA has no intention of using the measure to make interstate 
comparisons.

Core Traffic Records Data Systems

    The model performance measures cover the six core traffic data 
systems.
    1. Crash: The State repository that stores law enforcement officer 
crash reports.
    2. Vehicle: The State repository that stores information on 
registered vehicles within the State (also known as the vehicle 
registration system). This database can also include records for 
vehicles not registered in the State--e.g., a vehicle that crashed in 
the State but registered in another State.
    3. Driver: The State repository that stores information on licensed 
drivers within the State and their driver histories. This is also known 
as the driver license and driver history system. The driver file also 
could contain a substantial number of records for drivers not licensed 
within the State--e.g., an unlicensed driver involved in a crash.
    4. Roadway: The State repository that stores information about the 
roadways within the State. It should include information on all 
roadways within the State and is typically composed of discrete sub-
files that include: Roadway centerline and geometric data, location 
reference data, geographical information system data, travel and 
exposure data, etc.
    5. Citation/Adjudication: The component repositories, managed by 
multiple State or local agencies, which store traffic citation, arrest, 
and final disposition of charge data.
    6. EMS/Injury Surveillance: The component repositories, managed by 
multiple State or local agencies, which store data on motor vehicle-
related injuries and deaths. Typical components of an EMS/injury 
surveillance system are pre-hospital EMS data, hospital emergency 
department data systems, hospital discharge data systems, trauma 
registries, and long term care/rehabilitation patient data systems.

[[Page 20440]]

Performance Attributes

    The model performance measures are based on six core 
characteristics:
    1. Timeliness: Timeliness reflects the span of time between the 
occurrence of an event and entry of information into the appropriate 
database. Timeliness can also measure the time from when the custodial 
agency receives the data to the point when the data are entered into 
the database.
    2. Accuracy: Accuracy reflects the degree to which the data are 
error-free, satisfy internal consistency checks, and do not exist in 
duplicate within a single database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. In some cases, it is possible to 
determine that the values entered for a variable or data element are 
not legitimate codes. In other cases, errors can be detected by 
matching with external sources of information. It may also be possible 
to determine that duplicate records have been entered for the same 
event (e.g., title transfer).
    3. Completeness: Completeness reflects both the number of records 
that are missing from the database (e.g., events of interest that 
occurred but were not entered into the database) and the number of 
missing (blank) data elements in the records that are in a database. In 
the crash database, internal completeness reflects the amount of 
specified information captured in each individual crash record. 
External crash completeness reflects number or percentage of crashes on 
which crash reports are entered into the database. However, it is not 
possible to determine precisely external crash completeness as it is 
impossible to determine the number of unreported crashes. The measures 
in this report only address internal completeness by measuring what is 
not missing.
    4. Uniformity: Uniformity reflects the consistency among the files 
or records in a database and may be measured against some independent 
standard, preferably a national standard. Within a State all 
jurisdictions should collect and report the same data using the same 
definitions and procedures. If the same data elements are used in 
different State files, they should be identical or at least compatible 
(e.g., names, addresses, geographic locations). Data collection 
procedures and data elements should also agree with nationally accepted 
guidelines and standards (such as the Model Minimum Uniform Crash 
Criteria, [MMUCC]).
    5. Integration: Integration reflects the ability of records in a 
database to be linked to a set of records in another of the six core 
databases--or components thereof--using common or unique identifiers. 
Integration differs in one important respect from the first four 
attributes of data quality. By definition, integration is a performance 
attribute that always involves two or more traffic records subsystems 
(i.e., databases or files). For integration, the model performance 
measures offer a single performance measure with database-specific 
applications that typically are of interest to many States. The samples 
included are of course non-exhaustive. Many States will be interested 
in establishing links between databases and sub-databases other than 
those listed here, and therefore will be interested in measuring the 
quality of those other integrations. Note that some of the specific 
examples herein involve integration of files within databases rather 
than the integration of entire databases.
    6. Accessibility: Accessibility reflects the ability of legitimate 
users to successfully obtain desired data is different. For the other 
performance attributes, the owners and operators of the various 
databases and sub-files, examine the data in the files and the internal 
workings of the files. In contrast, accessibility is measured in terms 
of customer satisfaction. Every database and file in a traffic records 
system has a set of legitimate users who are entitled to request and 
receive data. The accessibility of the database or sub-file is 
determined by obtaining the users' perceptions of how well the system 
responds to their requests. Some users' perceptions may be more 
relevant to measurement of accessibility than others'. Each database 
manager should decide which of the legitimate users of the database 
would be classified as principal users, whose satisfaction with the 
system's response to requests for data and other transactions will 
provide the basis for the measurement of accessibility. Thus, the 
generic approach to measurement of database accessibility in the model 
performance measured by (1) identifying the principal users of the 
database; (2) Querying the principal users to assess (a) their ability 
to obtain the data or other services requested and (b) their 
satisfaction with the timeliness of the response to their request; and 
(3) documenting the method of data collection and the principal users' 
responses. How the principal users are contacted and queried is up to 
the database managers. Similarly, the extent to which the principal 
users' responses are quantified is left to the managers to determine. 
However, this measure does require supporting documentation that 
provides evidentiary support to the claims of accessibility. This 
measure would be best used to gauge the impact of an improvement to a 
data system. Surveying the principal users before and after the rollout 
of a specific upgrade would provide the most meaningful measure of 
improved database accessibility.

Performance Measure Criteria

    Each model performance measure was developed in accordance with the 
following criteria:
    Specific and well-defined: The measures are appropriate and 
understandable.
    Performance based: The measures are defined by data system 
performance, not supporting activities or milestones: ``awarded a 
contract'' or ``formed a Traffic Records Coordinating Committee'' are 
not acceptable performance measures.
    Practical: The measures use data that are readily available at 
reasonable cost and can be duplicated.
    Timely: The measures should provide an accurate and current--near 
real-time--snapshot of the database's timeliness, accuracy, 
completeness, uniformity, integration, and accessibility.
    Accurate: The measures use data that are valid and consistent with 
values that are properly calculated.
    Important: The measures capture the essence of this performance 
attribute for the data system; for example, an accuracy measure should 
not be restricted to a single unimportant data element.
    Universal: The measures are usable by all States, though not 
necessarily immediately.
    These criteria take a broad view of performance measures. For 
example, performance on some of the model measures may not change from 
year to year. Once a State has incorporated uniform data elements, 
established data linkages, or provided appropriate data file access, 
further improvement may not be expected. Some States cannot use all 
measures. For example, States that do not currently maintain a 
statewide data file cannot use measures based on this file (see in 
particular the injury data files). Some measures require States to 
define a set of critical data elements. Many measures require States to 
define their own performance goals or standards. The model measures 
should be a guide for States as they assess their data systems and work 
to improve their

[[Page 20441]]

performance. Each State should select performance measures most 
appropriate to the circumstance and should define and modify them to 
fit their specific needs.

Performance Measures

    Listed below are the 61 measures classified by data system and 
performance attribute.

Crash--Timeliness

    Timeliness always reflects the span of time between the occurrence 
of some event and the entry of information from the event into the 
appropriate database. For the crash database, the events of interest 
are crashes. States must measure the time between the occurrence of a 
crash and the entry of the report into the crash database. The model 
performance measures offer two approaches to measuring the timeliness 
of a crash database:
    C-T-1: The median or mean number of days from (A) the crash date to 
(B) the date the crash report is entered into the database. The median 
value is the point at which 50 percent of the crash reports were 
entered into the database within a period defined by the State. 
Alternatively, the arithmetic mean could be calculated for this 
measure.
    C-T-2: The percentage of crash reports entered into the database 
within XX days after the crash. The XX usually reflects a target or 
goal set by the State for entry of reports into the database. The 
higher percentage of reports entered within XX days, the timelier the 
database. Many States set the XX for crash data entry at 30, 60, or 90 
days but any other target or goal is equally acceptable.

Crash--Accuracy

    Accuracy reflects the number of errors in information in the 
records entered into a database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. Methods for detecting errors 
include: (1) Determining that the values entered for a variable or 
element are not legitimate codes, (2) matching with external sources of 
information, and (3) identifying duplicate records entered for the same 
event. The model performance measures offer two approaches to measuring 
crash database accuracy:
    C-A-1: The percentage of crash records with no errors in critical 
data elements. The State selects one or more crash data elements it 
considers critical and assesses the accuracy of that element or 
elements in all of the crash records entered into the database within a 
period defined by the State. Many States consider the following crash 
elements critical:
    Environmental elements: Record , Location (on/at/distance 
from; lat/long, location code), Date, time (can calculate day of week 
from this too), Environment contributing factors (up to 3) Location 
description (roadway type, location type, roadway-contributing 
factors--up to 3) Crash type, severity,  involved units, 
Harmful events (first harmful, most harmful).
    Vehicle/Unit elements: Crash record , vehicle/unit 
, VIN decoded sub-file of values for make, model, year, other 
decode values, Sequence of events (multiple codes), Harmful events (1st 
and most harmful for each vehicle), SafetyNet variables for reportable 
vehicles/crashes (carrier name/ID, additional vehicle codes, tow away 
due to damage).
    Person elements: Crash record , vehicle/unit , 
person , Person type (driver, occupant, non-occupant), 
Demographics (age, sex, other), Seating position, Protective device 
type (occupant protection, helmet, etc.), Protective device use, Airbag 
(presence, deployment: Front, side, both, none), Injury severity (if 
this can be sourced through EMS/Trauma/hospital records.
    C-A-2: The percentage of in-State registered vehicles on the State 
crash file with Vehicle Identification Number (VIN) matched to the 
State vehicle registration file.

Crash--Completeness

    Completeness reflects both the number of records that are missing 
from the database (e.g., events of interest that occurred but were not 
entered into the database) and the number of missing (blank) data 
elements in the records that are in a database. Completeness has 
internal and external aspects. In the crash database, external crash 
completeness reflects the number or percentage of crashes for which 
crash reports are entered into the database. It is impossible, however, 
to establish precisely external crash completeness as the number of 
unreported crashes cannot be determined. Internal completeness can be 
determined since it reflects the amount of specified information 
captured in each individual crash record. The model performance 
measures offer three approaches to measuring the internal completeness 
of a crash database:
    C-C-1: The percentage of crash records with no missing critical 
data elements. The State selects one or more crash data elements it 
considers critical and assesses internal completeness by dividing the 
number of records not missing a critical element by the total number of 
records entered into the database within a period defined by the State.
    C-C-2: The percentage of crash records with no missing data 
elements. The State can assess overall completeness by dividing the 
number of records missing no elements by the total number of records 
entered into the database within a period defined by the State.
    C-C-3: The percentage of unknowns or blanks in critical data 
elements for which unknown is not an acceptable value. This measure 
should be used when States wish to track improvements on specific 
critical data values and reduce the occurrence of illegitimate null 
values.

Crash--Uniformity

    Uniformity reflects the consistency among the files or records in a 
database and may be measured against some independent standard, 
preferably a national standard. The model performance measures offer 
one approach to measure crash database uniformity:
    C-U-1: The number of MMUCC-compliant data elements entered into the 
crash database or obtained via linkage to other database(s). The Model 
Minimum Uniform Crash Criteria (MMUCC) Guideline is the national 
standard for crash records.

Crash-Integration

    Integration reflects the ability of records in the crash database 
to be linked to a set of records in another of the six core databases--
or components thereof--using common or unique identifiers.
    C-I-1: The percentage of appropriate records in the crash database 
that are linked to another system or file. Linking the crash database 
with the five other core traffic records databases can provide 
important information. For example, a State may wish to determine the 
percentage of in-State drivers on crash records that link to the driver 
file.

Crash-Accessibility

    Accessibility reflects the ability of legitimate users to 
successfully obtain desired data. The below process outlines one way of 
measuring crash database accessibility:
    C-X-1: To measure crash accessibility: (1) Identify the principal 
users of the crash database; (2) Query the principal users to assess 
(A) their ability to obtain the data or other services requested and 
(B) their satisfaction with the timeliness of the

[[Page 20442]]

response to their request; (3) Document the method of data collection 
and the principal users' responses.

Vehicle-Timeliness

    Timeliness always reflects the span of time between the occurrence 
of some event and the entry of information from the event into the 
appropriate database. For the vehicle database, the State determines 
the events of principal interest that will be used to measure 
timeliness. For example, a State may determine that the transfer of the 
title of the vehicle constitutes a critical status change of that 
vehicle record. There are many ways to measure the timeliness of the 
entry of a report on the transfer of a vehicle title or any other 
critical status change. The model performance measures offer two 
general approaches to measuring vehicle database timeliness:
    V-T-1: The median or mean number of days from (A) the date of a 
critical status change in the vehicle record to (B) the date the status 
change is entered into the database. The median value is the point at 
which 50 percent of the vehicle record updates were entered into the 
database within a period defined by the State. Alternatively, the 
arithmetic mean could be calculated for this measure.
    V-T-2: The percentage of vehicle record updates entered into the 
database within XX days after the critical status change. The XX 
usually reflects a target or goal set by the State for entry of reports 
into the database. The higher percentage of reports entered within XX 
days, the timelier is the database. Many States set the XX for vehicle 
data entry at one, five, or 10 days, but any target or goal is equally 
acceptable.

Vehicle-Accuracy

    Accuracy reflects the number of errors in information in the 
records entered into a database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. Methods for detecting errors 
include: (1) Determining that the values entered for a variable or 
element are not legitimate codes, (2) matching with external sources of 
information, and (3) identifying duplicate records have been entered 
for the same event. The model performance measures offer one approach 
to measuring vehicle database accuracy:
     V-A-1: The percentage of vehicle records with no errors in 
critical data elements. The State selects one or more vehicle data 
elements it considers critical and assesses the accuracy of that 
element or elements in all of the vehicles records entered into the 
database within a period defined by the State. Many Stats have 
identified the following critical data elements: Vehicle Identification 
Number (VIN), Current registration status, Commercial or non-CMV, State 
of registration, State of title, Stolen flag (as appropriate), Motor 
carrier name, Motor carrier ID, and Title brands.

Vehicle-Completeness

    Completeness has internal and external aspects. For the vehicle 
database, external vehicle completeness reflects the portion of the 
critical changes to the vehicle status reported and entered into the 
database. It is not possible to determine precisely external vehicle 
database completeness because one can never know how many critical 
status changes occurred but went unreported. Internal completeness 
reflects the amount of specified information captured by individual 
vehicle records. It is possible to determine precisely internal vehicle 
completeness; for example, one can calculate the percentage of vehicle 
records in the database that is missing one or more critical data 
elements. The model performance measures offer four approaches to 
measuring the completeness of a vehicle database:
    V-C-1: The percentage of vehicle records with no missing critical 
data elements. The State selects one or more vehicle data elements it 
considers critical and assesses internal completeness by dividing the 
number of records not missing a critical element by the total number of 
records entered into the database within a period defined by the State.
    V-C-2: The percentage of records on the State vehicle file that 
contain no missing data elements. The State can assess overall 
completeness by dividing the number of records missing no elements by 
the total number of records entered into the database within a period 
defined by the State.
    V-C-3: The percentage of unknowns or blanks in critical data 
elements for which unknown is not an acceptable value. This measure 
should be used when States wish to track improvements on specific 
critical data values to reduce the occurrence of illegitimate null 
values.
    V-C-4: The percentage of vehicle records from large trucks and 
buses that have all of the following data elements: Motor Carrier ID, 
Gross Vehicle Weight Rating/Gross Combination Weight Rating, Vehicle 
Configuration, Cargo Body Type, and Hazardous Materials (Cargo Only). 
This is a measure of database completeness in specific critical fields.

Vehicle-Uniformity

    Uniformity reflects the consistency among the files or records in a 
database and may be measured against some independent standard, 
preferably a national standard. The model performance measures offer 
one general approach to measuring vehicle database uniformity.
    V-U-1: The number of standards-compliant data elements entered into 
a database or obtained via linkage to other database(s). These 
standards include the Model Minimum Uniform Crash Criteria (MMUCC).

Vehicle-Integration

    Integration reflects the ability of records in the vehicle database 
to be linked to a set of records in another of the six core databases--
or components thereof--using common or unique identifiers.
    V-I-1: The percentage of appropriate records in the vehicle file 
that are linked to another system or file. Linking the vehicle database 
with the five other core traffic record databases can provide important 
information. For example, a State may wish to determine the percentage 
of vehicle registration records that link to a driver record.

Vehicle-Accessibility

    Accessibility reflects the ability of legitimate users to 
successfully obtain desired data. The below process outlines one way of 
measuring the vehicle database's accessibility.
    V-X-1: To measure accessibility: (1) Identify the principal users 
of the vehicle database; (2) Query the principal users to assess (A) 
their ability to obtain the data or other services requested and (B) 
their satisfaction with the timeliness of the response to their 
request; (3) Document the method of data collection and the principal 
users' responses.

Driver-Timeliness

    Timeliness always reflects the span of time between the occurrence 
of some event and the entry of information from the event into the 
appropriate database. For the driver database, the State determines the 
events of principal interest that shall be used to measure timeliness. 
For example, the State may determine that an adverse action against a 
driver's license constitutes a critical status change of that driver 
record. There are many ways to measure the timeliness of the entry of a 
report on an adverse action against a driver's license or any other 
critical status change. The

[[Page 20443]]

model performance measures offer two approaches to measuring the 
timeliness of the driver database. The first is a true measure of 
timeliness from time of conviction to entry into the driver database, 
while the second is a measure internal to the agency with custody of 
the driver database.
    D-T-1: The median or mean number of days from (A) the date of a 
driver's adverse action to (B) the date the adverse action is entered 
into the database. This measure represents the time from final 
adjudication of a citation to entry into the driver database within a 
period defined by the State. This process can occur in a number of 
ways, from the entry of paper reports and data conversion to a seamless 
electronic process. An entry of a citation disposition into the driver 
database cannot occur until the adjudicating agency (usually a court) 
notifies the repository that the disposition has occurred. Since the 
custodial agency of the driver database in most States has no control 
over the transmission of the disposition notification many States may 
wish to track the portion of driver database timelines involving 
citation dispositions that it can control. Measure D-T-2 is offered for 
that purpose.
    D-T-2: The median or mean number of days from (A) the date of 
receipt of citation disposition notification by the driver repository 
to (B) the date the disposition report is entered into the driver's 
record in the database within a period determined by the State. This 
measure represents the internal (to the driver database) time lapse 
from the receipt of disposition information to entry into the driver 
database within a period defined by the State.

Driver-Accuracy

    Accuracy reflects the number of errors in information in the 
records entered into a database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. Methods for detecting errors 
include: (1) Determining that the values entered for a variable or 
element are not legitimate codes, (2) matching with external sources of 
information, and (3) identifying duplicate records have been entered 
for the same event. The model performance measures offer two approaches 
to measuring driver database accuracy:
    D-A-1: The percentage of driver records with no errors in critical 
data elements. The State selects one or more driver data elements it 
considers critical and assesses the accuracy of that element or 
elements in all of the driver records entered into the database within 
a period defined by the State. Several States have identified the 
following critical data elements: Name, Date of birth, Sex, Driver 
license number, State of driver license issuance, Date license issued 
or renewed, Social Security Number, License type, Restrictions, Crash 
involvement, Conviction offenses, Violation date per event, Conviction 
date per event, Driver control actions (Suspensions, Revocations, 
Withdrawals), and Date of each action.
    D-A-2: The percentage of records on the State driver file with 
Social Security Numbers (SSN) successfully verified using Social 
Security Online Verification (SSOLV) or other means.

Driver-Completeness

    Completeness has internal and external aspects. For the driver 
database, external completeness reflects the portion of critical driver 
status changes that are reported and entered into the database. It is 
not possible to determine precisely the external completeness of driver 
records because one can never know how many critical driver status 
change occurred but went unreported. Internal completeness reflects the 
amount of specified information captured in individual driver records. 
It is possible to determine precisely internal driver record 
completeness. One can, for example, calculate the percentage of driver 
records in the database that is missing one or more critical data 
elements. The model performance measures offer three approaches to 
measuring the internal completeness of the driver database:
    D-C-1: The percentage of driver records with no missing critical 
data elements. The State selects one or more driver elements it 
considers critical and assesses internal completeness by dividing the 
number of records not missing a critical element by the total number of 
records entered into the database within a period defined by the State.
    D-C-2: The percentage of driver records with no missing data 
elements. The State can assess overall completeness by dividing the 
number of records missing no elements by the total number of records 
entered into the database within a period defined by the State.
    D-C-3: The percentage of unknowns or blanks in critical data 
elements for which unknown is not an acceptable value. This measure 
should be used when States wish to track improvements on specific 
critical data values and reduce the occurrence of illegitimate null 
values.

Driver-Uniformity

    Uniformity reflects the consistency among the files or records in a 
database and may be measured against an independent standard, 
preferably a national standard. The model performance measures offer 
one general approach to measuring driver database uniformity:
    D-U-1: The number of standards-compliant data elements entered into 
the driver database or obtained via linkage to other database(s). The 
relevant standards include MMUCC.

Driver-Integration

    Integration reflects the ability of records in the driver database 
to be linked to a set of records in another of the six core databases--
or components thereof--using common or unique identifiers.
    D-I-1: The percentage of appropriate records in the driver file 
that are linked to another system or file. Linking the driver database 
with the five other core traffic record databases can provide important 
information. For example, a State may wish to determine the percentage 
of drivers in crashes linked to the adjudication file.

Driver-Accessibility

    Accessibility reflects the ability of legitimate users to 
successfully obtain desired data. The below process outlines one way of 
measuring the driver database's accessibility.
    D-X-1: To measure accessibility: (1) Identify the principal users 
of the driver database; (2) Query the principal users to assess (A) 
their ability to obtain the data or other services requested and (B) 
their satisfaction with the timeliness of the response to their 
request; (3) Document the method of data collection and the principal 
users' responses

Roadway-Timeliness

    Timeliness always reflects the span of time between the occurrence 
of some event and the entry of information from the event into the 
appropriate database. For the roadway database, the State determines 
the events of principal interest that will be used to measure 
timeliness. A State may determine that the completion of periodic 
collection of a critical roadway data element or elements constitutes a 
critical status change of that roadway record. For example, one 
critical roadway data element that many States periodically collect is 
annual average daily traffic (AADT). Roadway database timeliness can be 
validly gauged by measuring the

[[Page 20444]]

time between the completion of data collection and the entry into the 
database of AADT for roadway segments of interest. There are many ways 
to do this. The model performance measures offer two general approaches 
to measuring vehicle database timeliness:
    R-T-1: The median or mean number of days from (A) the date a 
periodic collection of a critical roadway data element is complete 
(e.g., Annual Average Daily Traffic) to (B) the date the updated 
critical roadway data element is entered into the database. The median 
value is the duration within which 50 percent of the changes to 
critical roadway elements were updated in the database. Alternatively, 
the arithmetic mean is the average number of days between the 
completion of the collection of critical roadway elements and when the 
data are entered into the database.
    R-T-2: The median or mean number of days from (A) roadway project 
completion to (B) the date the updated critical data elements are 
entered into the roadway inventory file. The median value is the point 
at which 50 percent of the updated critical data elements from a 
completed roadway project were entered into the roadway inventory file. 
Alternatively, the arithmetic mean could be calculated for this 
measure. Each State will determine its short list of critical data 
elements, which should be a subset of MIRE. For example, it could be 
some or all of the elements required for Highway Performance Monitoring 
System (HPMS) sites. The database should be updated at regular 
intervals or when a change is made to the inventory. For example, when 
a roadway characteristic or attribute (e.g., traffic counts, speed 
limits, signs, markings, lighting, etc.) that is contained in the 
inventory is modified, the inventory should be updated within a 
reasonable period.

Roadway-Accuracy

    Accuracy reflects the number of errors in information in the 
records entered into a database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. Methods for detecting errors 
include: (1) Determining that the values entered for a variable or 
element are not legitimate codes, (2) matching with external sources of 
information, and (3) identifying duplicate records have been entered 
for the same event. The model performance measures offer one approach 
to measuring roadway database accuracy:
    R-A-1: The percentage of all road segment records with no errors in 
critical data elements. The State selects one or more roadway data 
elements it considers critical and assesses the accuracy of that 
element or elements in all of the roadway records within a period 
defined by the State. Many States consider the HPMS standards to be 
critical.

Roadway-Completeness

    Completeness has internal and external aspects. For the roadway 
database, external roadway completeness reflects the portion of road 
segments in the State for which data are collected and entered into the 
database. It is very difficult to determine precisely external roadway 
completeness because many States do not know the characteristics or 
even the existence of roadway segments that are non-State owned, 
maintained, or reported in the HPMS. Internal completeness reflects the 
amount of specified information that is captured in individual road 
segment records. It is possible to determine precisely internal roadway 
completeness. One can, for example, calculate the percentage of roadway 
segment records in the database that is missing one or more critical 
elements (e.g., number of traffic lanes. The model performance measures 
offer four general approaches to measuring the roadway database's 
internal completeness:
    R-C-1: The percentage of road segment records with no missing 
critical data elements. The State selects one or more roadway elements 
it considers critical and assesses internal completeness by dividing 
the number of records not missing a critical element by the total 
number of roadway records in the database.
    R-C-2: The percentage of public road miles or jurisdictions 
identified on the State's basemap or roadway inventory file. A 
jurisdiction may be defined by the limits of a State, county, parish, 
township, Metropolitan Planning Organization (MPO), or municipality.
    R-C-3: The percentage of unknowns or blanks in critical data 
elements for which unknown is not an acceptable value. This measure 
should be used when States wish to track improvements on specific 
critical data elements and reduce the occurrence of illegitimate null 
values.
    R-C-4: The percentage of total roadway segments that include 
location coordinates, using measurement frames such as a GIS basemap. 
This is a measure of the database's overall completeness.

Roadway-Uniformity

    Uniformity reflects the consistency among the files or records in a 
database and may be measured against some independent standard, 
preferably a national standard. The model performance measures offer 
one general approach to measuring roadway database uniformity:
    R-U-1: The number of Model Inventory of Roadway Elements (MIRE)-
compliant data elements entered into a database or obtained via linkage 
to other database(s).

Roadway-Integration

    Integration reflects the ability of records in the roadway database 
to be linked to a set of records in another of the six core databases--
or components thereof--using common or unique identifiers.
    R-I-1: The percentage of appropriate records in a specific file in 
the roadway database that are linked to another system or file. For 
example, a State may wish to determine the percentage of records in the 
State's bridge inventory that link to the basemap file.

Roadway-Accessibility

    Accessibility reflects the ability of legitimate users to 
successfully obtain desired data. The below process outlines one way of 
measuring roadway database accessibility:
    R-X-1: To measure accessibility of a specific file in the roadway 
database: (1) Identify the principal users of the file; (2) Query the 
principal users to assess (A) their ability to obtain the data or other 
services requested and (B) their satisfaction with the timeliness of 
the response to their request; (3) Document the method of data 
collection and the principal users' responses.

Citation/Adjudication-Timeliness

    Timeliness always reflects the span of time between the occurrence 
of some event and the entry of information from the event into the 
appropriate database. For the citation and adjudication databases, the 
State determines the events of principal interest that will be used to 
measure timeliness. Many States will include the critical events of 
citation issuance and citation disposition among those events of 
principal interest used to track timeliness. There are many ways to 
measure the timeliness of either citation issuance or citation 
disposition. The model performance measures offer one general approach 
to measuring citation and adjudication database timeliness:
    C/A-T-1: The median or mean number of days from (A) the date a 
citation is issued to (B) the date the

[[Page 20445]]

citation is entered into the statewide citation database, or a first 
available repository. The median value is the point at which 50 percent 
of the citation records were entered into the citation database within 
a period defined by the State. Alternatively, the arithmetic mean could 
be calculated for this measure.
    C/A-T-2: The median or mean number of days from (A) the date of 
charge disposition to (B) the date the charge disposition is entered 
into the statewide adjudication database, or a first available 
repository. The median value is the point at which 50 percent of the 
charge dispositions were entered into the statewide database. 
Alternatively, the arithmetic mean could be calculated for this 
measure.

    Note: Many States do not have statewide databases for citation 
or adjudication records. Therefore, in some citation and 
adjudication data systems, timeliness and other attributes of data 
quality should be measured at individual first available 
repositories.

Citation/Adjudication-Accuracy

    Accuracy reflects the number of errors in information in the 
records entered into a database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. Methods for detecting errors 
include: (1) Determining that the values entered for a variable or 
element are not legitimate codes, (2) matching with external sources of 
information, and (3) identifying duplicate records that have been 
entered for the same event. The State selects one or more citation data 
elements and one or more charge disposition data elements it considers 
critical and assesses the accuracy of those elements in all of the 
citation and adjudication records entered into the database within a 
period of interest. The model performance measures offer two approaches 
to measuring citation and adjudication database accuracy:
    C/A-A-1: The percentage of citation records with no errors in 
critical data elements. The State selects one or more citation data 
elements it considers critical and assesses the accuracy of that 
element or elements in all of the citation records entered into the 
database within a period defined by the State. Below is a list of 
suggested critical data elements.
    C/A-A-2: The percentage of charge disposition records with no 
errors in critical data elements. The State selects one or more charge 
disposition data elements it considers critical and assesses the 
accuracy of that element or elements for the charge-disposition records 
entered into the database within a period defined by the State. Many 
States have identified the following as critical data elements: 
Critical elements from the Issuing Agency include the offense/charge 
code, date, time, officer, Agency, citation number, crash report number 
(as applicable), and BAC (as applicable). Critical elements from the 
Citation Data include the Offender's name, driver license number, age, 
and sex. Critical data elements from the Charge Disposition/
Adjudication include the offender's name, driver license number, age, 
sex, and citation number. From the charge Disposition/Adjudication: 
court, date of receipt, date of disposition, disposition, and date of 
transmittal to DMV (as applicable).

Citation/Adjudication-Completeness*

    Completeness has internal and external aspects. For the citation/
adjudication databases, external completeness can only be assessed by 
identifying citation numbers for which there are no records. Missing 
citations should be monitored at the place of first repository. 
Internal completeness reflects the amount of specified information that 
is captured in individual citation and charge disposition records. It 
is possible to determine precisely internal citation and adjudication 
completeness. One can, for example, calculate the percentage of 
citation records in the database that are missing one or more critical 
data elements. The model performance measures offer three approaches to 
measuring internal completeness:
    C/A-C-1: The percentage of citation records with no missing 
critical data elements. The State selects one or more citation data 
elements it considers critical and assesses internal completeness by 
dividing the number of records not missing a critical element by the 
total number of records entered into the database within a period 
defined by the State.
    C/A-C-2: The percentage of citation records with no missing data 
elements. The State can assess overall completeness by dividing the 
number of records missing no elements by the total number of records 
entered into the database.
    C/A-C-3: The percentage of unknowns or blanks in critical citation 
data elements for which unknown is not an acceptable value. This 
measure should be used when States wish to track improvements on 
specific critical data elements and reduce the occurrence of 
illegitimate null values.

    Note: These measures of completeness are also applicable to the 
adjudication file.

Citation/Adjudication-Uniformity *

    Uniformity reflects the consistency among the files or records in a 
database and may be measured against some independent standard, 
preferably a national standard. The model performance measures offer 
two general approaches to measuring database uniformity:
    C/A-U-1: The number of Model Impaired Driving Record Information 
System (MIDRIS)-compliant data elements entered into the citation 
database or obtained via linkage to other database(s).
    C/A-U-2: The percentage of citation records entered into the 
database with common uniform statewide violation codes. The State 
identifies the number of citation records with common uniform violation 
codes entered into the database within a period defined by the State 
and assesses uniformity by dividing this number by the total number of 
citation records entered into the database during the same period.

    * Note: These measures of uniformity are also applicable to the 
adjudication file.

Citation/Adjudication-Integration *

    Integration reflects the ability of records in the citation 
database to be linked to a set of records in another of the six core 
databases--or components thereof--using common or unique identifiers.
    C/A-I-1: The percentage of appropriate records in the citation 
files that are linked to another system or file. Linking the citation 
database with the five other core traffic record databases can provide 
important information. For example, a State may wish to determine the 
percentage of DWI citations that have been adjudicated.

    * Note: This measure of integration is also applicable to the 
adjudication file.

Citation/Adjudication-Accessibility *

    Accessibility reflects the ability of legitimate users to 
successfully obtain desired data. The below process outlines one way of 
measuring the citation database's accessibility.
    C/A-X-1: To measure accessibility of the citation database: (1) 
Identify the principal users of the citation database; (2) Query the 
principal users to assess (A) their ability to obtain the data or other 
services requested and (B) their satisfaction with the timeliness of 
the response to their request; (3) Document the method of data 
collection and the principal users' responses. The EMS/Injury 
Surveillance database is actually a set of related databases. The 
principal files of interest are: Pre-hospital

[[Page 20446]]

Emergency Medical Services (EMS) data, Hospital Emergency Department 
Data Systems, Hospital Discharge Data Systems, and State Trauma 
Registry File, State Vital Records. States typically wish to measure 
data quality separately for each of these files. These measures may be 
applied to each of the EMS/Injury Surveillance databases individually.

Injury Surveillance-Timeliness *

    Timeliness always reflects the span of time between the occurrence 
of some event and the entry of information from the event into the 
appropriate database. For the EMS/Injury Surveillance databases, the 
State determines the events of principal interest that will be used to 
measure timeliness. A State may, for example, determine that the 
occurrence of an EMS run constitutes a critical event to measure the 
timeliness of the EMS database. As another example, a State can select 
the occurrence of a hospital discharge as the critical event to measure 
the timeliness of the hospital discharge data system. There are many 
ways to measure the timeliness of the EMS/Injury Surveillance 
databases. The model performance measures offer two general approaches 
to measuring timeliness:
    I-T-1: The median or mean number of days from (A) the date of an 
EMS run to (B) the date when the EMS patient care report is entered 
into the database. The median value is the point at which 50 percent of 
the EMS run reports were entered into the database within a period 
defined by the State. Alternatively, the arithmetic mean could be 
calculated for this measure.
    I-T-2: The percentage of EMS patient care reports entered into the 
State EMS discharge file within XX* days after the EMS run. The XX 
usually reflects a target or goal set by the State for entry of reports 
into the database. The higher percentage of reports entered within XX 
days, the timelier the database. Many States set the XX for EMS data 
entry at 5, 30, or 90 days, but any target or goal is equally 
acceptable.

    * Note: This measure of timeliness is also applicable to the 
following files: State Emergency Dept. File, State Hospital 
Discharge File, State Trauma Registry File, & State Vital Records.

Injury Surveillance-Accuracy *

    Accuracy reflects the number of errors in information in the 
records entered into a database. Error means the recorded value for 
some data element of interest is incorrect. Error does not mean the 
information is missing from the record. Erroneous information in a 
database cannot always be detected. Methods for detecting errors 
include: 1) determining that the values entered for a variable or 
element are not legitimate codes, 2) matching with external sources of 
information, and 3) identifying duplicate records have been entered for 
the same event. The model performance measures offer one general 
approach to measuring the accuracy of the injury surveillance databases 
that is applicable to each of the five principal files:
    I-A-1: The percentage of EMS patient care reports with no errors in 
critical data elements. The State selects one or more EMS data elements 
it considers critical--response times, for example--and assesses the 
accuracy of that element or elements for all the records entered into 
the database within a period defined by the State. Critical EMS/Injury 
Surveillance Data elements used by many States include: Hospital 
Emergency Department/Inpatient Data elements such as E-code, date of 
birth, name, sex, admission date/time, zip code of hospital, emergency 
dept. disposition, inpatient disposition, diagnosis codes, and 
discharge date/time. Elements from the Trauma Registry Data (National 
Trauma Data Bank [NTDB] standard) such as E-code, date of birth, name, 
sex, zip code of injury, admission date, admission time, inpatient 
disposition, diagnosis codes, zip code of hospital, discharge date/
time, and EMS patient report number. Data from the EMS Data (National 
Emergency Medical Services Information System [NEMSIS] standard) 
includes date of birth, name, sex, incident date/time, scene arrival 
date/time, provider's primary impression, injury type, scene departure 
date/time, destination arrival date/time, county/zip code of hospital, 
and county/zip code of injury Critical data elements from the Death 
Certificate (Mortality) Data (National Center for Health Statistics 
[NCHS] standard) include date of birth, date of death, name, sex, 
manner of death, underlying cause of death, contributory cause of 
death, county/zip code of death, and location of death.

    * Note: This measure of accuracy is also applicable to the 
following files: State Emergency Dept. File, State Hospital 
Discharge File, State Trauma Registry File, & State Vital Records.

Injury Surveillance-Completeness*

    Completeness has internal and external aspects. For EMS/Injury 
Surveillance databases, external completeness reflects the portion of 
critical events (e.g., EMS runs, hospital admissions, etc.) that are 
reported and entered into the databases. It is not possible to 
determine precisely external EMS/injury surveillance completeness 
because once can never know the how many critical events occurred but 
went unreported. Internal completeness reflects the amount of specified 
information that is captured in individual EMS run records, State 
Emergency Department records, State Hospital Discharge File records, 
and State Trauma Registry File records. It is possible to determine 
precisely internal EMS/Injury Surveillance completeness. One can, for 
example, calculate the percentage of EMS run records in the database 
that are missing one or more critical data elements. The model 
performance measures offer three approaches to measuring completeness 
for each of the files:
    I-C-1: The percentage of EMS patient care reports with no missing 
critical data elements. The State selects one or more EMS data elements 
it considers critical and assesses internal completeness by dividing 
the number of EMS run records not missing a critical element by the 
total number of EMS run records entered into the database within a 
period defined by the State.
    I-C-2: The percentage of EMS patient care reports with no missing 
data elements. The State can assess overall completeness by dividing 
the number of records missing no elements by the total number of 
records entered into the database.
    I-C-3: The percentage of unknowns or blanks in critical data 
elements for which unknown is not an acceptable value. This measure 
should be used when States wish to track improvement on specific 
critical data values and reduce the occurrence of illegitimate null 
values. E-code, for example, is an appropriate EMS/Injury Surveillance 
data element that may be tracked with this measure.

    * Note: These measures of completeness are also applicable to 
the following files: State Emergency Dept. File, State Hospital 
Discharge File, State Trauma Registry File, & State Vital Records.

Injury Surveillance-Uniformity

    Uniformity reflects the consistency among the files or records in a 
database and may be measured against an independent standard, 
preferably a national standard. The model performance measures offer 
one approach to measuring uniformity that can be applied to each 
discrete file using the appropriate standard as enumerated below.
    I-U-1: The percentage of National Emergency Medical Services 
Information System (NEMSIS)-

[[Page 20447]]

compliant data elements on EMS patient care reports entered into the 
database or obtained via linkage to other database(s).
    I-U-2: The number of National Emergency Medical Services 
Information System (NEMSIS)-compliant data elements on EMS patient care 
reports entered into the database or obtained via linkage to other 
database(s).
    The national standards for many of the other major EMS/Injury 
Surveillance database files are: The Universal Billing 04 (UB04) for 
State Emergency Department Discharge File and State Hospital Discharge 
File; the National Trauma Data Standards (NTDS) for State Trauma 
Registry File; and the National Association for Public Health 
Statistics and Information Systems (NAPHSIS) for State Vital Records.

Injury Surveillance-Integration*

    Integration reflects the ability of records in the EMS database to 
be linked to a set of records in another of the six core databases--or 
components thereof--using common or unique identifiers.
    I-I-1: The percentage of appropriate records in the EMS file that 
are linked to another system or file. Linking the EMS file to other 
files in the EMS/Injury Surveillance database or any of the five other 
core databases can provide important information. For example, a State 
may wish to determine the percentage of EMS records that link to the 
trauma file that are linked to the EMS file.

    * Note: This measure of integration is also applicable to the 
following files: State Emergency Dept. File, State Hospital 
Discharge File, State Trauma Registry File, & State Vital Records.

Injury Surveillance-Accessibility *

    Accessibility reflects the ability of legitimate users to 
successfully obtain desired data.
    I-X-1: To measure accessibility of the EMS file: (1) Identify the 
principal users of the EMS file, (2) Query the principal users to 
assess (A) their ability to obtain the data or other services requested 
and (B) their satisfaction with the timeliness of the response to their 
request, and (3) Document the method of data collection and the 
principal users' responses

    Note: This measure of accessibility is also applicable to the 
State Emergency Dept. File, the State Hospital Discharge File, the 
State Trauma Registry File, & State Vital Records.

Recommendations

    While use of the performance measures is voluntary, States will be 
better able to track the success of upgrades and identify areas for 
improvement in their traffic records systems if they elect to utilize 
the measures appropriate to their circumstances. Adopting the measures 
will also put States ahead of the curve should performance metrics be 
mandated in any future legislation. The measures are not exhaustive. 
They describe what to measure and suggest how to measure it, but do not 
recommend numerical performance goals. The measures attempt to capture 
one or two key performance features of each data system performance 
attribute. States may wish to use additional or alternative measures to 
address specific performance issues.
    States that elect to use these measures to demonstrate progress in 
a particular system should start using them immediately. States should 
begin by judiciously selecting the appropriate measures and modifying 
them as needed. States should use only the measures for the data system 
performance attributes they wish to monitor or improve. No State is 
expected to use a majority of the measures, and States may wish to 
develop their own additional measures to track State-specific issues or 
programs.
    Once States have developed their specific performance indices, they 
should be measured consistently to track changes over time. Since the 
measures will vary considerably from State to State, it is unlikely 
that they could be used for any meaningful comparisons between States. 
In any event, NHTSA does not anticipate using the measures for 
interstate comparison purposes.

Notes on Terminology Used

    The following terms are used throughout the document:
    Data system: One of the six component State traffic records 
databases, such as crash, injury surveillance, etc.
    Data file (such as ``crash file'' or ``State Hospital Discharge 
file''): A data system may contain a single data file--such as a 
State's driver file--or more than one, e.g., the injury system has 
several data files.
    Record: All the data entered in a file for a specific event (a 
crash, a patient hospital discharge, etc.).
    Data element: Individual fields coded within each record.
    Data element code value: The allowable code values or attributes 
for a data element.
    Data linkages: The links established by matching at least one data 
element in a record in one file with the corresponding element or 
elements in one or more records in another file or files.
    State: The 50 States, the District of Columbia, Puerto Rico, the 
territories, and the Bureau of Indian Affairs. These are the 
jurisdictions eligible to receive State data improvement grants.

 Defining and Calculating Performance Measures

    Specified number of days: Some measures are defined in terms of a 
specified number of days (such as 30, 60, or 90). Each State can 
establish its own period for these measures.
    Defining periods of interest: States will need to define periods of 
interest for