Pipeline Safety: Using Meaningful Metrics in Conducting Integrity Management Program Evaluations, 72435-72438 [2012-29362]

Agencies

• Pipeline and Hazardous Materials Safety Administration
• DEPARTMENT OF TRANSPORTATION

[Federal Register Volume 77, Number 234 (Wednesday, December 5, 2012)]
[Notices]
[Pages 72435-72438]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2012-29362]


-----------------------------------------------------------------------

DEPARTMENT OF TRANSPORTATION

Pipeline and Hazardous Materials Safety Administration

[Docket No. PHMSA-2012-0279]


Pipeline Safety: Using Meaningful Metrics in Conducting Integrity 
Management Program Evaluations

AGENCY: Pipeline and Hazardous Materials Safety Administration (PHMSA), 
DOT.

ACTION: Notice; Issuance of Advisory Bulletin.

-----------------------------------------------------------------------

SUMMARY: PHMSA is issuing an Advisory Bulletin to remind operators of 
gas transmission and hazardous liquid pipeline facilities of their 
responsibilities, under Federal integrity management regulations, to 
perform evaluations of their integrity management programs using 
meaningful performance metrics.

FOR FURTHER INFORMATION CONTACT: Alan Mayberry by phone at 202-366-5124 
or by email at alan.mayberry@dot.gov. All materials in this docket may 
be accessed electronically at https://www.regulations.gov. General 
information about the PHMSA Office of Pipeline Safety (OPS) can be 
obtained by accessing OPS's Internet home page at https://www.phmsa.dot.gov/pipeline.

SUPPLEMENTARY INFORMATION:

I. Background

    PHMSA's integrity management regulations require operators to 
establish processes to evaluate the effectiveness of their integrity 
management programs. Program evaluation is one of the key required 
program elements as established in the integrity management rules. For 
hazardous liquid pipelines, Sec. Sec.  195.452(f)(7) and 195.452(k) 
require methods to measure program effectiveness:
    Sec.  195.452(f) What are the elements of an integrity management 
program? An integrity management program begins with the initial 
framework. An operator must continually change the program to reflect 
operating experience, conclusions drawn from results of the integrity 
assessments, other maintenance and surveillance data, and evaluation of 
consequences of a failure on the high consequence area. An operator 
must include, at minimum, each of the following elements in its written 
integrity management program:
* * * * *
    (7) Methods to measure the program's effectiveness (see paragraph 
(k) of this section);
    Sec.  195.452(k) What methods to measure program effectiveness must 
be used? An operator's program must include methods to measure whether 
the program is effective in assessing and evaluating the integrity of 
each pipeline segment and in protecting the high consequence areas. 
(See Appendix C of this part for guidance on methods that can be used 
to evaluate a program's effectiveness.)
    Appendix C provides more specific guidance on establishing 
performance measures, including the need to select measures based on 
the understanding and analysis of integrity threats to each pipeline 
segment. Appendix C also describes three general types of metrics that 
an integrity management program should have:
     Activity Measures that monitor the surveillance and 
preventive activities that are in place to control risk. These measures 
indicate how well an operator is implementing the elements of its 
integrity management program.

[[Page 72436]]

     Deterioration Measures that monitor operational and 
maintenance trends to indicate if the program is successful or 
weakening, or if the desired outcome is being achieved or not, despite 
the risk control activities in place.
     Failure Measures that reflect whether the program is 
effective in achieving the objective of improving integrity. These are 
typically lagging indicators that measure the number of releases, the 
volume spilled, percent recovered, etc.
    Section 13 ``Program Evaluation'' of API Standard 1160, Managing 
Integrity for Hazardous Liquid Pipelines also provides additional 
guidance on the program evaluation process in which these measures are 
used to improve performance.
    For gas transmission pipelines, Sec. Sec.  192.911(i) and 192.945 
define the requirements for establishing performance metrics and 
evaluating integrity management program performance.


Sec.  192.911  What are the elements of an integrity management 
program?

    An operator's initial integrity management program begins with a 
framework (see Sec.  192.907) and evolves into a more detailed and 
comprehensive integrity management program as information is gained and 
incorporated into the program. An operator must make continual 
improvements to its program. The initial program framework and 
subsequent program must, at minimum, contain the following elements. 
(When indicated, refer to ASME/ANSI B31.8S incorporated by reference, 
see Sec.  192.7) for more detailed information on the listed element.)
* * * * *
    (i) A performance plan as outlined in ASME/ANSI B31.8S, section 9 
that includes performance measures meeting the requirements of Sec.  
192.945.


Sec.  192.945  What methods must an operator use to measure program 
effectiveness?

    (a) General. An operator must include in its integrity management 
program methods to measure whether the program is effective in 
assessing and evaluating the integrity of each covered pipeline segment 
and in protecting the high consequence areas. These measures must 
include the four overall performance measures specified in ASME/ANSI 
B31.8S (incorporated by reference, see Sec.  192.7 of this part), 
section 9.4, and the specific measures for each identified threat 
specified in ASME/ANSI B31.8S, Appendix A. An operator must submit the 
four overall performance measures as part of the annual report required 
by Sec.  191.17 of this subchapter.
    (b) External Corrosion Direct Assessment (ECDA). In addition to the 
general requirements for performance measures in paragraph (a) of this 
section, an operator using direct assessment to assess an external 
corrosion threat must define and monitor measures to determine the 
effectiveness of the ECDA process. These measures must meet the 
requirements of Sec.  192.925.
    The gas transmission requirements invoke ASME B31.8S-2004, Managing 
System Integrity of Gas Pipelines. Section 9 of this standard provides 
guidance on the selection of performance measures. It describes three 
categories of measures that are directly analogous to those noted above 
in Appendix C of Part 195. These are:
     Process or Activity Measures used to evaluate preventive 
and mitigation activities. These determine how well an operator is 
implementing the various elements of its integrity management program.
     Operational Measures, which include operational and 
maintenance trends that measure how well the system is responding to 
the integrity management program.
     Direct Integrity Measures, which include leaks, ruptures, 
injuries, and fatalities.
    Furthermore, the hazardous liquid and gas transmission integrity 
management rules also require that operators retain adequate records to 
support integrity management program decisions and activities. These 
include the information that supports the selection of performance 
metrics, the performance metric data and trends, and the decisions that 
are based in whole or in part on these metrics. Specifically, the 
hazardous liquid integrity management program requirements are:
    Sec.  195.452(l) What records must be kept? (1) An operator must 
maintain for review during an inspection:
* * * * *
    (ii) Documents to support the decisions and analyses, including any 
modifications, justifications, variances, deviations and determinations 
made, and actions taken, to implement and evaluate each element of the 
integrity management program listed in paragraph (f) of this section.
    (2) See Appendix C of this part for examples of records an operator 
would be required to keep.
    Appendix C further states:
    Sec.  195.452 Appendix C. VI. Examples of types of records an 
operator must maintain.
* * * * *
    (22) methods used to measure the program's effectiveness.
    The comparable gas transmission integrity management program 
requirements are:


Sec.  192.947  What records must be kept?

    An operator must maintain, for the useful life of the pipeline, 
records that demonstrate compliance with the requirements of this 
subpart. At minimum, an operator must maintain the following records 
for review during an inspection.
* * * * *
    (d) Documents to support any decision, analysis, and process 
developed and used to implement and evaluate each element of the 
baseline assessment plan and integrity management program. Documents 
include those developed and used in support of any identification, 
calculation, amendment, modification, justification, deviation and 
determination made, and any action taken, to implement and evaluate any 
of the program elements;
    PHMSA's inspection protocols currently address the need to examine 
operator compliance with these requirements.
    In its report on the September 9, 2010, gas pipeline accident in 
San Bruno, California, the National Transportation Safety Board (NTSB) 
identified concerns with Pacific Gas and Electric Company's (PG&E) 
self-assessments of its integrity management program. NTSB concluded 
that the company's self-assessments were ``superficial and resulted in 
no improvements to the integrity management program.'' As a result, 
NTSB recommended that PG&E:
    Assess every aspect of your integrity management program, paying 
particular attention to the areas identified in this investigation, and 
implement a revised program that includes, at a minimum,
* * * * *
    (4) an improved self-assessment that adequately measures whether 
the program is effectively assessing and evaluating the integrity of 
each covered pipeline segment. (Recommendation P-11-29)
    In this same investigation, NTSB raised some concerns with PHMSA's 
oversight of performance-based safety programs such as integrity 
management. NTSB concluded that greater focus is needed on how 
performance-based safety systems are implemented, executed and 
evaluated, and whether problem areas are being detected and corrected. 
Critical to this overall process is the selection of meaningful metrics 
by operators that allow them to quantify,

[[Page 72437]]

understand, and improve their own performance.
    Following its investigation, NTSB issued two related 
recommendations for enhancing PHMSA's oversight of operator programs to 
assess the effectiveness of PHMSA's programs using performance metrics. 
These recommendations are:
    Revise your integrity management inspection protocol to:
    (1) incorporate a review of meaningful metrics;
    (2) require auditors to verify that the operator has a procedure in 
place for ensuring the completeness and accuracy of underlying 
information;
    (3) require auditors to review all integrity management performance 
measures reported to the Pipeline and Hazardous Materials Safety 
Administration and compare the leak, failure, and incident measures to 
the operator's risk model; and
    (4) require setting performance goals for pipeline operators at 
each audit and follow up on those goals at subsequent audits. 
(Recommendation P-11-18)
    (1) Develop and implement standards for integrity management and 
other performance-based safety programs that require operators of all 
types of pipeline systems to regularly assess the effectiveness of 
their programs using clear and meaningful metrics and to identify and 
then correct deficiencies; and (2) make those metrics available in a 
centralized database. (Recommendation P-11-19)
    These recommendations reinforce the importance of a rigorous 
evaluation of a company's integrity management program in improving 
performance. Through this Advisory Bulletin, PHMSA is reminding 
operators of the importance of these regulation-required program 
elements. Operators should review their current programs for evaluating 
integrity management program effectiveness and the performance metrics 
used in these programs to be sure they provide a current and accurate 
representation of integrity management program performance. Further, 
operators should ensure that program improvements and corrective 
actions identified by these evaluations are implemented in a timely 
manner.
    As a result of NTSB's recommendations, PHMSA is initiating efforts 
to strengthen its protocols and oversight of these key integrity 
management program elements. Beginning immediately, PHMSA's inspections 
will emphasize reviewing operator methods for integrity management 
program evaluation as required by Sec.  192.945 and Sec.  195.452(k) 
for gas transmission and hazardous liquid pipelines, respectively. 
PHMSA will evaluate specific metrics operators use to assess program 
effectiveness and how those metrics are used in a process of continuous 
improvement. PHMSA will also confirm that operators are maintaining 
adequate records of their program effectiveness evaluations and their 
performance metrics data, as well as the activities and decisions 
associated with all required integrity management program elements. Our 
inspectors will check to confirm that information and data gaps are 
aggressively being addressed and that assumptions are appropriately 
based on location-specific data.

II. Advisory Bulletin (ADB-20l2-10)

    To: Owners and Operators of Hazardous Liquid and Gas Transmission 
Pipeline Systems
    Subject: Using Meaningful Metrics in Conducting Integrity 
Management Program Evaluations
    Advisory: To further enhance PHMSA's safety efforts and as an 
initial step in addressing NTSB Recommendations P-11-18 and P-11-19, 
PHMSA is issuing this Advisory Bulletin concerning operator integrity 
management program evaluation using meaningful metrics.
    A critical program element of an operator's integrity management 
program is the systematic, rigorous evaluation of the program's 
effectiveness using clear and meaningful metrics. When executed 
diligently, this self-evaluation process will lead to more robust and 
effective integrity management programs and improve overall safety 
performance. This process is critical to achieving a mature integrity 
management program and a culture of continuous improvement. Program 
evaluation is a required integrity management program element as 
established in Sec. Sec.  192.911(i) and 195.452(k) for gas 
transmission and hazardous liquid pipelines, respectively. In light of 
NTSB's findings following the San Bruno gas transmission incident, 
PHMSA is reminding operators about the importance of these 
requirements.
    Operators are advised to critically review their processes and 
methods for evaluating integrity management program performance and 
take action to strengthen these processes where warranted. An effective 
operator performance evaluation process is expected to have the 
following characteristics:
     A well-defined description of the scope, objectives, and 
frequency of program evaluations.
     The use of periodic self-assessments, internal or external 
audits, management reviews, performance metrics analysis, benchmarking 
against other operators, or other self-critical evaluations to assess 
program effectiveness.
     Clear performance goals and objectives to measure the 
effectiveness of key integrity activities.
     Clear assignment of responsibility for implementing 
required actions.
     Review and follow-up of program evaluation results, 
findings, and recommendations, etc., by appropriate company managers.
    Operators are also advised that a clear and meaningful set of 
performance metrics is essential to program effectiveness. An effective 
program for measuring integrity management program effectiveness should 
have the following characteristics:
     A description of the type of performance measures to be 
used, along with the data sources, data validation and quality 
assurance activities, the frequency of data collection, and any 
normalization factors.
     A means to update the performance measures (if needed) to 
assure they are providing useful information about the effectiveness of 
integrity management program activities.
     The use of performance metrics data to check and calibrate 
the operator's risk analysis tools to assure these best represent the 
performance of the operator's specific assets.
    The performance metrics that are required to be reported to PHMSA 
annually, such as the number of miles of pipeline assessed, number of 
anomalies found requiring repair or mitigation, etc., are a small 
subset of the overall suite of metrics used by an operator to evaluate 
its program. A much larger set of operator-specific metrics to be used 
internally is needed to effectively evaluate an integrity management 
program performance. Metrics should be developed for each of the 
following:
     Overall program effectiveness indicated by the number of 
releases, number of injuries or fatalities, volume released, etc.
     Specific threats that include both leading and lagging 
indicators for the important integrity threats on an operator's 
systems. These include:
    [cir] Activity Measures that monitor the surveillance and 
preventive activities that are in place to control risk.
    [cir] Deterioration Measures that monitor operational and 
maintenance trends to indicate if the program is successful or 
weakening despite the risk control activities in place. (Also

[[Page 72438]]

identified as Operational Measures in ASME B31.8S.)
    [cir] Failure Measures that reflect whether the program is 
effective in achieving the objective of improving integrity. (Also 
identified as Direct Integrity Measures in ASME B31.8S)
     Metrics that measure and provide insights into how well an 
operator's processes associated with the various integrity management 
program elements are performing. Examples of such processes would 
include integrity assessment, risk analysis, the identification of 
preventive and mitigative measures, etc.
    While operator-level rollups of metrics are useful for small 
operators, a robust program for large operators should also include 
metrics at a more granular level. The metrics should enable operators 
to drill down to understand the performance of specific systems or 
segments within systems. This is particularly important for the threat-
specific metrics mentioned previously.
    Finally, as required by Sec. Sec.  195.452(l) and 192.947, 
operators must keep records supporting the decisions, analyses, and 
processes developed and used in their evaluation of integrity 
management program effectiveness. These records should include those 
justifying the selection of performance metrics, the performance metric 
data and trends, and how these metrics are used to improve the 
integrity management program. Operators should also be diligently 
working to eliminate information and data gaps throughout their entire 
integrity management program.

    Issued in Washington, DC, on November 29, 2012.
Jeffrey D. Wiese,
Associate Administrator for Pipeline Safety.
[FR Doc. 2012-29362 Filed 12-4-12; 8:45 am]
BILLING CODE 4910-60-P