Information Notice No. 95-24: Summary Of Licensed Operator Requalification Inspection Program Findings

                          UNITED STATES
                  NUCLEAR REGULATORY COMMISSION 
               OFFICE OF NUCLEAR REACTOR REGULATION
                   WASHINGTON, D.C. 20555-0001

                          April 25, 1995


NRC INFORMATION NOTICE 95-24:  SUMMARY OF LICENSED OPERATOR
                               REQUALIFICATION
                               INSPECTION PROGRAM FINDINGS


Addressees

All holders of operating licenses or construction permits for
nuclear power reactors.

Purpose

The U.S. Nuclear Regulatory Commission (NRC) is issuing this
information notice to alert addressees to deficiencies and
weaknesses uncovered while conducting its licensed operator
requalification inspection program.  It is expected that
recipients will review the information for applicability to their
facilities and consider actions, as appropriate, to avoid similar
problems.  However, suggestions contained in this information
notice are not NRC requirements; therefore, no specific action or
written response is required.

Background

Effective March 11, 1994, the NRC amended Part 55, "Operators'
Licenses," of Title 10 of the Code of Federal Regulations (10
CFR) to eliminate the requirement for licensed operators to pass
a comprehensive requalification written examination and an
operating test conducted by the NRC during the term of the
operator's license.  The amendment enabled the NRC to shift its
focus from examining individual operators for the purpose of
license renewal to evaluating the effectiveness with which
facility licensees conduct their requalification programs.

The NRC developed an inspection procedure (IP 71001), "Licensed
Operator Requalification Program Evaluation," to implement the
new requalification oversight program and to guide inspectors as
they review the subject programs.  The procedure includes
assessments of facility licensee effectiveness in:

     -    evaluating trainee (operator and crew) mastery of the
     training objectives as required by 10 CFR 55.59(c) and by
     element 4 of a systems approach to training (SAT)-based
     program as defined in 10 CFR 55.4;

     -    evaluating and revising the requalification program based on
     operator performance as required by 10 CFR 55.59(c) and by
     element 5 of a SAT-based program;

     -    ensuring the integrity of requalification examinations and
     tests as required by 10 CFR 55.49; and

9504190049.                                     IN 95-24
                                                April 25, 1995
                                                Page 2 of 3


     -    ensuring that licensed operators satisfy the conditions of
     their licenses as specified in 10 CFR 55.53.

The NRC is using the inspection procedure to evaluate each
licensed operator requalification program at least once per
Systematic Assessment of Licensee Performance (SALP) cycle.

Discussion

During the period that the NRC conducted requalification
examinations for the purpose of renewing operator licenses (i.e.,
1987 to 1993), the staff noted significant improvements in the
performance of the individual operators and the quality of
facility licensee evaluators and testing materials.  As noted in
the March 1994 rule change, the NRC discontinued conducting
routine requalification examinations because licensees had
established a high standard of performance under the regulations
and NRC examiners were largely duplicating tasks that were
required of, and routinely performed by, facility licensees.  The
NRC resolved that it would not duplicate facility licensee
efforts to examine operators as long as the NRC staff remained
confident that the requalification program was maintaining
licensed operator competence.

Facility licensees are expected to comply with the 10 CFR Part 55
requirements for licensed operator requalification training and
testing and with the commitments contained in their respective
NRC-approved requalification programs.  Facility licensees having
SAT-based requalification programs are required by the NRC
regulations to implement five program elements (i.e., job
analysis, objective development, training design and
implementation, trainee evaluation, and program evaluation and
revision) to ensure that licensed operators and crews maintain
the job performance standards necessary for continued safe plant
operation.  Furthermore, facility licensees must ensure that
operators comply with their 10 CFR Part 55 license conditions. 
As noted earlier, the requalification program inspections
conducted in accordance with IP 71001 focus on many of these
elements and factors.  When necessary, the NRC may inspect
additional training program elements in accordance with IP 41500,
"Training and Qualification Effectiveness." 

Since January 1993, the NRC has completed more than 50
requalification program inspections using IP 71001 or its
predecessor, Temporary Instruction 2515/117.  A number of
specific findings, some of which were observed at several
facilities, are listed in Attachment 1.  

The findings in Attachment 1 suggest that some facility licensees
are relying largely on the guidelines in NUREG-1021, "Operator
Licensing Examiner Standards," for the development and
administration of their requalification examinations.  NUREG-1021
provides instructions for conducting NRC examinations only; it is
not intended to be guidance on how to implement a SAT-based
training program or to ensure compliance with all the regulations
applicable to requalification examinations.  For example, the
crew-based dynamic simulator evaluation procedure in NUREG-1021
does not ensure that each licensed operator will be individually
evaluated during an operating test as required by 10 CFR
55.59(a).
.                                                 IN 95-24
                                                  April 25, 1995
                                                  Page 3 of 3


The findings in Attachment 1 also indicate that the level of
difficulty of examinations at some facilities was questioned as
to whether facility licensees could determine that the operators
had mastered their job performance requirements as stipulated by
element 4 of a SAT-based training program or whether the
examinations would sufficiently require an operator to
demonstrate an understanding of and the ability to perform the
actions referenced by 10 CFR 55.59(a)(2)(ii).  Simulator
scenarios that verify operator ability to implement the emergency
operating procedures (EOPs) were questioned as to whether they
were at the level of difficulty necessary to adequately complete
the assessment.

The requalification inspections have identified a number of
weaknesses and deficiencies.  10 CFR 55.59(c) allows a facility
licensee significant latitude in the implementation of its
requalification program if the licensee adopts a systems approach
to training.  Many of the issues described in Attachment 1 are
performance-based issues that raise questions regarding the
effectiveness of facility licensee training and testing programs.

Although the staff has not judged the findings at specific
facilities to be of sufficient concern, to date, to warrant NRC
conducting requalification examinations, it has concluded that
the findings are sufficient in number and significance to share
them with the industry.  If an NRC inspection determines that a
requalification program is ineffective or if the staff concludes
that the inspection process will not provide the insight
necessary to confirm the adequacy of the program, the NRC may
exercise its discretion, per 10 CFR 55.59(a)(2)(iii), and conduct
requalification examinations in accordance with NUREG-1021.

This information notice requires no specific action or written
response.  If you have any questions about the information in
this notice, please contact one of the technical contacts listed
below or the appropriate Office of Nuclear Reactor Regulation
(NRR) project manager.

                              s/s'd by BDLiaw/for


                              Brian K. Grimes, Director
                              Division of Projects Support
                              Office of Nuclear Reactor Regulation

Technical contacts:  Stuart Richards, NRR       Mark Ring, RIII
                     (301) 415-1031             (708) 829-9703

                     Glenn Meyer, RI            John Pellet, RIV
                     (610) 337-5211             (817) 860-8159

                     Thomas Peebles, RII        Neal Hunemuller, NRR
                     (404) 331-5541             (301) 415-1152

Attachments:
1.  Program Deficiencies Identified by Inspections
2.  List of Recently Issued NRC Information Notices

.                                                  Attachment 1
                                                   IN 95-24
                                                   April 25, 1995
                                                   Page 1 of 2

          PROGRAM DEFICIENCIES IDENTIFIED BY INSPECTIONS

Trainee Evaluation

     -    The dynamic simulator scenario banks at some facilities did
     not contain any shutdown scenarios or failed to adequately
     exercise the contingency actions of the EOPs.

     -    Some of the dynamic simulator scenarios consisted of
     unrelated events or had critical tasks that could not
     discriminate between acceptable and unacceptable operator
     performance because they were impossible to fail.

     -    In one instance, written test items were worded in such a
     way that the person taking the test could possibly select
     the correct answer based solely on question construction.

     -    The job performance measures were sometimes overly simple
     and had little evaluative merit (e.g., push one button) or
     they had procedural verification steps that were
     inappropriately identified as critical to task completion.

     -    Some facility licensees were not able to explain their SAT-
     based rationale for selecting the control manipulations that
     were included in their training syllabus, the appropriate
     mode of completion (i.e., performance, supervision, or
     observation), or the method for evaluating whether the
     operators had mastered the job performance requirements.

     -    The written examinations and operating tests (walk-through
     and dynamic simulator) at some facilities were so basic that
     it was questionable whether the examinations and tests could
     adequately evaluate operator performance or the
     effectiveness of the training or identify areas needing
     improvement.  The questions, job performance measures, and
     scenarios did not test the operators at the comprehension
     and analysis levels of knowledge, but strictly at the
     memorization level.

     -    Some facilities did not sufficiently control how many test
     items were repeated between practice and comprehensive
     examinations or among successive examinations (i.e., week-
     to-week or year-to-year).  Other facilities attempted to
     avoid duplication by revising their dynamic simulator
     scenarios between administrations, but the revisions were so
     superficial that the types and sequence of malfunctions and
     the required operator actions and mitigation strategies were
     essentially unchanged.

     -    One facility that almost always operates with only two
     reactor operators (ROs) on a control room crew, used three
     ROs on some of its dynamic simulator examination crews in
     order to reduce the number of scenarios required to conduct
     the examinations.

     -    The operators at some facilities were given little or no
     retraining on weak areas unless they failed the examination. 
     Sometimes retesting did not sufficiently address areas
     identified as weak.
.                                             Attachment 1
                                              IN 95-24
                                              April 25, 1995
                                              Page 2 of 2


     -    The evaluators at some facilities did not identify areas in
     which retraining was needed to upgrade licensed operator
     knowledge because they graded their operators exclusively on
     the basis of completing critical tasks and did not conduct
     any individual competency evaluations unless an operator
     failed.

Program Evaluation and Revision

     -    Some licensed operator requalification training programs did
     not always close the feedback loop by informing the
     originators of training comments how their concerns were
     resolved.

Examination and Test Integrity

     -    Some facility licensees permitted training personnel who had
     specific knowledge of the examination content to continue
     their routine training activities, thereby introducing the
     appearance of impropriety and the possibility that
     examination integrity could be compromised.

     -    Some facility licensees took minimal action to keep their
     operators separated while individual examinations were in
     progress or to review the examination results for possible
     indications that security had been compromised.

Compliance with Operator License Conditions

     -    One facility licensee failed to ensure that all of its
     licensed operators completed the requalification training
     required by 
          10 CFR 55.53(h) and 55.59(a)(1).

     -    Some licensees were in violation of 10 CFR 55.53(e) and (f)
     because they performed or directed licensed activities
     without meeting the requirements for maintaining an active
     license or because they returned to licensed duties before
     completing the required reactivation training.

     -    Some licensees were in violation of 10 CFR 55.53(i) because
     they did not receive the required biennial medical
     examination. 

     -    In some instances, facility licensees neglected to inform
     the NRC of permanent changes in licensed operator medical
     status (e.g., a medical defect that might necessitate a
     conditional license or disqualify the operator) as required
     by 10 CFR 55.25.
.