INTRODUCTION AND BACKGROUND
The management audit of the Sierra PERMITS automation
system was approved by the Council in the 1991 Auditor's Office Work Program. The audit
was prompted by Council concerns regarding the quality of BALD management and budget
information available during the 1991 budget process, and difficulties in obtaining land
development data for the East Sammamish Community Planning Area "Pipeline Study"
in 1989.
The Sierra PERMITS automation system, a modified
"off-the-shelf" automated permit information and land development management
system, was funded in the 1987 County budget. The system was approved as part of a
comprehensive effort to streamline and improve accountability for the management of the
County's land development and building permit processes. The PERMITS system purchased was
designed to operate from a Hewlett-Packard mainframe computer with users entering and
accessing data on-line from either terminals or personal computers.
The PERMITS software provided a tracking component for
permit activities or land development projects which could be linked with a property
database the situs file. The situs file describes characteristics of individual parcels
and contains Assessor's data and other geographic or special purpose data such as census
tracts, community planning areas, sewer local service areas, etc. The situs database
contains approximately 500,000 parcels.
The initial system acquisition costs were approximately
$650,000 for hardware and software. In late 1990 BALD secured Council approval of a new
5-year lease/purchase agreement for replacement and expansion of the central processing
unit and other enhancements at a net cost of $1,045,329 through 1995.
In 1990 an estimated 47,500 permit activities were recorded
on the system. In mid-1991 BALD reported 400 system users in BALD and Environmental
Division sections and other County agencies linked to the system, I.E., the Environmental
Health, Surface Water Management, and Comprehensive Planning Divisions.
- OBJECTIVES
-
The audit's objectives were to evaluate the effectiveness
of the automated permit processing system in achieving benefits projected during system
development, funding and acquisition phases; to review the current system's operational
effectiveness (specifically management reporting) and recommend needed changes; and to
identify potential EDP audit topics.
- GENERAL CONCLUSIONS
-
The general audit conclusions were that cost-saving and
staffing benefits projected to result from the automated system could not be demonstrated.
System development and implementation had failed to complete key steps which would have
resulted in a system better matched with operational needs. Improvements in training,
documentation, user support, data quality and report generation were needed to improve the
system's effectiveness in serving BALD and other County agencies reliant on system
information.
- MAJOR FINDINGS AND RECOMMENDATIONS
-
Finding 1. The PERMITS system
was installed and operated without a detailed implementation plan, annual work programs,
or a post-implementation review.
Although preliminary, summary level implementation plans
were prepared during the feasibility and RFP phases of system development, detailed
implementation plans were not subsequently used to guide and communicate the
implementation process or provide accountability to BALD in County management. As a
result, Automation Services staff were described as spending time "putting out
fires" and did not complete all phases initially proposed, or work toward established
objectives. Because a post-implementation review was not conducted to determine the
effectiveness of the system elements, operational deficiencies were not identified and
used as a basis for system enhancements.
The audit recommended *that: BALD establish
direction and accountability for Automation Services by requiring an annual work program
and overall system implementation plan for unimplemented features and future enhancements.
The post-implementation review should be conducted to identify unmet user needs and
identify needed enhancements. The Council may wish to authorize an audit focusing on
system controls.
Finding 2. The training,
documentation, and system support provided to PERMITS users was inadequate and
insufficient.
-
A
survey of more than 150 BALD staff in the Subdivision and Commercial/Multi-Family Sections
revealed the following deficiencies:
-
Minimal
training had occurred since the initial system implementation: 26% of respondents received
no formal training; many staff reported being trained only by training from co-workers or
self-training.
-
System
documentation available was either outdated or nonexistent: 40% reported having no user
manuals or other reference materials; 42% reported documentation available was outdated or
inadequate.
-
The
PERMITS system users' group met infrequently, system information was not communicated to
users, and system operational assistance was insufficient: 80% reported receiving no
notification of system changes; 37% reported inadequate access to support staff.
The audit recommended that: BALD provide a
comprehensive training program of introductory and refresher classes on relevant system
features to ensure each user has a common understanding of the system's operation. BALD
should distribute current documentation materials, including descriptions of custom King
County features and a data dictionary, to all system users. BALD should actively expand
the users' group process, designate and train additional system leads in operating
sections, and review and determine Automation Services staffing requirements through task
time logs.
Finding 3. BALD had not
established a systematic data quality control program to ensure the validity of PERMITS
data, and the situs database had not been maintained and updated as planned.
Significant percentages of BALD staff surveyed questioned
the reliability of system data (57% reported not incomplete, 40% reported not current, 35%
reported not inconsistent, 32% reported not inaccurate). The reported problems indicated a
need to objectively determine the validity of system data.
Survey respondents and other County staff reported that the
PERMITS situs data was also incomplete or inaccurate. Updating for some assessment data
and other special purpose district information has not occurred since 1988. Only half (11)
of the 21 data types initially recommended for inclusion were subsequently entered. Those
not entered included park and recreation districts, critical water supply and groundwater
management areas, retention/detention facilities, and on-site system failures.
The audit recommended that: BALD determine the
validity of system data through initial sampling, and establish data quality goals and an
on-going monitoring program through error and exception reports. Designated data entry
staff and mandatory data entry fields should also be considered.
The audit also recommended that: BALD develop a
custom computer program to ensure timely transfer and update of Assessor's data to the
situs database, establish a Situs Maintenance and Data Quality Committee with
representatives from other County user agencies including Assessments, and Systems
Services to establish goals, and recommend appropriate responsibilities for updating and
funding the situs files.
Finding 4. BALD had not
effectively implemented on-line or report writing software to meet operational and
management reporting needs.
Despite the availability of on-line reports and the
expenditure of $50,000 for additional report writing software, the lack of system reports
was a primary focus of user criticisms. Few reports were generated by Automation Services
and a survey of unit supervisors and section leads indicated that only 3 had access to the
on-line report menu.
BALD staff and other agency users cited numerous reports
which would enhance management and productivity. However lack of training, equipment,
documentation and access limited ability to generate reports without Automation Services
assistance.
The audit recommended that: BALD provide effective
reporting capability by facilitating the use of on-line PERMITS reports, training and
equipping unit/sections leads to use reporting software, and improve Automation Services
production of non-routine and standardized reports.
Finding 5. BALD had not
developed or applied criteria to objectively assess need or allocate system hardware.
BALD's administrative practice made staff individually
responsible for entering data related to their work activities into the PERMITS system.
Given budgetary constraints, this practice resulted in competition for system equipment,
frustration and minimal system usage in units with limited equipment availability.
Moreover BALD management had not developed or applied procedures or formal criteria for
objectively determining need and allocating equipment.
The audit recommended that BALD: improve the
effectiveness of hardware distribution by establishing an allocation process based on
usage information to document need and criteria to allocate equipment purchased.
Evaluating lower cost clone hardware and portable field units such as scanners was also
recommended.
Finding 6. The system was
underutilized by other county agencies due to minimal documentation, training, and
reporting capabilities, unreliable remote connections and data quality.
The automated system was developed, justified, funded and
purchased through the efforts of an interdepartmental committee. On-line PERMITS
workstations are remotely located in the Environmental Division (Public Health), the
Surface Water Management Division (Public Works) and the Comprehensive Planning Section
(Parks, Planning and Resources), and the agencies are assessed annual user fees.
Although the type of system use and frequency varied
considerably among the agencies, common problems were reported which generally validated
the deficiencies cited by BALD staff surveyed. We also noted that the inter-agency focus
which characterized the systems' development had diminished.
The audit recommended that BALD and the management
of the respective user agencies implement the following:
