TABLE OF CONTENTS

Introduction
Audit Objectives
General Conclusions
Major Findings:

Finding - Although King County has an information technology strategic plan, the plan has not been updated and was not followed when selecting projects.
Finding - King County has not used a standard, repeatable process to plan, select, and monitor technology projects; projects were implemented without a clear understanding of their costs, benefits, or risks; project managers were not accountable for meeting performance goals; and a history of projects has not been developed for use in future projects.
Finding - King County does not use a standard, comprehensive methodology to conduct cost-benefit analyses for information technology projects.
Finding - King County lacks a policy regarding the use of contingency funds for information technology projects.
Finding - The IRC operates in place of the Data processing policy review committee (DPPRC), although the DPPRC is mandated by the King County Code. Consequently, there is no legal authority for the IRC, its current membership, or its responsibilities.
Finding - The IRC and its subcommittees have not been fully effective in providing project review, primarily because the IRC structure is not conducive to unbiased decision-making or critical review and analysis.

The management audit of the information technology planning, development, and implementation processes was requested by the Metropolitan King County Council and included in the 1998 Auditor’s Office work program. The audit was prompted by council concerns regarding the adequacy of the processes used to plan, develop, and implement information technology projects throughout the county.

AUDIT OBJECTIVES

GENERAL CONCLUSIONS

The general conclusions are that:

• King County is not using a strategic plan to select which information technology projects to fund;
• projects are approved without a clear understanding of their costs, benefits, or risks;
• project managers are not accountable for meeting project performance goals;
• a standard, comprehensive methodology is not used to conduct cost-benefit analyses of projects, including the establishment of contingency funds;
• the IRC is operating in place of the Data Processing Policy Review Committee (DPPRC) although the DPPRC was established by ordinance and still legally exists; and
• the IRC structure is not fully effective in providing the level of project review needed to ensure that informed decisions are made regarding information technology projects.

FINDING 2-1  Although King County has an information technology strategic plan, the plan has not been updated and was not followed when selecting projects.

The county’s first information technology strategic plan, published in January 1996, stated that it could be considered accurate for only 12 to 18 months. However, responsibility for updating the plan was not established; consequently, it has not been updated. Thus, there was no comprehensive direction to drive which projects were approved, and some high priority projects in the plan were never funded while other projects not in the plan were funded. The lack of a current plan means that projects are evaluated mostly on their individual merit and not necessarily as part of the county’s broader technology needs, or that projects may be evaluated based solely on short-term criteria, such as cost, rather than on how they meet the county’s long-term strategic needs. Without a current strategic plan and a clear process for project approval, projects will continue to receive funding on an ad hoc basis.

The audit recommended that the executive establish responsibility for developing and updating the information technology strategic plan, and that the responsible entity create a new strategic plan and develop a policy regarding how to consider projects not in the strategic plan.

Finding 3-1.  King County has not used a standard, repeatable process to plan, select, and monitor technology projects; projects were implemented without a clear understanding of their costs, benefits, or risks; project managers were not accountable for meeting performance goals; and a history of projects has not been developed for use in future projects..

The county’s procedures did not meet industry guidelines for a standard, repeatable process that would maximize the benefits of technology investments while minimizing the risks. None of the project plans contained all the elements of a complete business case, and the elements included often lacked substance and clarity. Despite these shortcomings, as well as errors and inconsistencies in project planning documents, the projects were approved and funded.

The results of not having a complete business case to aid in screening, monitoring, and reviewing technology projects were:

• project approvals were not based on a clear understanding of costs, benefits, or risks;
  the likelihood of scope creep was high due to the lack of clearly defined project requirements and deliverables;
• project managers were not held accountable for adherence to performance, cost, and schedule goals; and
• lessons learned in post implementation reviews were not used to refine the planning and management process.

The audit recommended that the executive define and clarify the components of the business case that must be submitted for project funding; establish a consistent process for screening, monitoring, and post implementation review; and build and maintain a history of projects to aid in future planning.

Finding 4-1   King County does not use a standard, comprehensive methodology to conduct cost-benefit analyses for information technology projects.

King County does not have a clear and thorough process for estimating the capital or operating and maintenance (O&M) costs of its information technology projects. Based on industry standards of what a cost-benefit analysis should include, the county’s analyses contained numerous deficiencies, including the lack of supporting documentation, detailed cost-benefit estimates, O&M cost estimates, and cost estimate updates during project implementation. Other deficiencies included inconsistencies in how estimates were developed, the inability to track changes in estimated costs, and the exclusion of relevant costs.

Criteria established by industry professionals recognize that a project’s final results should be evaluated against the project plan to provide accountability for the planning process and credibility for future project plans. However, deficiencies in the county’s cost-benefit analyses indicated poor project planning, gave little or no assurance that projects would be implemented or maintained within their established budgets, and increased the potential to need additional funds to complete projects.

The audit recommended that the executive, in coordination with the Budget Office, develop or adopt a cost model to assist county agencies in developing their cost-benefit analyses for information technology projects, and that the executive develop policies and procedures that require:

• detailed documentation to support cost-benefit analyses;
• the business case to include an O&M cost payment plan;
• tracking of cost and benefit changes among documents;
• independent validation of cost and benefit estimates and documentation of reasons for deviations from those amounts;
• project managers to manage costs by cost category and report to the executive when a category exceeds its budget by more than 10%, based on the percentage of completion;
• accountability for project results based on the approved business case; and
• lessons learned to be fed back into the planning methodology to improve the reliability of future cost-benefit analyses.

Finding 4-2   King County lacks a policy regarding the use of contingency funds for information technology projects.

The lack of a policy regarding how to establish a contingency amount for information technology projects resulted in significant differences in the amount of contingency funds allocated to projects. Although industry standards recommend linking the contingency to risk factors specific to a project, this was not done and the contingency factors used appeared to be purely arbitrary. Additionally, once contingency funds were included in a project’s adopted budget, project managers used them to compensate for poor project planning rather than treating them as reserve funds to be used for unknown or uncertain conditions.

The audit recommended that the executive develop policies for determining an contingency factor appropriate to the information technology project being considered and requiring contingency funds to be managed separately from the project account.

Finding 5-1   The IRC operates in place of the Data processing policy review committee (DPPRC), although the DPPRC is mandated by the King County Code. Consequently, there is no legal authority for the IRC, its current membership, or its responsibilities.

The DPPRC was established by ordinance in 1991 and still legally exists, but the IRC has been operating in its place since 1996 without legal authority to do so. Because the DPPRC was established by ordinance, it must be abolished by ordinance. In addition, the membership of the IRC differs from that of the DPPRC, and some functions of the DPPRC have become outdated and been omitted from the IRC charter.

The audit recommended that the executive draft and present to the Metropolitan King County Council an ordinance to abolish the DPPRC and establish the IRC.

Finding 5-2   The IRC and its subcommittees have not been fully effective in providing project review, primarily because the IRC structure is not conducive to unbiased decision-making or critical review and analysis.

The IRC was established, in part, to ensure that decisions regarding information technology project investments are made in the best interests of the county as a whole. However, projects submitted have not undergone in-depth review or analysis in either the IRC or its subcommittees. Although all projects were supposed to go through a review process, there were no criteria regarding which projects had to go through the IRC or for the level of detail required. This meant that projects could be approved and implemented without IRC review. One primary reason for the lack of detailed review was that the responsibilities of IRC members were secondary to their primary responsibilities as department directors. Thus, other demands on their time limited their ability to devote the time necessary to make informed decisions regarding information technology projects.

The audit recommended that the council choose to either:

A.  retain the IRC with its current structure and responsibilities, or

B.  retain a modified form of the IRC and establish a permanent group of project review staff to provide technical assistance during project planning and implementation.