Full Report
OVERVIEW
The Issue
The NASA Office of Inspector General (OIG) conducted this audit to determine whether the fiscal year (FY) 2008 budget for the Constellation Program (CxP) was prepared in accordance with Office of Management and Budget (OMB) and NASA policy and whether the CxP budget request was supported by well-documented cost estimates. We focused our review on the five major CxP projects: the Crew Launch Vehicle, Crew Exploration Vehicle, Ground Operations, Commercial Crew Cargo Capability, and Mission Operations. The Commercial Crew Cargo Capability is considered a project under CxP in NASA's 2008 budget request and in this report; however, for program management purposes, it is its own program and is not a project under CxP. The CxP projects are intended to send a new generation of explorers to the Moon, Mars, and beyond. The project mission for the Crew Launch Vehicle (Ares I) is to deliver a safe, reliable launch system for the Crew Exploration Vehicle that will take astronauts to the International Space Station and then to the Moon by 2020. Reliable cost estimates can help ensure that budget requests are adequate to meet project and program milestones and goals.
NASA's FY 2008 budget request1 totaled $17.3 billion and included $3.1 billion (17.7 percent) for CxP. Budget requests for the five major CxP projects totaled $2.8 billion, which included $2.3 billion in direct costs. Our review covered $1.1 billion of those direct cost estimates. Details of the audit's scope and methodology are in Appendix A.
Results
NASA could improve its budgeting process by providing additional guidance to ensure that budget requests are supported by well-documented cost estimates. We found that while the CxP project managers had supporting documentation for the $1.1 billion of the direct cost estimates that we reviewed, the majority was at the summary level and not sufficiently detailed. Specifically, CxP project personnel provided summary-level documentation for $914.6 million and more detailed documentation for $164.4 million. Although the summary-level documentation met Agency requirements for documenting budget requests and provided insight into the process used to estimate the direct costs, it did not include detailed documentation, such as the source data, calculations, and explanations, that would facilitate reproducing and independently evaluating the estimate.
For example, project personnel for one of the five projects, Commercial Cargo Crew Capability, provided limited supporting documentation for that project's FY 2008 budget request of $193 million in direct costs. Project personnel stated that they used market research and vendor quotes to develop the cost estimates that supported the budget request and provided summary-level documentation that illustrated various cost scenarios in a PowerPoint presentation. However, the summary-level data did not include detailed evidence to show how the cost estimates were derived by market research, company responses, or vendor quotes.
A reasonable and defensible budget request should be supported and documented by sound cost estimates. OMB requires agencies to collect, identify, and analyze data for cost estimates; document cost estimates; and explain the cost estimating process used so that the estimate can be reproduced, allowing the quality of the estimate to be independently evaluated.
NASA requirements for formulating, approving, implementing, and evaluating programs and projects requires cost estimates to be consistent with guidance in NASA's "Cost Estimating Handbook," 2004. The Handbook provides useful information to assist the NASA community in the development of reliable, comprehensive, defensible, and well- documented cost estimates, stating that documentation for the estimates should provide sufficient information about how the estimate was prepared to allow an independent analyst to reproduce the estimate. However, the Handbook does not explain what constitutes sufficient information.
Given the absence of detailed guidance on what constitutes sufficient documentation, we evaluated the CxP budget request against standards recommended by the Government Accountability Office (GAO) in its July 2007 exposure draft, "Cost Assessment Guide: Best Practices for Estimating and Managing Program Costs." Although not available at the time the FY 2008 CxP budget request was submitted, and still in draft format as of August 4, 2008, the guiding principles for preparing fully documented cost estimates contained in the GAO Cost Guide are a compilation of best practices from across the Federal Government that can serve as a benchmark.
We found that the CxP project managers' documentation did not include the level of detail called for in either NASA's Handbook or the GAO Cost Guide. With respect to the Handbook, the documentation provided by the CxP project managers did not provide sufficient information to allow an independent analyst to reproduce the estimates. The GAO Cost Guide states that the documentation should include source data and significance, clearly detailed calculations and results, and explanations of why particular methods and references were chosen. The Guide also states that the data in a well- documented cost estimate must be traceable to its source.
NASA could improve its budgeting process by adopting the standards recommended by the GAO Cost Guide and ensuring that budget requests incorporate cost estimates based on historical or actual cost data, vendor quotes, and spreadsheets with detailed calculations prepared by subject matter experts showing how they arrived at the cost estimates. Also, given that NASA programs and projects have historically experienced cost overruns, cost estimates with detailed, empirical data that explain the rationale for decisions could help minimize the risk of cost overruns by providing additional assurance that budget requests are adequate to achieve program and project goals.
Management Action
We recommended that the Associate Administrator for Program Analysis and Evaluation incorporate into NASA's Cost Estimating Handbook appropriate best practices identified in the exposure draft of the GAO Cost Guide and require personnel to document and retain cost estimates used to support budget requests in a manner that explicitly identifies the primary methods, calculations, results, rationales or assumptions, and source data used to generate each estimate. We also recommended that the Constellation Program Manager require that budget requests be prepared using cost estimates that are sufficiently documented as to allow an independent cost analyst to reproduce the estimate.
In response to a draft of this report issued August 12, 2008, the Associate Administrator for Program Analysis and Evaluation nonconcurred with our first recommendation but stated that NASA incorporated the best practices from the exposure draft of the GAO Cost Guide into its 2008 Cost Estimating Handbook. By incorporating the GAO standards into the Handbook, management's actions are responsive to the recommendation.
The Associate Administrator for Program Analysis and Evaluation concurred with our second recommendation and stated that CxP has been and will continue to apply the techniques and best practices outlined in NASA's Cost Estimating Handbook. The techniques and best practices are in concert with the GAO Cost Guide. We consider management's comments to be responsive. Both recommendations will be closed upon verification of management's actions.
1 http://www1.nasa.gov/pdf/168652main NASAFY08 Budget Request.pdf, also available at http://www.nasa.gov/news/budget/FY2008.html; accessed August 12, 2008.
Full Report
