Managing Cloud Spending is a Challenge; Here Are Some Ways Agencies Can Improve

3d rendering of five piggy banks in the clouds.

By Daniel Proctor, TCG Data Analyst

Over the last ten years, spending on cloud services has increased across federal agencies. In 2010, OMB made cloud computing an essential part of government’s plan to modernize IT and IT management, and early in 2011 they issued the Federal Cloud Computing Strategy. Since 2012, agencies have been required to assess all IT investments for cloud services as part of their annual budget submission. According to a GAO report from 2019, 13 of 16 agencies surveyed reported an aggregate $291 million in savings from the cloud. However, the same report details issues on tracking data about costs and savings. In particular, the report states that, “Spending data is not consistently tracked due to a failure to create targets of anticipated savings, inconsistent definitions of cloud spending and savings, and issues with investment management systems tracking capability.” A lack of consistency in efforts to track, capture, and report quality data on cloud spending suggests that agencies need more robust management, governance, and tools for cloud services. Fortunately, open frameworks and tools are emerging to help address these challenges.

Part of the challenge in establishing effective management and governance of cloud services stem from the nature of the services themselves and the ways in which they are different from other IT investments. Measuring the cost savings and cost avoidance of many investments centers on fixed pricing. However, cloud services are based on usage, where the customer pays for performance. Therefore, budgeting for cloud requires organizations to forecast demand, not assess technology for different components. In a recent survey of its membership, the FinOps Foundation (a non-profit trade organization focused on cloud management) reported that forecasting remains a high priority challenge for its members.

Forecasting is particularly difficult for new projects because there is little-to-no historical data upon which to base projections. In these cases, organizations need to create benchmarks by making comparisons to other projects and run scenarios to get a sense of the range of variability—probable minimum and maximum usage and cost. These strategies are certainly not foolproof because the data is often imperfect, so risk management and mitigation strategies are needed for cost overruns. This is particularly true when definitions are fuzzy and there is a lack of clarity around what variables to include when creating models to project demand and cost.

To begin to address this problem and to provide consistency across the government, in 2019, OMB began requiring agencies to use the Technology Business Management Framework for reporting IT spending, including cloud spending by investment. TBM provides a highly structured taxonomy that includes ways to break down costs for services to a granular level, and clearer definitions for tracking cloud spending and savings. As TBM is a data driven approach, it also emphasizes locating authoritative data sources that exist in different organizational areas such as finance, HR, office of the CIO, and project management. While TBM does provide a way to track and report more granular data, it is a solution that requires buy-in from stakeholders across an agency. The work is in locating the data sources and putting systems and processes in place for proper tracking.

TBM is not the only approach to unpacking the challenges of budgeting cloud services. Across the tech sector, resources are emerging to help organizations evaluate the business cases for potential cloud services and ways to increase the efficiency of their current cloud configurations. The FinOps Foundation is focused on codifying and promoting cloud financial management best practices and standards. FinOps is an operating model for the cloud that helps organizations understand costs and make tradeoffs. Like TBM, it is an approach centered on transparency, consistency, and collaboration between stakeholders across an organization, and data-centered decision making. Both TBM and FinOps espouse a change in culture and buy-in from leadership as necessary elements for establishing collaboration for effective tracking and reporting high quality data. To create this culture, FinOps calls for the creation of a cross organization-team from finance, engineering/ops/infrastructure, business/product owners, and the C‑Suite, as well as a FinOps center of excellence.

The complexity of cloud management requires specific tools that can help organizations deal with the heavy lift of cloud management. Spreadsheets are not sufficient. For example, Cloud Custodian is an open source tool that helps automate processes and strengthen security, governance, and management. “Cloud Custodian is a rules engine for managing public cloud accounts and resources.” The product was designed for use in private industry, so there may be issues when using it in the federal government. However, it is an example of a resource that potentially could be leveraged, and as an open source tool it may be applicable to the federal context.

To begin assessing cloud governance and management, we recommend organizations begin discussing some basic questions:

Does your agency have a standard model for forecasting cloud expenditures?
Do modernization efforts of moving from on-premise to cloud have a low or negative cost variance? (What is the cost compared to expectations?)
Are you able to show that cloud use is slowing the growth of your O&M expenditures?
What changes in policies, processes, procedures have you instituted within your organization to better manage cloud resources and spending?
What tools or resources do you leverage to get the most out of your cloud investment?
Do you benchmark internally or against similar agencies’ IT financial management related to the cloud?
What factors or scenarios do you evaluate when moving assets to the cloud? For example, security concerns, cost overruns, software compatibility issues, patching and maintenance, etc.)
Are you able to show your internal clients the cost to the organization of their cloud-systems-and-services usage so they understand the factors that affect the bottom line?

These questions may serve as a useful starting point for developing an understanding of cloud expenditures among stakeholders at your organization.

Within the federal government, cloud has been a priority for over a decade, and agencies have been steadily increasing their use of it. This has generated savings but management and tools need to be as robust as the usage. IT financial management methodologies represented in TBM, best practices represented in FinOps, and specialized tooling like Cloud Custodian enable government leaders to effectively manage the cloud consistently in their agencies.

Daniel Proctor, a Data Analyst at TCG has worked his entire professional life in IT. He was the Project Manager for Data Services supporting the OMB’s Office of the Federal Chief Information Officer during the transition to Technology Business Management (TBM) as part of the annual IT budget exercise and submission to the ITDashboard.gov. He is Lean/Six Sigma, Project Management, and TBM certified and the author of several papers on how to mature an Agency’s TBM implementation in the Federal Government.