Sustainable Growth and Improvement for as Little as 1%: How Evaluation Capacity Democratizes Evidence Building
2020 Invest in What Works Federal Standard of Excellence
The Foundations for Evidence-Based Policymaking Act of 2018 (Evidence Act) incentivizes building evaluation capacities in the federal government; however, the law did not come with funding attached for implementation, leaving agencies to locate or redistribute resources to fully implement the Act, including its evaluation components. The resources criteria (criteria 3) in Results for America’s Invest in What Works Federal Standards of Excellence (Federal Standard of Excellence) helps agencies to quantify their evaluation related spending. Criteria 3, which quantifies current spending as well as changes from previous years, allows agencies to track their evaluation spending over time and provides a snapshot of whether investments have increased to meet the additional requirements created by the Evidence Act.
Specifically, the resources criteria in Federal Standard of Excellence (3.1) encourages agencies to calculate their evaluation spending; (3.2) has them track their evaluation budget from year to year; and (3.3) establish that resources are available to build evaluation capacity of grantees, including local and state governments. These sub-criteria encourage agencies to fund their research and evaluation capacities with at least 1% of their budget to conduct a fair and thorough evaluation of programs in order to build towards more effective programming, funding, and overall action.
This spending allows agencies to meet both the spirit and letter of the Evidence Act. It is by dedicating increased resources to evaluations that agencies can consistently build on their practices and identify what is working, what is underperforming, for whom, and why. From national evaluations, impact evaluations, process evaluations, technical assistance, open-source data, and even staff training, providing resources to the evaluation and activities is incredibly important and builds a reliable infrastructure for continuous improvement.
Zooming out, tracking the level of federal evaluation spending is particularly important because the public sector in investments in research compared to the private sector. According to the National Science Foundation, in 2018, the United States invested close to $580 billion into research and experimental development, $422 billion of which came from the private sector. Overall, R&D investments represented over 2.8% of America’s GDP. However the federal agencies featured in the Federal Standard of Excellence only spent .82% on average within their own budget. If federal invested in R&D as much as the private sector does, it would allow the U.S. public sector to optimize results faster, quickly, and with more impact.
Of the nine agencies featured in Results for America’s 2020 Federal Standard of Excellence, the Millennium Challenge Corporation (MCC), Administration for Community Living with HHS (ACL), U.S. Agency for International Development (USAID), and AmeriCorps spent at least 1% of their overall budgets on research and evaluation. These agencies that have made the resource investment into evaluation through the budget have not only been able to build a framework that makes their evaluations more accessible, it also moves to make their evaluations more efficient.
An example of such efficiency is demonstrated by the Administration for Community Living (ACL) within the Department of Human and Health Services (HHS), which has invested $22.2 million on evaluations, evaluation technical assistance, and evaluation capacity-building, representing slightly over 1% of the agency’s $2.0 billion FY20 budget. From this relatively small yet impactful investment, ACL has been able to identify trends in their programming where efficiency could be enhanced or where it has diminished. One insightful example of this is a look at ACL’s evaluation of the Older Americans Act Nutrition Services Program. In 2020, the ACL reported that based on changes in program costs or funding levels many state agencies reported reducing staff or staff hours (47%), reducing the number of days of service per week at congregate locations (34%), reducing the number of congregate nutrition sites (33%), and reducing the frequency of home-delivered meals (32%). The ability to reference the detraction of so many variables from the previous years was a direct result of the consistent investment ACL made in evaluations over a number of years.
Likewise, grantees and programs that participate in evaluations under the Department of Labor (DOL) receive technical assistance related to evaluation activities and implementation such as the Evaluation and Research Hub (EvalHub). DOL program offices are making a concerted effort to help states and local areas build evaluation capacity to meet evaluation requirements for the Workforce Innovation and Opportunity Act (WIOA) and Reemployment Services and Eligibility Assessment (RESEA) through tools such as the RESEA program evaluation technical assistance. This suite of evaluation technical assistance resources that include webinars and other helpful tools supports states in understanding, building, and using evidence. Importantly, steps are also being taken to ensure that the 11 technical assistance webinars are posted online for ease-of-access. Initiatives such as these are crucial to building resources for evidence building and supporting federal partners at the state and local levels.
That said, DOL has the ability to increase its investments in evaluation and achieve even greater impact. Since 2014, Congress has authorized the U.S. Secretary of Labor to set aside DOL operating funds for evaluations. Between FYs 2014 and 2016 the secretary was authorized to transfer 0.5 percent of DOL operating funds to the Department’s Chief Evaluation Office, and between FYs 2017 and 2020 the secretary was authorized to set aside 0.75 percent. However, despite Congress’s authorizations, DOL has not taken full advantage of this ability to make robust investments in evaluation. For example, in FY20 DOL transferred only .09% of its budget to the Chief Evaluation Office. But even this small amount represented an increase of the .01% set aside for evaluation in FY19. As such, new leadership at DOL has the opportunity to increase investments in evaluation by taking full advantage of this authority to set aside operating funds for evaluation purposes.
Building on DOL’s authority to set aside funds, the White House could increase the government’s evidence-generating capacity by directing agencies to use their existing administrative authority to set aside no less than 1% of discretionary competitive grant program funds for evidence-building. Likewise, Congress could authorize or mandate such spending.
This is especially relevant in the context of the American Rescue Plan. The Plan provides an opportunity for the federal government, states, and localities to build knowledge of what works (and for whom) as the country spends unprecedented amounts of money to recover from the public health and economic crises.
The federal government as a whole has the opportunity to be more expansive in its thinking about how to fund research and evaluations by investing in various types of methodologies or research tools to enhance capacity building across all agencies to understand the impacts of programs in communities across various demographics. The federal government has the power to serve as a funder of research and evaluation, and thus, in the interest of continuous improvement, it should take on this role to support the services provided to communities and residents across the country.
Read Results for America’s 2020 Invest in What Works Federal Standard of Excellence here. And read more about the Resources Criteria here.
