2021 Invest in What Works Federal Standard of Excellence Agency Snapshots: The Who and The What

Launched in 2013, Results for America’s annual Invest in What Works Federal Standard of Excellence (Federal Standard of Excellence) serves as a “north star” for how federal agencies and departments can consistently and effectively use data and evidence in budget, policy, and management decisions to achieve better outcomes for their residents.

The following Federal Standard of Excellence Agency Snapshots provide a glimpse into nine agencies’ evidence-based policy agendas. The Snapshots highlight key accomplishments and areas for future progress. Many of these practices and efforts described below are exemplary models that all federal agencies and departments can follow as they seek to improve results for communities, families, and children across the country.

In the recently released 2021 Invest in What Works Federal Standard of Excellence, the nine leading federal agencies and departments include:

• Millennium Challenge Corporation (Gold)
• U.S. Department of Education (Gold)
• U.S. Agency for International Development (Gold)
• Administration for Children and Families (within HHS) (Silver)
• AmeriCorps (Silver)
• U.S. Department of Labor (Bronze)
• U.S. Department of Housing and Urban Development (Bronze)
• Administration for Community Living (within HHS) (Bronze)
• Substance Abuse and Mental Health Services Administration (within HHS) (Bronze)

To learn more about how these agencies and departments continue to lead in federal evidence-based policy, visit the 2021 Federal Standard of Excellence here. More information on the Foundations for Evidence-Based Policymaking Act is available via Results for America’s Evidence Act Resource Hub and Evaluation.Gov. Further information about executive orders and memorandum on using data and evidence to advance equity in government is available here.

Millennium Challenge Corporation

As a foreign assistance agency, accountability is core to the Millennium Challenge Corporation’s (MCC) organizational culture. This culture is reinforced by MCC’s evaluation leaders who oversee the agency’s performance, research, and evaluation activities. This commitment is further supported by the agency’s robust investment in research and evaluation: 2.2% of the agency’s budget, or $17.6 million in FY21.

Beyond monitoring, MCC has increased its focus on sharing research and evaluation information with country partners, development assistance organizations, and the general public in recent years. This approach equips stakeholders to make better use of evidence-based approaches and accelerate results. Specifically, in 2021, MCC will launch a new Evidence Platform that provides a one-stop, virtual data enclave for users to access and use public- and restricted-use data. The Platform encourages research, learning, and reproducibility and connects datasets to analytical products across the portfolio.

The Evidence Platform will also host the Evaluation Briefs, launched in FY19, which distill key findings and lessons learned from MCC’s independent evaluations. As of October 2021, MCC has published 107 Evaluation Briefs, which are also published in local country languages. As of 2021, MCC will produce Evaluation Briefs for each evaluation moving forward, and is in the process of writing Evaluation Briefs for the backlog of all completed evaluations.

MCC is working to strengthen its evidence frameworks for research and evaluation. Further, the agency has named climate inclusion & gender as key priorities for the agency, and is working to incorporate the existing women’s economic empowerment learning agenda to include evidence generation and utilization around gender and inclusion (in addition to women’s economic empowerment) in MCC’s programming.

Read more about the Millennium Challenge Corporation in the 2021 Invest in What Works Federal Standard of Excellence here.

U.S. Department of Education

The U.S. Department of Education (ED) has been a long-time leader in evidence-based policy. Convening key personnel from across the Department has been a driver of ED’s efforts to use information about what works to drive decision-making. This engagement of important staff has continued with the implementation of the Foundations for Evidence-based Policymaking Act (Evidence Act) where the Evidence Leadership Group, ED Data Governance Board, the Office of Planning, Evaluation, and Policy Development, Office of the Chief Data Officer, Grants Policy Office, Office of Evidence-Based Practices and State and Grantee Relations, the Institute for Education Sciences (IES), and other units lead the Department’s Evidence Act implementation. According to evaluation.gov, ED has published an Annual Evaluation Plan for FY22 and will be releasing a Learning Agenda in February 2022.

Beyond Evidence Act implementation, the Evidence Leadership Group helps program staff to use evidence in grantmaking in programs across the agency, including support of the Departments legislation: the Every Students Succeeds Act (ESSA). ESSA has been a key driver of increased evidence use in states across the country (including in Nevada as detailed in a 2019 Results for America case study). ED’s five largest grant programs require some form of an evaluation report on a yearly basis to build evidence, demonstrate performance improvement, and account for the utilization of funds. The Department’s What Works Clearinghouse, through its evidence reviews, Intervention Reports, and Practice Guides, plays a key role in helping teachers, leaders, and researchers identify and apply evidence-based interventions.

ED also provides robust technical assistance through the Regional Education Laboratories and Comprehensive Centers, which help states and districts build and use evidence. In 2020, the 10 RELs collaborated to produce a series of evidence-based COVID-19 resources and guidance on teaching and learning in a remote environment and on how to address other issues that have arisen for schools as a result of the pandemic.

Another example of ED’s support to states and districts includes the state longitudinal data systems grant administered by IES, which in FY21 provided up to $1 million over three years for states and local education agencies to better gather, analyze, and evaluate data about student performance. ED is focused on efforts to disaggregate outcomes by race and other demographics and to communicate those results to internal and external stakeholders. For example, to facilitate clear and open reporting on the implementation of the Coronavirus Aid, Relief, and Economic Security (CARES) Act, in Q1 of FY21, the Office of the Chief Data Officer launched the Education Stabilization Fund (ESF) Transparency Portal, allowing ED to track performance, hold grantees accountable, and provide transparency to taxpayers and oversight bodies. Moving into FY22, the portal will include other disaggregated student data collected through the Elementary and Secondary School Emergency Relief (ESSER) Fund program such as the number of students that participated in various activities to support learning recovery or acceleration for subpopulations disproportionately impacted by the COVID-19 pandemic.

The department should proceed with its Evidence Act and OPEN Data Government Act implementation, based on forthcoming White House Office of Management and Budget guidance.

Read more about the U.S. Department of Education in the 2021 Invest in What Works Federal Standard of Excellence here.

U.S. Agency for International Development

The U.S. Agency for International Development (USAID) continues to be a leader in data-driven innovation and evidence-based investing. As part of these efforts, the Agency invests in research and development to scale effective innovations through the Grand Challenges for Development grant competition. To date, this initiative has funded $155 million in grants and technical assistance for 528 innovations in 107 countries. More than $614 million in follow-on funding has been generated from external sources as a result of these innovative investments. A similar program, the Development Innovations Ventures (DIV), considers evidence of effectiveness to fund and scale grantees with innovative solutions. Since 2010, DIV has invested $149 million in nearly 225 innovations across 47 countries, benefitting over 55 million people globally.

To solidify these approaches, USAID has continued to build its capacity for innovation and evidence-based policymaking with the support of key research and evaluation leaders. In FY19, the agency appointed a Chief Innovation Officer to advocate and promote a multi-sector innovation strategy. In FY20, USAID increased the coordination of its evidence and data leaders by holding regular meetings between its Chief Data Officer, Chief Evaluation Officer, Statistical Officer, and the leaders of the Office of Learning, Evaluation and Research. These leaders are focused on continuous learning to make sure that the Agency is continually improving results, while also implementing the Foundations for Evidence-based Policymaking Act. For example, in FY21, the USAID Bureau for Policy, Planning, and Learning (PPL) began the process of revising the agency learning agenda to incorporate priorities from the Biden-Harris Administration. In March 2021, USAID published its Annual Evaluation Plan for FY22, on the Agency’s Development Experience Clearinghouse (DEC), which is the repository for disseminating evaluation, research, and other evidence publicly. The Annual Evaluation Plan includes significant evaluations aligned with the agency’s learning priorities from the learning agenda.

Beyond evaluation, USAID has a sophisticated approach to collecting and leveraging disaggregated data. While LGBTQ+ data may be politically complicated or potentially unsafe to collect in certain country contexts, USAID data can be disaggregated by geographic location, region, or state, which can be mapped with other demographic data to build a picture of disparities where data gaps exist. Country expertise can then be applied to analyze racial and ethnic equity dimensions. Taken together, all of these capacities, tools, and structures help USAID continue to leverage evidence to build knowledge and drive results-oriented investments.

In the coming year, USAID should continue to gather public and stakeholder input on its draft common evidence framework to boost the quality of its evaluations. Further, the agency should consider consolidating its disparate, issued-based clearinghouses into a publicly available, unified evidence clearinghouse, much like the platform on which the FY22 Annual Evaluation Plan is published.

Read more about the U.S. Agency for International Development in 2021 Invest in What Works Federal Standard of Excellence here.

Administration for Children and Families
U.S. Department of Health and Human Services

Even prior to the Foundations for Evidence-Based Policymaking Act (Evidence Act), the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services invested in a robust approach to infuse data and evidence in budget, policy, and management decisions. ACF was among the first of federal agencies to publicly release an agency-wide evaluation framework with its FY12 policy “to govern [the agency’s] planning, conduct, and use of evaluation.” The agency’s Deputy Assistant Secretary for Planning, Research, and Evaluation leads the Office of Planning, Research, and Evaluation (OPRE) and oversees an evaluation team of 70 staff. In FY21, ACF had a total research and evaluation budget of $175 million.

ACF’s long-standing expertise in research and evaluation helps provide support to grantees on evaluation, evidence-building, data-driven innovation, and implementation of evidence-based programs. One example of this capacity building is through the agency’s efforts to support culturally responsive evaluation. The new African American Child and Family Research Center is intended to lead and support research on the needs of African American populations served by ACF. Another example is the TANF Data Innovation Project that supports cohorts of states to improve the effectiveness of Temporary Assistance for Needy Families (TANF) programs by helping them better leverage human services data. In FY21, the agency published findings from a TANF Data Innovation Needs Assessment to improve agency technical assistance while supporting states with linking data for evidence building.

In FY21, the HHS annual evaluation plan features planned evaluations related to three ACF priority learning questions on TANF and job retention; quality of Head Start programs and the experiences of children and families in the programs; and programs for child safety and well-being. These activities are helping states and grantees improve program and service delivery for children, families, and parents across the country.

ACF’s Head Start competitive grant program considers grantee past performance as a condition of continued funding through the Head Start Designation Renewal System. In two smaller grant programs, Personal Responsibility Education Program and Sexual Risk Avoidance Education, grantees must implement evidence-informed or research-based interventions. The Family First Prevention Services Act (FFPSA) enables states to use federal funds to provide enhanced support to children and families and prevent foster care placements through the provision of evidence-based services. ACF sponsors an independent systematic review to designate programs as evidence-based and eligible for federal funds.

To advance the agency’s investments in evidence-based policymaking, Results for America recommends that ACF consider expanding the program areas covered by robust evidence clearinghouses to better identify what works. RFA recommends that Congress increase funding for research and evaluation activities, including the recruitment of data and evaluation staff and experts, even through statutory changes.

Read more about Administration for Children and Families in the 2021 Invest in What Works Federal Standard of Excellence here.

AmeriCorps

Over the past several years, AmeriCorps has been the federal government’s leader on evidence-based investing. In FY21, the agency’s flagship grant program, AmeriCorps State and National, invested the majority of its grants in interventions with a moderate or strong evidence base. The allocation of 68% of funds to evidence-based grantees in FY21, a 17% increase from FY20, constitutes a major achievement and is delivering real impact in communities across the country.

This milestone is a result of the agency’s approach to grantmaking which provides preference to grantees that propose evidence-based programs. In 2021, the agency also released a new toolkit to help grantees prepare to scale effective evidence-based interventions, determine if interventions are ready for scaling, and ensure that scaled interventions produce meaningful outcomes. Through this toolkit, the agency is supporting the field to embrace evidence-based practices, build robust evidence, while also building their capacity to compete in the evidence-based market.

The Office of Research and Evaluation provides critical support in increasing the agency’s evidence-based investments. The Office created resources to help the national service field identify and implement evidenced-based interventions and also provided individualized technical assistance to grantees to help them evaluate their efforts.

Along the way, the Office of Research and Evaluation also continued to build the overall capacity of the agency by developing key resources such as the Strategic Evidence Plan and Evidence Exchange. The agency’s learning officer and other colleagues in the Office led the development of a concept paper for a National Service Equity, Evidence, and Innovation Fund to guide the use of data and evidence to advance equity. Finally, although a non-CFO Act agency, AmeriCorps hired its first Chief Data Officer, which will support Evidence Act and enhance agency wide data practices and policies.

To advance the agency’s investments in evidence-based policymaking, AmeriCorps should address its need for an updated and comprehensive suite of data practices, policies, and processes that allow the agency to strategically leverage and incorporate national datasets into decision-making processes.

Read more about AmeriCorps in the 2021 Invest in What Works Federal Standard of Excellence here.

U.S. Department of Labor

The U.S. Department of Labor (DOL) was one of the first agencies to create the position of a Chief Evaluation Officer, paving the way for the Foundations for Evidence-based Policymaking Act (Evidence Act), which requires agencies to designate such a position in order to build a centralized capacity for research and evaluation. DOL’s Evaluation Policy was also a model for federal guidance for agencies. Several examples of the long-standing federal leadership are demonstrated by DOL’s Clearinghouse for Labor Evaluation and Research(CLEAR), a model federal evidence clearinghouse and DOL’s commitment to publishing public use data for researchers’ use. The data are generated from DOL-funded evaluations.

With the passage of the Evidence Act, DOL built on the important leadership of the Chief Evaluation Office (CEO) by formalizing evidence leadership across the department, including the roles of the Statistical Official, Evaluation Officer, and Chief Data Officer (CDO), who leads the department’s Data Board. The department publicly shared this information by developing a new website, dol.gov/evidence.

An important coordination brought about by the Evidence Act was to bring together evidence-building activities led by the Chief Evaluation Office and performance management activities and planning coordinated by the Performance Management Center. The aligned FY22–26 Strategic Plan and Evidence-Building Plans will be made available on the department’s Evidence Hub at dol.gov/evidence in February 2022. An example of performance improvements include updates to the Workforce Innovation and Opportunity Act (WIOA) allows state and local governments to build their own performance infrastructure by using federal funds for data collection, performance management, research, and evaluation activities. WIOA encourages state and local governments to link funding to performance and evaluation data through performance-based grants and contracts.

Prior to 2021, the appropriation for evaluation was constant at $8 million, with the Secretary’s given authority to transfer an additional set aside amount (up to 0.75%) from Department accounts to fund research and evaluations efforts led by CEO. Additionally, DOL invests in evidence-based grantees through three competitive grant programs: Senior Community Service Employment Program, the National Farmworker Jobs Program, and the national YouthBuild program. Evidence-building benefits the public through responsible expenditures of taxpayer resources on effective practices. To advance its efforts, DOL should stabilize evidence-building investments to support improvements in the department’s expansive policy and investment portfolios.

Read more about the U.S. Department of Labor in the 2021 Invest in What Works Federal Standard of Excellence here.

U.S. Department of Housing and Urban Development

The U.S. Department of Housing and Urban Development (HUD) has been a consistent leader in taking a strategic approach to research and evaluation. Even before agency learning agendas were required by the Foundations for Evidence-based Policymaking Evidence Act (Evidence Act), HUD’s learning agenda, the Research Roadmap, linked the Department’s performance management, research, and evaluation activities. With the Evidence Act in place, HUD has issued an updated Research Roadmap, informed by an exemplary stakeholder engagement process that the Department has developed over the years to identify key research questions from the field. In addition to HUD’s robust research and evaluation portfolio, HUD remains a leader in federal performance management systems — for example, in FY21, the agency has engaged its Chief Financial Officer in performance monitoring. The Department has made strides in leveraging administrative data in a complementary relationship with evidence-building and program demonstration. Additionally, the department has increased its emphasis on generating interim reports during long-term impact evaluations.

Beyond using its own research to build evidence, HUD provides resources to help states and localities build their own capacity for using evidence and data. In FY20 and FY21, the Community Compass program provided $91 million for technical assistance to help grantees effectively use federal funding, including improving program management, evaluation, and performance measurement. Also in FY21, HUD offered $3.45 billion for the Department’s Community Development Block Grant program (CDBG), in which funding for housing, community and economic development, and disaster recovery is designed to also build fiscal health and administrative capacity, such as capacity for data collection, analysis, and outcome tracking. The program authorizes recipients to use up to 20% of their allocations for administration and planning costs that may include evaluation-capacity building efforts and evaluations.

HUD has exemplary equity practices. For example, the department publishes foundational research to promote more equitable community development and housing policy via the HUD USER platform on issues such as housing discrimination and programs that increase economic opportunity for disadvantaged and underserved populations. Key findings of evaluations are disaggregated by effect size, population, and sub-group demographics. Additionally, as a part of the agency’s COVID-19 response, HUD published a suite of racial equity housing resources, toolkits, and research to reduce overrepresentation of people of color experiencing homelessness.

To improve its evidence-building and knowledge about effective housing programs, HUD and congress should provide clear guidance for states and localities about leveraging the CDBG 20% set aside for evaluations, research, evidence-building, and data activities, as described above. The agency could also benefit from a set of department-wide evidence definitions, stemming from its existing evaluation policy, for research and funding purposes that create alignment and clarity on the spectrum of evidence-based practices.

Read more about the U.S. Department of Housing and Urban Development in the 2021 Invest in What Works Federal Standard of Excellence here.

Administration for Community Living
U.S. Department of Health and Human Services

The Administration for Community Living (ACL), an operating division within the U.S. Department of Health and Human Services, first participated in the 2018 Invest in What Works Federal Standard of Excellence, and has since accelerated its efforts to build an organizational culture focused on performance and research.

ACL’s centralized capacity for performance, research, and evaluation is housed in the Office of Performance and Evaluation (OPE). The Director of OPE serves as the agency’s evaluation and performance officer with responsibility for coordinating Foundations for Evidence-Based Policymaking Act (Evidence Act) implementation within the operating division. The OPE Director also serves on the HHS data council, HHS Data Governance Board, and Federal Interagency Council on Evaluation Policy. In 2021, OPE gained more staff and capacity to support learning and research. As part of its growing efforts to increase the agency’s evidenced-based policy capacity, OPE provides staff training on evidence-based grantmaking, which will enhance the agency’s ability to invest in better results and outcomes.

Of particular note, ACL is committed to implementing the Evidence Act even though, because of its status as a component of a CFO Act agency, it is not mandated to do so. In 2020–2021, the agency made major strides in meeting the requirements of the Evidence Act by issuing a FY23 Evaluation Plan, developing an Interim Learning Agenda, participating in the development of an HHS-wide Evidence Capacity Assessment, and drafting a primer on data governance (akin to the Data Governance Body that sets and enforces priorities for managing data as a strategic asset required by the Evidence Act). The agency is also using National Standards for Culturally and Linguistically Appropriate Services in Health and Healthcare to inform its evaluation framework. This growing investment in, and capacity for, research and evaluation will benefit the agency as it continues to execute its mission to serve aging populations across the country.

These efforts to build evidence are also expanded on through the ACL’s grants. ACL supports evidence-building activities through non-competitive grants, which aim to gather and report best practices in the Caregiver Support Services program; improve service delivery through the State Councils on Developmental Disabilities planning processes; and adapt and scale evidence-based programs for children and older adults with disabilities through the RESILIENCE Rehabilitation Research and Training Center. During the pandemic, the agency’s focus and emphasis on learning was applied to the National Institute on Disability, Independent Living, and Rehabilitation Research, which produced research on the access challenges face coverings pose for the deaf and hard of hearing as well as recommendations to better support those with hearing impairments.

In future years ACL will continue to focus on improving evidence-based grantmaking strategies as formal practices in ACL competitive and non-competitive grant awards.

Read more about Administration for Community Living in the 2021 Invest in What Works Federal Standard of Excellence here.

Substance Abuse and Mental Health Services Administration

The Substance Abuse and Mental Health Services Administration (SAMHSA) has demonstrated a commitment to evidence-based grantmaking. In FY21, the agency actively participated in the Federal Standard of Excellence for the first time since FY19, when the agency opted not to participate. The agency described efforts to implement provisions of the Evidence Act in coordination with HHS’s department-wide efforts. For example, SAMHSA has developed a learning agenda that is going through OMB clearance, contributed to HHS departmental evaluation plan, and an internal-facing evaluation policy, the Evaluation Policy and Procedure. Beyond that, the agency reported spending 1.8% of its FY21 agency budget on research, evaluation, and evaluation-related activities, among the highest percentages of the nine participating agencies.

Beyond these core and foundational evaluation activities, SAMHSA has developed guidance for its competitive grants. The Developing a Competitive SAMHSA Grant Application provides information applicants will likely need for each section of the grant application. The agency’s five largest competitive grants require grantees to describe their evidence-based practices (EBPs). In addition, if applicants plan to implement services or practices that are not evidence-based, they must show that these services/practices are effective by citing research provided by the Evidence-Based Practice Resource Center. Similarly, the SAMHSA Community Mental Health Block Grant, a noncompetitive formula grant, still maintains a 10% set aside for evidence-based interventions to address the needs of individuals with early serious mental illness, including psychotic disorders.

All SAMHSA grant programs require grantees to submit data on race, ethnicity, gender, and sexual orientation (among other demographic data). Finally, SAMHSA’s surveys collect national data in these areas allowing SAMHSA’s Office of Behavioral Health Equity (OBHE), to utilize federal and community data to identify, monitor, and respond to behavioral health disparities.

In 2021, the OBHE engaged in a renewed focus on racial equity, diversity, and inclusion. Through this office, grantees submit Disparity Impact Statements (DIS) to ensure SAMHSA programs are inclusive of underserved racial and ethnic minority populations in their services, infrastructure, prevention, and training grants. The DIS framework is based on the principles of Access, Use, and Outcomes:

• Access: Who are the subpopulations being served by the program?
• Use: What types of services does each subpopulation get?
• Outcomes: Given the specified outcomes of the program, how do these vary by subpopulations?

This OBHE office is poised to support the agency in advancing its equity agenda aligned with focus on evidence, research, and evaluation.

In the years prior to FY20, SAMHSA had a public-facing evaluation policy that governed research and evaluation activities across the agency. The agency should consider making its Evaluation Policy and Procedure (P&P) public to demonstrate SAMHSA’s commitment to transparent and sophisticated approaches to research and evaluation. Further, SAMHSA should improve the Evidence-Based Practice Resource Center to resemble other national evidence clearinghouses that are designed with states and grantees users in mind to aid their selection and implementation of mental health and substance abuse evidence-based interventions.

Read more about the Substance Abuse and Mental Health Services Administration in the 2021 Invest in What Works Federal Standard of Excellence here.