A Next-Generation Definition of “Evidence-Based”

Since Congress codified it in 2016, the multi-tiered definition of “evidence-based” in the Every Student Succeeds Act (ESSA) has helped advance the use of data and evidence to drive continuous improvement in education. Now, it’s time to continuously improve the definition itself. Results for America is excited to announce a new collaborative effort to articulate a next-generation definition of “evidence-based.” Our plan is to consult diverse stakeholders throughout the education ecosystem, reflect together on the strengths and weaknesses of the current definition, and ultimately generate a new proposal for the field that updates the current definition by addressing shortcomings, centering equity, promoting innovation, and supporting the thoughtful use of research evidence. Below, we explain why we’re pursuing this effort, why we’re beginning it now, and how we plan to approach the work.

Why should we revise the definition? In our recent 2021 Moneyball for Education Policy Recommendations report, we made the case for exploring a next-generation definition:

ESSA’s definition of “evidence-based” . . . represented an important breakthrough in evidence-based policymaking, but it was only a first step that would need improvement, and there is a growing consensus that the time is ripe to improve it. For instance, the current definition does not acknowledge either the importance of considering a full range of evidence as part of selecting an intervention or the relevance of the existing evidence to the problem being solved. A revised definition would incorporate these concepts and seek to align ESSA’s programmatic definitions, IES’s research definitions and, ideally, the definitions in other federal statutes both within and beyond education laws. This alignment would make it easier for practitioners to navigate the evidence base as they search for the best strategies to support students. Furthermore, any revision process would create space for ED and the broader education field to grapple with important questions about the relationship between evidence-based interventions and equity, including emerging questions about how to democratize evidence-based decision-making.

In addition to improving the definition for use in education, we hope this process will ultimately lead to better alignment across different policy areas if other relevant laws adopt it as well. Greater coherence and consistency could make it easier for policymakers and practitioners to collaborate across policy sectors and blend and braid federal funding streams to improve outcomes for children and families.

Five years into implementation of the current definition (and its accompanying federal guidance), we’ve seen meaningful progress but also some important challenges. Particularly in school improvement plans and competitive grants, ESSA’s use of the definition helped shift more federal funding to evidence-based approaches. The Education Innovation and Research program built upon this momentum through its tiered-evidence grantmaking structure, through which proposals with stronger supporting evidence are eligible for larger grants. State departments of education, school districts, and technical assistance providers have taken a variety of approaches to build practitioners’ understanding of the definition, from building clearinghouses of evidence-based approaches to integrating the definition into support structures and continuous improvement models. They’ve also formed communities of practice, peer networks, and other opportunities to help practitioners and policymakers better understand and use evidence well. Meanwhile, all federal agencies, including the Department of Education, are continuing to implement the Foundations of Evidence-Based Policymaking Act.

Like with any new federal policy, however, implementation of the definition has revealed shortcomings, and both practitioners and researchers have run into setbacks. Some issues stem directly from the specific words in ESSA’s definition. After five years, we think we all have enough experience with the current definition to revisit the language and attempt to make improvements.

It’s important to note that some challenges are less about the definition itself and more related to, among other things, the inherent complexities of evidence-based policymaking, the lack of capacity in the field to use evidence thoughtfully, the shortcomings of the existing evidence base, and systemic inequities. Even though our work on a next generation definition cannot fully address these challenges, we will catalogue them along the way and seek new ways to help address them, particularly with respect to racial equity concerns that are not directly addressable via a revised definition.

Why are we doing this now?

We believe the best time to reflect and revise is before Congress is in active negotiations over a potential legislative vehicle for a new definition. Doing so will allow for a more thorough and inclusive approach as well as provide time to build consensus for any proposed revisions. There are a number of potential entry points for a new definition, including reauthorizations of ESSA, the Education Sciences Reform Act, or the Higher Education Act, all of which expired in the past few years and thus could become opportunities for adopting a revised definition.

There are other overlapping and important contexts that make now a strategic time to begin developing a next-generation definition. Particularly in light of the ongoing pandemic, investing our resources in approaches that are most likely to be effective has perhaps never been more critical. Given the unprecedented circumstances of the moment, we must also continue to invest in building new evidence to understand whether, how, and for whom recovery and redesign investments are making a positive impact.

Similarly, in the ongoing efforts to make education more equitable, we must grapple with every aspect of the existing system, including how we decide what is (and isn’t) “evidence-based.” When we rely on definitions that are not inclusive or useful and especially when we do not respond to valid criticisms from the field, we run the risk of practitioners and policymakers rejecting the whole evidence movement outright. There are important efforts underway to revisit how we build and use evidence; we seek to join these efforts with a specific focus on the definition and how it can contribute to dismantling rather than reinforcing systemic inequities.

How are we planning to approach this?

Although we are still developing our specific process for the next several months of this initiative (and expect to revise it along the way especially with the input and advice of others), we are committed to the following design principles:

• Our process will be inclusive — We will engage with a wide range of voices and perspectives from across all corners of the education ecosystem, including those most proximate to schools and communities. We will consistently ask who might benefit and who might be harmed by any potential changes. We will see and reconsider long standing assumptions and biases.
• Our process will be additive — We know many others have been doing important work on various aspects of this topic for a long time. We are not looking to reinvent any wheels and will instead seek to understand and build upon existing ideas.
• Our process will be practical — We know that policies are only as effective as their ability to be implemented. We will keep practical considerations in mind throughout the process, reflecting on the prior five years of implementing the current definition and listening to the experiences of practitioners, researchers, and other stakeholders who have been grappling with the existing statutory language for some time.

How can you get involved?

One initial effort to embrace all three design principles is to make an open call for feedback on the current definition and/or suggestions for how it could be revised. We also welcome recommendations for what to read, who to reach out to, and how we can best approach this important work. Please click here to share your thoughts.

For more information, please reach out to Sara Kerr, Vice President of Education Policy Implementation at Results for America.