Evidence-Based Policymaking in Our Nation’s Capital: A Resource Guide to 7 Top Practices
By Carrie S. Cihak, Chief of Policy, King County & Results for America Local Government Fellow
As a Results for America (RFA) Local Government Fellow, I am in a cohort of 16 performance and policy directors from large cities and counties across the United States who are working to advance the use of data and evidence in local government decision making. In our convenings together, I’ve had the opportunity to learn from RFA and my peers about leading-edge practices and the successes and challenges faced when implementing evidence-based policy making.
Earlier this year, RFA offered to sponsor a site visit for me and a group of senior King County performance leaders to one of the other governments in my cohort to engage in more in-depth learning from each other. At the top of my list were Philadelphia and the District of Columbia (DC) whose Local Government Fellows — Dr. Anjali Chainani and Jennifer Reed, respectively — are spearheading strong academic-government research partnerships that are producing impressive results for their residents. Philadelphia and DC are pursuing these partnerships through two different models, and I spent quite a bit of time debating over which site visit would be more relevant for our work in County. In the end, I couldn’t choose, so our King County team decided to visit both!
Accompanying me on the site visits to Philadelphia and DC were three of King County’s performance and strategy leaders: Michael Jacobson, Deputy Director of Performance and Strategy and Priti Mody-Pan and Vince Vu, both Senior Performance and Strategy Analysts. We each come to evidence-informed decision making from different yet complementary professional perspectives: I’ve been working in policy for a couple of decades, Michael is a nationally-respected leader in performance management (recently elected for induction as a Fellow in the National Academy of Public Administration (NAPA)!), Priti has deep experience with program evaluation and social science research, and Vince is a data scientist with both public and private sector experience.
Interested readers can learn about our 5 takeaways from the Philadelphia portion of our trip in an earlier Medium post here. In this post, I provide a guide to 7 top evidence-based policymaking practices in our nation’s capital.
The King County team’s learning objectives for our DC site visit were:
- Understand how different governmental entities are advancing a culture of evidence across large organizations;
- Learn from the practical experience of experts who have applied tools such as learning agendas, rigorous evaluation, and data analytics in government;
- Explore models for academic-government partnerships.
We spent two days meeting with representatives from a variety of different institutions who have all played an instrumental role in advancing the evidence-based policymaking movement: an independent federal agency that provides support across federal departments, an independent non-profit research institute, a bipartisan policy organization, a private center on government effectiveness, and DC’s local government — a state, county, and city government all wrapped into one. Though media reports tend to focus on our nation’s capital as stymied in partisan politics, we came away impressed by how much is happening behind the scenes to help government deliver results.
Here’s an overview of what we learned — our resource guide to 7 top efforts to strengthen evidence-based policymaking in our nation’s capital.
- The Evidence Act
Many readers may be familiar with major acts of Congress such as the Affordable Care Act, Every Student Succeeds Act, and the Lilly Ledbetter Fair Pay Act. But have you heard of the Foundations of Evidence-Based Policymaking Act, signed into law at the beginning of 2019?
One of the King County team’s first meetings in DC was with Nick Hart, former Director of the Evidence Project at the Bipartisan Policy Center and now CEO of Data Coalition. (and recently elected with Michael to be a NAPA Fellow). The Evidence Act, sponsored by Senators Paul Ryan (R-WI) and Patty Murray (D-WA), adopts many of the recommendations of the U.S. Commission on Evidence-Based Policymaking, which delivered its final report to the President and Congress in 2017. The Evidence Act is an important milestone in supporting the use of government data to produce evidence that can be used in policy and decision making.
Nick provided us with a great overview of the key components of the Evidence Act and related guidance working its way through federal agencies. The Evidence Act recognizes government data as a strategic asset, and sets guidelines for data management to promote research and evidence while strengthening data privacy and security. The Evidence Act also requires federal agencies to invest in the leadership and practices needed to support a culture of evidence — Chief Data Officers, Chief Evaluation Officers, and the development of learning agendas (see more below), evaluation plans, and open data plans.
As is the case for any major local government, King County’s work is profoundly affected by what happens at the federal level. So, it was incredibly valuable for the King County team to hear firsthand about the Evidence Act’s implementation from someone who has been instrumental in its development. As well, we in King County are interested in advancing some of these policies and practices here at home.
Want to learn more about the Evidence Act? RFA is a strong supporter of the Act and has developed this great resource guide! You may also be interested in following RFA’s Federal Standards of Excellence (I serve on the Advisory Board), which assesses federal agencies’ implementation of evidence-based practices.
2. The Office of Evaluation Sciences
The U.S. General Services Administration (GSA) is an independent federal agency that provides mission support services across the federal government — think managing federal real estate, purchasing, technology, and the like. In 2015, through the establishment of the Office of Evaluation Sciences (OES), GSA added rigorous evidence-based research methods to the services and support it provides across US government agencies.
We met with Dr. Kelly Bidwell, Director of OES, and a team of about 20 applied researchers to hear how they are working with federal agencies to test and learn about what works. In partnership with more than a dozen federal agencies, OES has completed over 70 tests, achieved $8 million in cost savings and revenue, and helped millions of Americans save for retirement, get vaccinated, or enroll in college and federal benefits.
We benefited from hearing about how OES has applied lessons from behavioral science to increase the positive impact of government services. Even more importantly, we learned so much from how OES has established strong working relationships between academic researchers and government policymakers and the best practices they have established to guide their work — a strong benchmark against which to measure our emerging work in King County. This includes practices such as pre-registration of evaluation designs and publication of all results that help establish the credibility of their work and more rapidly spread learning. The culture of partnership and these best practices have resulted in a virtuous cycle where the OES fellowship program attracts top academic talent eager to work with and learn from government decision makers. If you are interested in being an OES fellow, check out their recruitment process underway now for fall of 2020.
You can go on your own “site visit” of OES by watching Kelly and OES fellows talk about their work in this video. Better yet, if you are in the DC area, join OES’s annual event on October 30 where they will discuss what they’ve learned over the past year.
3. The Lab @ DC
The Lab @ DC is an exciting local government initiative to advance evidence-informed decision making. Based in the Office of the City Administrator, The Lab is comprised of about 20 full- and part-time data scientists, operations experts, and social science researchers who seek to improve the lives of DC residents by using science to design and rigorously test the impact of various policies, programs, and services. The Lab @ DC has applied their work to everything from police body worn cameras, to flexible rent subsidies to prevent homelessness, to the redesign of forms to make city services more accessible.
I’m so excited by The Lab’s work for three reasons.
First, by centering a team of scientists in the heart of DC city government operations, The Lab is tackling the some of the city’s most significant issues. For example, DC has a high number of 911 calls relative to its population and many calls result in transporting people to emergency rooms for conditions that could be treated in primary or urgent care clinics. Using a randomized control trial evaluation, The Lab’s data and social scientists are testing whether a nurse triage line linked to the 911 center improves care. In designing the evaluation, The Lab had to figure out a way to randomize callers to connect with a nurse without interfering with the 911 operators’ ability to respond to callers as quickly as they always do. The pilot is now complete, but in some respects The Lab’s work is just beginning. The analysis of results relies on Medicaid billing data which takes months to settle and be accurate. The Lab expects to report results in 2020.
I’m also excited by the approach The Lab is taking to work directly with front-line DC government employees to improve their engagement with and service delivery to residents. A great example of this participatory approach is the Lab’s annual Form-A-Palooza. Through Form-A-Palooza, The Lab and DC employees have worked with DC residents in day-long workshops to redesign 37 city forms. The user experience with forms is important, not only because forms are often an access point (or barrier) for residents to city services, but forms can also communicate (intentionally or not) the District’s values. For example, in redesigning the District’s school enrollment packet, gender nonbinary residents pointed out the form’s male-female checkboxes didn’t apply to them. Adding a nonbinary checkbox was a simple way for DC government to acknowledge and respect nonbinary students.
Finally, I’m excited about The Lab’s focus on advancing good public policy both inside and outside DC through stellar communication about its work. The Lab adheres to “open science” principles such as pre-registration of research designs and publication of study results regardless of the findings. In addition, The Lab @ DC makes learning about social science research not only informative, but fun! Check out The Lab’s blog posts, lunch speakers, and podcasts.
The King County team was totally inspired by our visit. Perhaps our future holds The Lab @ KC?
4. Learning Agendas
One of the key practices emphasized in the Evidence Act is the development of agency learning agendas. Simply put, a learning agenda prioritizes the key questions an agency needs to answer to increase the impact of its work and identifies the activities the agency will undertake to answer these questions. Ideally, the development of a learning agenda engages a wide range of internal and external stakeholders and identifies a range of activities from qualitative interviews, to reviewing existing literature, to analyzing descriptive data, to conducting rigorous experimentation and evaluation.
The King County team was fortunate to meet with Demetra Nightingale and Keith Fudge at the Urban Institute. Demetra led development of one of the federal government’s first agency-wide learning agendas during her tenure at the Department of Labor and Keith was engaged in development of the Department of Housing and Urban Development’s Research Roadmap. Now at Urban Institute, Demetra and Keith have documented their knowledge and experience, along with that of other federal agencies, regarding this best practice in two toolkits found here and here.
Leaning on this work, King County has developed a learning agenda to guide evidence-building around the impact of fares on access to public transportation. King County Metro developed one of the country’s first and largest transit fare discounts for people with low income. We are seeking to understand how the lower fare program — known as ORCA LIFT — impacts mobility and other outcomes such as access to employment, housing, and education. This evidence building is tied to upcoming decisions regarding how we can further improve mobility (both physical and economic) to our residents. For example, through our research partnerships with the Wilson Sheehan Lab for Economic Opportunities at Notre Dame, University of California-Irvine, and Stanford University, we are working to understand how even the discounted ORCA LIFT fare may pose a barrier for the lowest-income people, along with other factors that impact their access to mobility. In addition, we are working with several other jurisdictions around the country who are also exploring fare discounts based on income to learn from one another’s research — a multi-jurisdiction learning agenda on this topic.
As the transportation landscape evolves quickly, and our population continues to grow rapidly, King County is also exploring development of a broader learning agenda around mobility and public transportation. Learning firsthand from colleagues at the federal level on how they advanced this work has been incredibly valuable.
5. IBM Center for the Business of Government
Most readers probably know of IBM for its technology services and the hardware driving many of our computers. But did you know that IBM also hosts the DC-based IBM Center for the Business of Government that focuses on bridging the gap between research and practice to improve the effectiveness of government management and operations?
The King County team met with John Kamensky, a Senior Fellow at the IBM Center. John has a couple of decades of experience in public service at the federal level, where he played a leadership role in Vice President Gore’s work on Reinventing Government and the passage of the Government Performance and Results Act (affectionately known as GPRA).
Listening in on the conversation between John and Michael Jacobson was like listening to a duet between two accomplished musicians who really know and like one another’s style. They played off of, as well as with, one another. Our conversation explored the key complementarities as well as distinctions between performance management and evidence-based policymaking practices. For example, where we already have quite a bit of evidence about an intervention, performance management can be the best tool to create impact by maintaining our focus on program fidelity and efficiency. In places where government is innovating and trying something new, we may need to build evidence about impact through rigorous evaluation of whether the innovation delivers the intended results. This complementarity and distinction is evident, for example, in King County’s Evaluation and Performance Measurement Plan for our Best Starts for Kids initiative.
6. The National Museum of African-American History & Culture
King County is committed to leading with racial equity, so a visit to the National Museum of African-American History and Culture was an important part of our time in DC. While we typically think of “evidence” in the government policymaking context as rigorous experimentation and quantitative analysis of program interventions, the type of evidence gathered at the NMAAHC is foundational to understanding so much about our present-day communities as well as our lived experience as individuals. We cannot hope to eliminate racial disparities without a deep exploration of both the injustices and the resilience documented at so vividly at the NMAAHC.
Interestingly, both types of evidence are meeting in a project at the Lab @ DC. The Lab @ DC is working with the Metropolitan Police Department (MPD) on an evaluation of a new training program for DC police officers that includes a visit to the NMAAHC. The hope is that, by equipping officers with a better understanding of the role of policing in racial injustice in DC and the United States, MPD can help improve community policing outcomes. We all look forward to learning from the results!
7. Evidence: Fueled by DC’s Amazing Restaurants!
As the King County team shared our site visit agenda with our colleagues in DC, the most common response we received was kudos for our dining selections and a question, “How’d you get a reservation at The Dabney?”
Our answer to this question exemplifies our learnings from the site visit about how to advance evidence-based policymaking in local government. It requires rigorous research and preparation, persistence in keeping an eye out for opportunities, and a focus on building relationships. Sometimes, like with dining at The Dabney, you really need to plan ahead. At other times, like eating at my favorite restaurant Little Serow, you have to be adaptive and willing to adjust your plans based on circumstance.
We DID make great dining choices. But where we ate was (perhaps) not as important to the success of our visit as the act of listening and responding to one another at the end of the day — each of us was excited about different dishes as well as different ideas, the entrée or topic chosen by one was enhanced by the beverage or observation added by another member of the team. And though we may soon forget specifically what was on the menu or agenda for our visit, the camaraderie and relationships we’ve built in “the other Washington” will endure.
