A Performance Director, Program Evaluator, Data Scientist, and Policy Wonk Walk into Philadelphia City Hall…
By Carrie S. Cihak, Chief of Policy, King County & Results for America Local Government Fellow; with Michael Jacobson, Deputy Director of Performance & Strategy; Priti Mody-Pan, Senior Performance & Strategy Analyst; Vince Vu, Senior Performance & Strategy Analyst
As a Results for America (RFA) Local Government Fellow, I am in a cohort of 16 performance and policy directors from large cities and counties across the United States who are working to advance the use of data and evidence in local government decision making. In our convenings together, I’ve had the opportunity to learn from my peers about leading-edge practices and the successes and challenges faced when implementing evidence-based policy making.
Over the last two years, I’ve been inspired by many conversations with Dr. Anjali Chainani, Director of Policy and GovLabPHL for Philadelphia Mayor Jim Kenney. I’ve followed the progress of GovLabPHL — an initiative Anjali has spearheaded to harness behavioral science approaches to improve results of city programs. So, when RFA offered to sponsor a site visit for me and a group of senior King County performance leaders to one of the other governments in my cohort, Philadelphia was at the top of my list!
Accompanying me on the site visit to Philadelphia were three of King County’s performance and strategy leaders: Michael Jacobson, Deputy Director of Performance and Strategy and Priti Mody-Pan and Vince Vu, both Senior Performance and Strategy Analysts. We each come to evidence-informed decision making from different yet complementary professional perspectives: I’ve been working in policy for a couple of decades, Michael is a nationally-respected leader in performance management, Priti has deep experience with program evaluation and social science research, and Vince is a data scientist with both public and private sector experience.
The King County team’s learning objectives for our engagement with Philadelphia were:
· Understand how Philadelphia has strengthened and systematized evidence-informed policymaking work across a complex government organization;
· Explore how Philadelphia has built successful academic-government research partnerships;
· Learn how Philadelphia has applied behavioral science approaches in government programs and what results this has produced.
We spent two full days meeting with Anjali, her policy team, departmental champions implementing data and evidence practices in Philadelphia government, and their university research partners. We achieved our learning objectives and learned more than we envisioned — both from the team in Philadelphia and from each other. We came away impressed by the number of completed, active, and in-queue behavioral science projects that Philadelphia is undertaking, and the strong partnerships between city staff and academic researchers. From increasing recycling, to reducing sales of tobacco to youth, to engaging the city’s most vulnerable populations in the design of services — City of Philadelphia residents can take a lot of pride in their government.
Here are 5 key takeaways from the King County team — lessons other local governments can learn from the City of Philadelphia.
1. Evidence is Powered by Relationships (Carrie)
Philadelphia’s biggest investment has been in building the relationships — inside and outside the government — that are fueling continuous learning and improvement. This was evident in the variety of meetings we joined throughout our two days in Philadelphia.
First, Anjali has built a core policy team of four people who provide the project management, process support, and encouragement for the many behavioral science projects underway in Philadelphia. The team has worked together to compile their learnings into useful materials such as a project map, and the criteria that make for a successful project.
Second, the policy team supports a monthly GovLabPHL meeting of about 20 representatives from across departments who are working with the policy team on projects. Department representatives rotate in setting the agenda and chairing the meeting, and this community of practice serves as a place to learn from one another and support each other in overcoming challenges.
Finally, the team has invested in strong and sustained relationships with researchers from Philadelphia’s academic community — particularly from University of Pennsylvania’s Behavioral and Decision Sciences program and Swarthmore College. We observed relationships between city staff and academic partners that extend beyond transactional exchanges on individual projects to a commitment to work together to advance positive outcomes for Philadelphia’s residents.
2. Think Prospectively, Act Locally (Priti)
Evaluation has traditionally been used to take a retrospective look at whether a program has produced results. As much as these tools can be used to benefit our work, they’ve also been used to cut resources for programs and services, creating a confusing culture that can undermine the use of data and evidence.
Taking a prospective view and engaging with program staff in a process of learning oriented toward program improvement builds a culture of trust and support for evidence-based decision making. The more we approach our work from a client-centered perspective to meet program staff needs (e.g. How can we quantify and measure program success? What problems are departmental leaders trying to solve? How can we test different approaches to enhance outcomes?), the better-equipped we’ll be to bring value to our residents.
GovLabPHL’s exceptional customer service to city agencies was evident in the glowing terms their stakeholders, such as the leadership in Human Resources, spoke of the policy team as having an “anything is possible” attitude, impeccable follow-through, and taking the time to build trust.
3. All Roads Lead to Evidence (Vince)
In a cycle of continuous learning and practical constraints, methodologically sound is a better strategy to pursue than methodologically pure. An analysis or experiment can try to incorporate scientific rigor in many different forms, even if it does not take the form of a classic randomized control trial. There are many low-cost ways for us to improve the systematic collection of data and put it to use — including harnessing existing processes and finding new uses for administrative data.
For example, Philadelphia conducted a series of experiments in an attempt to reduce street litter. In order to get accurate data on if the experiment was working, Philadelphia needed to incorporate visual surveys of street litter. Rather than introduce brand new data collection processes, the city instead leveraged existing staff activities. Since Sanitation staff were already on the streets performing routine tasks and collecting other types of data, it was easy to introduce a step to collect data for the experiment. By combining this with existing administrative data, such as the incidents of illegal dumping or excessive street litter, the city was able to build a robust data set without major impacts to budget or operations.
4. Starting Simple has Big Payoffs (Michael)
Small changes can produce big results. Starting with a project not too complex lets you quickly demonstrate value and build from there. Behavioral approaches and rigorous evaluation can be integrated into what we are already doing and support our other performance management efforts.
One challenge every jurisdiction faces is we are already maxed out on existing programs and initiatives, so finding a way to get something new started can be a challenge. Philadelphia’s focus on “nudging” behavior through changing the wording of a letter, evaluating different signage, or doing customer journey mapping were ways to start small to build interest so that departments would see value and pull more innovation support from the central policy team.
One example of a small scale intervention and assessment in Philadelphia was finding ways to have youth in the city’s summer job program sign up for a direct deposit. Since research shows that having a bank account is a success factor for youth, getting the youth get a direct deposit into a bank account (vs. a check) allowed them to create an account with a track record to help them establish credit and promote savings.
5. Innovation Fueled by Philly’s Amazing Food Scene!
The evidence is in! Philly has a great food scene and we came away believing that this must — at least in part — fuel the innovation and excellence we saw in city government. We ate our way through James Beard award-winning chef Michael Solomonov’s restaurants: Israeli laffa and mezze at Zahav, the eponymous treats at Federal Donuts, the creamiest hummus at Dizengoff, and falafel and tahina shakes (vegan!) at Goldie.
Back in King County, we shared our learnings with our colleagues from the Office of Performance, Strategy & Budget over a hummus lunch and learn. Carrie made a creamy Turkish hummus from the Zahav cookbook and chef Vince made flat breads and tehina shakes! Though we didn’t conduct a rigorous experiment, our logic model supports the provision of these Philadelphia treats as a factor in strong interest and attendance.
Be on the lookout for additional posts on the second leg of our trip — to Washington, D.C.!
