Kresge Strategic Learning and Evaluation Officer Arturo Garcia recently spoke on the webinar, “Shifting perspectives: Reinventing impact measurement with nonprofits,” organized by ResultsLab and Grantmakers for Effective Organizations (GEO). Co-speaker Makisha Boothe, founder of the SistahBiz Global Network, together with Garcia and moderator Cindy Eby of ResultsLab, invited participants to reimagine the relationship between nonprofits and funders in the area of evaluation. Including perspectives from a nonprofit and a funder, the webinar explored how meeting nonprofit needs lends itself to greater community impact.

In this blog post, Arturo shares highlights from the webinar around advancing equitable evaluation at a national foundation, as well as additional insights from his lived experience and decades of work on the ground in community and with nonprofits.

To watch the full webinar, check out this video recording of the panel.

The purpose of data collection

For nonprofits, the most important purpose behind collecting and using data is to benefit communities and individual residents on the ground. Having previously worked as an evaluation consultant on the ground with nonprofits, I observed that community organizers and social justice organizations often had to jump through hoops for funders to meet foundation requirements that made little to no sense to community partners. All organizations collect data – formally or informally – whether it be observations, logs, interviews, journals, reflections, mistakes we made or things we did right. Nonprofits already know collecting data is important. The real issue is we can’t collect anything and everything. We have to be strategic.

To be strategic, funders need to take a step back and think about what is our purpose for collecting this data, and how does it benefit our grantees. We need to ask: how are we going to use the data, and for whose benefit. And we need to make sure our values and principles are infused in how we collect and use the data.

It is our responsibility as funders to make sure we really listen to what our grantees and communities need and want – and that we go about collecting data in a completely respectful, honest and culturally responsive way. Sometimes we as funders get stuck in our processes and policies, and we don’t take a second to think about how harmful these practices and processes are to the communities we serve. Basic and unintentional data compliance doesn’t help anyone learn from success, possible impact, why strategies worked and didn’t work and what improvements need to be made.

Kresge’s approach to supporting grantee data and learning needs

Kresge has incorporated an equitable evaluation approach using the Equitable Evaluation Framework – which believes that evaluation, learning and data should always be in service to and for the benefit of the community and nonprofits to advance equity. This equitable evaluation approach is in sync with how our Kresge program teams are leaning more and more into a trust-based philanthropy approach. So, we are all working hard to listen to grantees, understand their needs and incorporate their input into our grantmaking strategies and evaluation design.

Kresge also has some program teams considering more participatory grant models and evaluation approaches. Relationships and building trust are at the heart of them. We need to always start with getting to know what grantees are doing, and what they are trying to accomplish in the long and short term. I also believe it’s important to understand what our grantee partners value as an organization – even on a personal level.

My colleagues and I in Kresge’s Strategic Learning, Research and Evaluation practice work with each program team to help them think through different learning and evaluation approaches that would support grantee learning and organizational needs. This includes program teams considering the other supports which grantees need to succeed in their work – general operating dollars, trainings, connecting them with others outside their network, strengthening existing relationships, technical assistance and more.

An example of this in action is how I’m working with a Kresge program team and a third-party evaluator to evaluate a community of practice cohort. Although we worked with the evaluators to design the evaluation approach and questions, we also made a point of bringing the framework to the grantees to ask for their input on the approach, evaluation questions and timing. Grantees also helped the evaluation team decide on whether to have service providers and families be participants in the evaluation, considering that many service providers and families are still dealing with COVID-19 and its aftermath.

Out of all this work, I’m reminded that we need to demonstrate respect for grantees’ and residents’ time by providing them incentives for participating in evaluations – such as giving gift cards, meals or snacks or childcare. This approach aligns with Kresge’s notion of equitable evaluation and centering community.

Commitment to equitable evaluation bound up in lived experience

My personal commitment to equitable evaluation comes out of my professional and lived experiences. I have a 15-year career in research and evaluation, which fortunately started with community-based, participatory evaluation and research on the ground with nonprofits, social justice organizations and residents in southern California. I experienced first-hand what nonprofits needed to do their work, the importance of their work, and how funder requests were often misaligned with what nonprofits really needed in terms of data and evaluation.

I also had a memorable moment in my work when I realized that I needed to take a step back and figure out what clients and communities really needed, so I could best support their work. I discovered that some of what was needed was clearer, more simplified explanations on what evaluation is, what it can and can’t do, and how to use evaluation findings to make adjustments to programs. Most of all, I found that I needed to help nonprofit partners and communities figure out what they really wanted to know from an evaluation, and how they wanted to use the data and findings. It was important to help partners move beyond traditional ways of knowing and collecting data – not just sticking to quantitative data and surveys.

I also leaned into my lived experiences and philosophical approach and values. I grew up poor, living in low-resourced and disinvested communities in southern California. I faced racism and discrimination as a dark-skinned Mexican American. So, when I was working on the ground as an evaluator in communities around southern California, I could see myself, my parents, families and friends in them. Also core to my equitable evaluation approach is the work of Paolo Freire, a Brazilian educator, activist and philosopher, along with ideas and practices of liberation theology, participatory action research and community organizing have impacted my commitment to ensuring equity in evaluation.

It is important to remember that evaluation is more than the conventional notion of passing judgment on a client’s programs, strategies, ways of thinking or approaching the work. Evaluation is really about centering the communities and people being served. Planning, implementing and adjusting strategies and conducting evaluations should all be with and in service of communities. At the end of the day, all people want is to be heard and valued. So how can we make that a part of what we do, of who we hire to do the important work needed in our communities, and who we hire for our evaluations?

Grantee responses to Kresge’s equitable evaluation approach

Kresge grantees have been asking for more inclusive, equitable and responsive evaluation processes for some time. They have asked us to be more honest with them and to make evaluation processes less cumbersome. In my experience at Kresge, grantees have been appreciative of the steps that Kresge is taking, and the things we’re talking about to be better partners and supporters of grantees’ work.

Grantees want us to keep pushing ourselves to meet them and communities where they are at. It’s up to us to hold our feet to the fire, to listen to grantees and the community when we don’t always get it right. We need to be okay with being uncomfortable – because these are often uncomfortable conversations.

Kresge is on this journey and growing. As more program teams use equitable and inclusive processes in grantmaking and evaluation, we’ll be hearing more from grantees on what we’re doing well, and where we still need to improve.

Advice for funders on an equitable evaluation journey

For funders who are beginning to walk this journey, I urge you not to be afraid to examine diversity, equity and inclusion throughout your foundation’s processes, practices and policies. Examine grant funding strategies and evaluation and data collection practices. Center grantees and communities as you make improvements.

I also encourage foundation leaders to spend time talking to staff, listening to what’s working and not working in terms of processes, policies, protocols and work environment. Most importantly, let’s spend time building and strengthening relationships with our grantees, partners and other stakeholders, seeking to understand their needs and values. Allow the journey to start from there.

Lastly, foundations need to be open to feedback and be okay with sitting with the uncomfortableness that can arise out of it. Don’t react defensively out of fear. Tension is necessary for change. And we need to connect with other peer funders on this journey, whether they’re at the same point in the journey or farther along. We can all help each other out, working together to reimagine the relationship between nonprofits and funders – all for the good of community.