Collecting data with a clear purpose is an important first step. Once you know why you’re collecting it, you can now ask: What does this data actually tell us?
Nonprofits collect plenty of information about their work. Staff track participants served, events held, trainings delivered, community members reached, referrals made, resources distributed, and countless other activities. These numbers can provide useful information about what a program is doing and whether it’s reaching the people it intended to reach. However, problems arise when we expect those numbers to tell us something they can’t. A measure of activity isn’t automatically a measure of impact.
When Attendance Becomes the Measure of Success
Several years ago, I facilitated a focus group for participants involved in a community engagement project. During the conversation, one participant reflected on an in-person event she had organized. She didn’t think the event had been particularly impactful because “not many people” attended.
Her assessment made sense. She had organized an event for community members, so of course attendance mattered. If you spend time planning an event, promoting it, preparing materials, and getting everything ready, you want people to show up. A low turnout can feel disappointing.
As we continued talking, however, she described what happened with the people who did attend. They actively participated and stayed for the entire event. They completed the post-event evaluation, and some remained in contact afterward to learn more about advocating for positive breast and chest feeding experiences and formula support in their communities.
Suddenly, the attendance number didn’t seem like enough information to determine whether the event had made a difference. Attendance told us that the event hadn’t reached as many people as intended. The participants’ experiences told us something else about the depth of engagement among those who were there.
Outputs Tell Us What Happened
In evaluation, we often refer to measures such as attendance, services delivered, and resources distributed as outputs. Outputs document what a program did and the immediate products of those activities.
For example, a nonprofit might report that:
- 75 people attended a financial literacy workshop.
- Staff distributed 300 resource guides.
- 25 community organizations participated in a coalition meeting.
- A mentoring program matched 40 young people with mentors.
- 500 people registered for an online training.
Those numbers aren’t meaningless. In fact, they can answer important questions about reach, implementation, and capacity. If your organization expected 100 people to participate in a program and only 20 enrolled, that’s information worth examining. Similarly, if your funding supported 10 workshops and your organization delivered all 10, documenting that activity matters.
I’ve been asking some version of “does it make a difference?” for years. The problem starts when we treat an output as evidence of an outcome.
Seventy-five people attending a workshop doesn’t tell us whether they learned anything. Distributing 300 resource guides doesn’t tell us whether people used them. Bringing 25 organizations into a meeting doesn’t tell us whether those organizations developed stronger relationships or collaborated afterward. Matching 40 young people with mentors doesn’t tell us about the quality of those relationships.
Outputs can tell us what happened. Outcomes help us understand what changed.
Bigger Numbers Can Feel Like Better Results
Large numbers carry psychological weight. A program that reaches 1,000 people can sound more impressive than one that reaches 100. An event with 200 attendees can appear more successful than one with 25. Organizations also operate in environments where they need to communicate their value to funders, boards, partners, prospective participants, and the broader public. Under those conditions, it’s understandable that reach and volume receive attention.
This is also where vanity metrics creep in. A number can look impressive without providing much insight into whether an organization accomplished what it hoped to accomplish.
Consider community engagement. An organization might report that staff contacted 50 potential community partners during a quarter. Another team might maintain ongoing relationships with 10 organizations, regularly exchange resources, make referrals, collaborate on events, and coordinate responses to community needs. If we only compare the number of organizations contacted, the first team appears to have greater reach. Once we ask about the purpose and quality of those relationships, however, we have a very different set of questions to explore.
Quantity and quality aren’t inherently in competition. Sometimes reaching more people is exactly the goal. The key is knowing what your numbers represent and resisting the temptation to assign meaning they don’t contain.
Pair What You Can Count With What You Need to Understand
If attendance alone couldn’t tell the participant in my focus group whether her event made a difference, what else could she have collected?
That depends on what she hoped the event would accomplish. If the goal was to increase knowledge, she might explore what attendees learned. If she wanted participants to connect with resources, she could examine whether they accessed those resources afterward. If the event aimed to strengthen community relationships, follow-up conversations might help her understand whether new connections continued beyond the event.
This is why meaningful data collection starts with clarity about the change you’re trying to create. Otherwise, organizations can accumulate spreadsheets full of numbers without gaining much insight into their programs.
Quantitative and qualitative data can also complement each other. Attendance data could show how many people participated, while survey responses, interviews, focus groups, observations, or follow-up conversations could provide insight into their experiences. Depending on the program, organizations might also track changes over time rather than relying on information collected immediately after an activity.
The goal isn’t to collect everything. Adding more surveys, interviews, reporting requirements, and data points can create additional work without producing additional insight. Instead, staff should identify the information they need to understand whether their programs are moving in the intended direction.
Let Your Program Goals Guide the Data
Before deciding what to measure, return to what the program is designed to accomplish. If a program exists to increase access to services, participation numbers matter, but so does understanding whether participants successfully accessed those services. If a coalition aims to strengthen collaboration, counting organizations at the table provides one piece of information. Staff may also need to understand whether members trust one another, share resources, coordinate their work, or make decisions collectively.
This approach also helps organizations have more productive conversations about funder requirements. Some metrics may primarily demonstrate accountability, and organizations will still need to collect them. At the same time, staff can identify additional information they need for learning and program improvement.
Rather than expecting every metric to answer every question, give each measure a job. Know what it tells you, what it doesn’t tell you, and what other information you need to understand the bigger picture.
Key Takeaway
Numbers can tell powerful stories about nonprofit programs, but only when we understand what those numbers represent. Attendance can tell you who showed up. Participation counts can tell you how many people you reached. Service numbers can tell you what your organization delivered. None of those measures, on their own, can tell you whether people’s circumstances changed or whether your program achieved its intended outcomes.
As you review your organization’s data, consider what conclusion the data actually allows you to make. Then identify what else you would need to know.
Raise Your Voice: What’s one metric your organization regularly tracks, and what does it actually tell you about the difference your work is making? Share in the comments section below.
Was this useful? Subscribe to the Raise Your Voice newsletter, and explore my consulting services.
