12 Jan, 2017

Try This: The World Cafe

By Nicole Clark, LMSW|2021-08-19T19:02:13-04:00January 12th, 2017|Categories: Research & Evaluation|Tags: program evaluation, Research, Try This|0 Comments

What’s the first image that comes to mind when you think of a focus group?

Mostly likely a small group of people–typically between 8-12 in size–gathered around a table, responding to a facilitator’s questions while being recorded. There may be some snacks involved.

Focus groups are a qualitative data activity used to gather feedback on a product, program or service from a group of people who have some type of commonality, such as age, race, gender, work experience, etc. The information they share can help the organizations or businesses gain a better understanding of why something is working, why it’s not working, and how that something impacts their lived experience.

If you’ve ever facilitated a focus group, you might know how boring they can be. And if you’ve been in a focus group, you definitely know how boring they can be. But what I’ve noticed is this: Focus groups are not boring, technically. It’s how they are structured that makes them boring. There are more engaging ways to conduct a focus group. One of those methods is the World Cafe.

First, some background

From The World Café: Living Knowledge through Conversations that Matter, the World Cafe is a methodology that invites large group dialogue. While in a basic focus group, participants are asked a question and discuss it openly, the World Cafe takes it a step further by allowing for a larger group of participants to be in the space. While they are discussing the question amongst themselves, the conversation flows more freely because it’s a conversation amongst the group rather than the group responding directly to the facilitator. Also, the group is helping the facilitator collect data in a more dynamic, participatory way.

Based on recommended World Cafe design principles, the focus of the World Cafe:

Set the context– What is the purpose for bringing people into the space, and what do you (and the group) want to achieve?
Create hospitable space– Pay attention to how the space is set up. Is it comfortable and inviting? If accessibility is a need, does the space function so that those with certain needs are comfortable in the space?
Explore questions that matter– You can explore a single question or you can develop questions that build on each other. Either way, you’ll be able to synthesize the data to explore common themes
Encourage everyone’s contribution– Encourage everyone to participate in ways that work for them. While some find it easier to express themselves verbally, others may find it better to draw or simply listen
Connect diverse perspectives– The key to the World Cafe is sharing perspectives. When participants move about the room (more on this later), they’re meeting new people, seeing ideas and thoughts that have been generated, and sharing new insights
Listen together for patterns and insights– Shared listening is the key factor in the success of a World Cafe. This is the way to look for themes. Encourage people to listen for what is not being spoken along with what is being shared
Share collective discoveries– Once the World Cafe is complete, invite participants to reflect on patterns, themes and questions expressed and to share them with the larger group

Here’s a video explanation of the process.

Here’s what you’ll need:

(more…)

2 Nov, 2016

Ask Nicole: What’s the Best Way to Deliver Bad News?

By Nicole Clark, LMSW|2021-08-19T18:57:13-04:00November 2nd, 2016|Categories: Research & Evaluation|Tags: Ask Nicole, program evaluation|0 Comments

ig-post-3

If you have a question that you’d like to share with the Raise Your Voice community , contact me.

It’s the worst thing ever. That moment when you’ve been working with a client, community members, or some other form of stakeholder, and you have to bring the bad news.

I recently got this email from a nonprofit professional (and FYI: I’ve removed identifying information):

My nonprofit has created a program that seeks to increase the importance of physical activity among young indigenous youth in a rural community where there’s a lack of access to gyms and other places that would make it easier for youth to be more active. The stakeholders were expecting that the activities included in the program would resonate with the youth. In my nonprofit, I’ve been charged to carry out an evaluation of this program. We used surveys and focus groups with the youth participants. The results of the evaluation were that the participants weren’t interested in the activities, which aligned with the lack of participation. In fact, the results showed that the participants have developed more creative means to get in physical activity, but they brought up the need for other quality of life services that the program wasn’t addressing. The results could potentially impact the funding that was given to this program, as the funders were expecting that the program would be a success. What’s the best way to handle this?

Dealing with funders and leadership can be tricky, and nonprofits know all too well the stress of proving that a program or service is successful to stakeholders.

So, how do you share unexpected results in a way that is diplomatic and addresses concern head on?

Make it participatory from the start

I’ve worked with clients who had the expectation that I would come in, ready to go, with all the surveys, focus group questions, and in-depth interviews scheduled. They just want someone to come in and do the work for them. When I noticed this happening, I began to push back against working with clients in this way, and in encouraging current clients and potential clients in developing a participatory way of working together. From determining data collection tools to developing questions to ask participants (and even getting everyone together to interpret the data), when you make feedback gathering participatory from the start, it creates buy-in, puts everyone on the same page, and makes everything more transparent. When people are more involved, it makes this process more fun (at least for me), and everyone learns in the process.

And here’s a secret: When you make it participatory, it improves the likelihood that recommendations from the evaluation are actually implemented.

Address expectations and potential consequences

When you ask your stakeholders what they intend the outcome of their program to be, also ask this:

“What if what we’re expecting doesn’t happen?”

Ideally, we create programs or services based on theory, research, and what’s happen in our community. It builds the foundation to do some meaningful work. Can you believe there are nonprofits actually create programs or services because it just sounds like a good idea? You’d be surprised. So, can we really feel some type of way when we get results that we weren’t expecting, and in the case of the nonprofit above, it sounds like a program was created to address a need that the community has already dealt with.

But when we follow the theory, research, and community input, yet the outcome is still not what we’re expecting?

Determine if it really is bad

(more…)

6 Jul, 2016

How Can Nonprofits Balance Positive and Negative Feedback?

By Nicole Clark, LMSW|2021-08-19T18:55:19-04:00July 6th, 2016|Categories: Public Health & Social Work|Tags: Capacity Building, program evaluation|0 Comments

When you sit down with your supervisor for your annual review or evaluation, it can go one of two ways.

When your supervisor spends more time on areas that “need improvement”, you may walk out of her office feeling defeated.

And how do you feel when you receive a glowing review? Pretty happy. Makes you feel like you’re excelling.

If you care about what you do, you welcome praise as well as recommendations for how to improve. Too much positive and you don’t feel the need to grow. If you’ve ever asked about things that you can improve of, and you don’t receive much of a response because everything is going well, how do you feel?

I’ve had my fair share of working with executive directors and program directors who only wanted me to focus on positive outcomes. And it makes me suspicious.

Of course, you want the people invested in what you’re doing to be happy. These people—the stakeholders—can range from anyone that is impacted directly on indirectly by the programs, services, or initiatives you’ve created for them.

Stakeholders want to see what’s going well. What’s going well can mean more media, more opportunities and more funding. “Negative” findings (and I use quotation marks because negative is subjective) can also lead to more media, and a lot of nonprofits fear this. Negative findings can give the impression that things are worse off than they really are.

But too much of the positive can give the impression that nothing needs to change. Let’s face it: Some nonprofits are out here designing surveys, in-depth interviews, and focus group questions that are so biased that one can’t expect anything but positive results. And that’s not valuable either.

How can positive findings give your staff the credit they deserve, and how can you address “negative” findings in a way that allows for your stakeholders to see opportunities?

(more…)

6 Apr, 2016

When You’re Clear on What You Need, It’s Easier to Measure Your Impact

By Nicole Clark, LMSW|2021-08-19T18:52:39-04:00April 6th, 2016|Categories: Public Health & Social Work|Tags: Capacity Building, program evaluation|2 Comments

I talked with two potential clients this week, and both ended up being great conversations in how they plan to dive deeper into what makes their programming valuable to their audiences. There were lots of aha moments—on their end as well as mine—in how they conceptualize a potential evaluation project or training for their staff, the various evaluation theories thy can draw inspiration from, and how prepared their staff is to embark on a small or large-scale evaluation project.

A few of those aha moments centered on my process for conducting an evaluation, and in how I assist clients in incorporating evaluative thinking in their work. Oftentimes, discussions on evaluation don’t come into the very end of a project, so I encourage clients at the onset of a program to think more about what value their programming is expected to have on their audience.

While I typically have potential clients complete my client questionnaire prior to speaking with me, most of the time I’ll meet a potential client in person via a networking opportunity before setting up a time to discuss further.

During these recent calls, I found that we spent most of the time discussing how I go about conducting an evaluation or setting up a staff training on aspects of evaluation and how they can compliment their project. In those conversations, I touched on three key factors an organization needs to consider, thus impacting how to measure the value of their program:

Clarity

A potential client questionnaire allows for a client to conceptualize a potential evaluation project, and an in-person meeting or a phone call allows for deeper understanding and relationship building. Regardless of which precedes the other, clarity on what you want to do is important. One of the benefits of being an independent evaluator is that I’m able to provide objective feedback on a client’s project and outline the factors that may impact the process of the evaluation project. Another role for developing clarity is in deciding if you really need an external evaluator to take a lead on this project or if there’s another way to add more value to this process. Which leads into my second key factor. (more…)

23 Mar, 2016

“But Does It Make A Difference?”

By Nicole Clark, LMSW|2021-08-19T18:50:57-04:00March 23rd, 2016|Categories: Research & Evaluation|Tags: program evaluation, reproductive justice|0 Comments

I was scrolling through my Twitter timeline a few nights ago, and came across a tweet from the American Evaluation Association’s Twitter account, highlighting a blog post from program evaluator and research designer Dr. Molly Engle of Evaluation is an Everyday Activity. Dr. Engle focused on how she starts and ends her day with gratitude, and how that gratitude extends to her work in program evaluation. What stood out the most was this quote:

Doing evaluation just for the sake of evaluating, because it would be nice to know, is not the answer. Yes, it may be nice to know; [but] does it make a difference? Does the program (policy, performance, product, project, etc.) make a difference in the lives of the participants[?]

As I’ve mentioned before, conducting an evaluation can lead to insights into how well a program is performing and what can be improved. How valuable is this program in the lives of the individuals, families, and communities you work with?

I’ve been thinking of this a lot, and how it connects to the Reproductive Justice movement and its application of the framework. I try to incorporate a gender-focused, intersectional analysis in everything I do. However, I can’t figure out the onset, but I started to burn out from the RJ movement.

I don’t see myself leaving the RJ movement anytime soon, so I began searching for another entry point into the RJ movement of the traditional ways I’ve approached the work in the past. Program design and evaluation has been a way to reinvigorate my approach to RJ.

While it doesn’t sound as “sexy” or “trendy” as RJ has becomes more mainstream, evaluation incorporates my engagement skills as a social worker, and I’ve found a way in my business to assist organizations in thinking more critically on how they design programs and services, as they relate to social justice work. While it may not be as exciting as a rally, I use my evaluation skills to gauge how an organization thinks of their program, what assistance may be needed to realize their vision, what their perceived “wins” (expected outcomes) are, and what those actual outcomes are.

Going back to Dr. Engle’s quote, it got me to thinking: When an organization develops a program based on the RJ framework, what are the major similarities of RJ-based programs who receive funding from major donors or foundations? Do organizations evaluate RJ programs with the same criteria as other programs based on a completely different framework? There are plenty of theories out their related to program design and evaluation, with lots of evaluation tools to choose from. Are there are separate set of evaluation tools that we can use to evaluate RJ-based programs by, and are we evaluating these programs based on what funders deem as important, or rather what makes sense to the organization applying the RJ framework? If the evaluation tools don’t exist, what could they potentially look like?

(more…)

Try This: The World Cafe

First, some background

Here’s what you’ll need:

Ask Nicole: What’s the Best Way to Deliver Bad News?

Make it participatory from the start

Address expectations and potential consequences

Determine if it really is bad

How Can Nonprofits Balance Positive and Negative Feedback?

When You’re Clear on What You Need, It’s Easier to Measure Your Impact

Clarity

“But Does It Make A Difference?”

This Is A Custom Widget