21 Oct, 2020

Try This: A Simple Way to Get Program Feedback

By Nicole Clark, LMSW|2021-08-19T21:00:38-04:00October 21st, 2020|Categories: Research & Evaluation|Tags: program design, program evaluation, Try This|0 Comments

*Try this out and let me know how it goes.*

I enjoy coming up with things on the fly, and today’s Try This is one of those moments.

I was hired by a client organization earlier on in my consulting business to conduct several focus groups for their program participants, all of whom were young high school girls attending a 6-week summer program.

During the first two focus groups, one question I asked was, “What would you change about this program?”

Interestingly, the participants enjoyed the program so much that they didn’t have any ideas on how they could improve the program.

When I got to the last two focus groups, I decided to change the question.

Before I share how I changed the question, let’s make one thing clear: It’s very possible your program participants really do enjoy your program, and there’s a lot of information to glean from this. But this client organization specifically wanted feedback on how to evolve the program and they noticed that, outside of exclaiming they enjoy the program, participants didn’t give in prior feedback on what they enjoyed and what they’d like to see based on previous evaluations.

In response to my client’s request, I shifted the question.

This activity is ideal for:

Anyone responsible for developing, running and evaluating programs and services
Anyone interested in applying evaluative thinking

Here’s what you need:

Whatever setup you use for focus groups (though this can also be created as a survey, key informant interview, or as a World Cafe)

The steps:

(more…)

16 Oct, 2019

Try This: The Cookie Exercise

By Nicole Clark, LMSW|2021-08-19T20:44:55-04:00October 16th, 2019|Categories: Research & Evaluation|Tags: program evaluation, Try This|0 Comments

*You don’t have to eat the cookie, but it might be tempting.*

Evaluative thinking involves identifying assumptions, posing thoughtful questions, pursuing deeper understanding through reflection and perspective-taking and making informed decisions in program planning, implementation, and evaluation.

It’s a key component to building evaluation capacity within an organization, company, or other entity. While I still conduct evaluations for clients, I’m finding myself more drawn to helping organizations build up this capacity so they can do it all themselves.

The primary reasons organizations conduct evaluations is to find out if 1) the program is meeting the needs of its intended audience(s), 2) if it’s financially feasible to maintain the program or service as is or if it needs to be scaled up or down, or 3) if it should be scrapped altogether.

It’s not enough for your staff to know the ins and outs of doing an evaluation. They need to harness the ability to think evaluatively about the programs they’re developing.

And what better way to try this out than with cookies?

Grab 3 different brands of cookies and let’s get started.

This activity is ideal for:

Anyone responsible for developing, running and evaluating programs and services
Students interested in evaluative thinking

Here’s what you need:

If you’re the baking type: Bake three different types of the same cookie (using different chips or filling, different ingredients, etc.)
If you’re the store-bought type: Buy 3 different brands of the same cookie type. In order for this to work, the brands cannot have any recognizable marks in the design that will give clues on who makes them. In other words, you can have 3 brands of cookies that look like Oreos, as long as each brand doesn’t have “Oreo” labeled on them (we’re trying to avoid biases here)
Sheets of paper
Writing utensils (pens, pencils, markers)

As a heads-up: You don’t have to eat the cookies. But you might, for testing purposes.

(more…)

20 Feb, 2019

Try This: Increase Your Staff and Stakeholders’ Engagement and Collaboration

By Nicole Clark, LMSW|2021-08-19T20:25:18-04:00February 20th, 2019|Categories: Research & Evaluation|Tags: Try This|0 Comments

When a program is developed, it’s done so for various reasons, including based on current and relevant research, anecdotal and collected feedback and evidence from participants, funders, and more.

But even the most well-developed programs can suffer if your staff and stakeholders aren’t engaged enough to care about how the program is designed to work beyond focusing on outcomes and impact only.

As I’ve continued to shift more into engaging my clients in a more participatory way, we’ve been working together to develop a more collaborative process that generates a culture of learning about how they’re developing their programs and how their program recipients are responding to it, rather than merely meeting metrics commissioned by someone else.

Compared to previous Try This posts, what I’m sharing today is more of a process rather than actual activities. And for the purposes of today’s post, I’m labeling this process the “staff and stakeholder engagement and collaborative process“. The actual name depends on the client project I’m working on (for example, with one client, it’s called the “evaluation working group” while the process is called the “evaluation working group” with another client).

Regardless of the name, the process is the same: Bringing together staff and stakeholders to develop program goals, discover insights into how participants are engaging with the program, and develop a strategy for routine program evaluation and course-correction and generating buy-in among your staff and stakeholders.

This activity is ideal for:

Nonprofit and social services staff interested in developing more engagement and collaboration between staff and stakeholders (i.e. program participants, board members, and other supporters)
Nonprofit and social services staff that are responsible for developing and overseeing the implementation of programs, services, and strategies

Here’s what you need:

(more…)

5 Oct, 2018

Ask Nicole: What Exactly Are You Evaluating?

By Nicole Clark, LMSW|2021-08-19T20:18:45-04:00October 5th, 2018|Categories: Research & Evaluation|Tags: Ask Nicole, Capacity Building, program evaluation|0 Comments

Have a question you’d like to be featured? Let me know.

I recently had a video meeting with one of my client organizations. We’re preparing for a presentation in a few weeks to orient some members of her staff to a newly-developed evaluation working group. They will be working directly with me on guiding the organization through the development of an evaluation framework for its programs and strategies, guided by the organization’s current strategic plan.

As we planned out the agenda and what topics to include, the staff member and I discussed various aspects of the evaluation design process, including logic models, theories of change, data collection and dissemination. In this discussion I touched on one aspect of the evaluation process that many would see as a given, but it’s actually more complex:

What exactly are you evaluating?

We ordinarily associate evaluation with the ending of a program, where we want to collect data to find out if what the program set out to do actually achieved its goals. But you can also evaluate the program as it’s being developed, or even evaluate if the program is appropriate enough to implement.

While there are multiple evaluation theories, there are five common types of evaluation:

(more…)

22 Aug, 2018

Tailor Your Strategy to Capture Your Stakeholders’ Attention

By Nicole Clark, LMSW|2021-08-19T20:17:35-04:00August 22nd, 2018|Categories: Research & Evaluation|Tags: program evaluation, Research|0 Comments

Whether you’re a nonprofit, community group, foundation, agency, school (or somewhere in between), you have stakeholders: people who are impacted–directly and indirectly–by the success and outcomes of your programs and strategies.

You should have a plan in place for how you engage your stakeholders with the information you want to share. Before you can create your plan, let’s identify your three stakeholder types.

Step 1: Identify your stakeholders

Each stakeholder has a particular set of needs and wants, with levels of influence and varied interests. This can differ greatly across programs and strategies. Choose a program or strategy your currently implementing, and identify all possible stakeholders for that program or strategy. Next, break them down into these stakeholder categories:

Primary– A primary stakeholder is the group that most closely touches the program or strategy. For example, one of my past evaluation projects was for a local nonprofit. They wanted to conduct an internal evaluation to discover reasons for low volunteer engagement. Volunteers–both active and inactive–would be considered primary stakeholders.

Secondary– Secondary stakeholders are indirectly affected by the outcomes of a program or strategy. They serve as intermediaries. With our example above, the staff (both organizational and the clinic staff the volunteers worked for) can be secondary stakeholders.

Tertiary– Tertiary stakeholders are usually far removed from the impact of the program or strategy’s outcomes, but they can serve in an advisory capacity. In our example, the board of directors would be concerned a tertiary stakeholder.

Where you stakeholder falls depends on the program or strategy. In other words, a primary stakeholder for one program can turn into a tertiary stakeholder for another program.

Step 2: Use the Five Ws (and the H)

Now, let’s figure out how to engage your stakeholders based. And what better way to determine how to engage your stakeholders than using the Five Ws (and the H)?

(more…)

Try This: A Simple Way to Get Program Feedback

Try This: The Cookie Exercise

Try This: Increase Your Staff and Stakeholders’ Engagement and Collaboration

Ask Nicole: What Exactly Are You Evaluating?

What exactly are you evaluating?

Tailor Your Strategy to Capture Your Stakeholders’ Attention

Step 1: Identify your stakeholders

Step 2: Use the Five Ws (and the H)

This Is A Custom Widget