With my program and evaluation clients, I offer the Evaluation Capacity Measure. It’s an assessment created by Ann Price of Community Evaluation Solutions , and I’ve revised it a bit to include some open-ended questions. The assessment assesses an organization’s current capacity to evaluate its programs, strategies and services and assesses its current support around evaluative thinking. Everyone from organizational leadership to board of director members are encouraged to take the assessment.

This assessment helps me to see what the current needs are, and it also gives clients a sense of what we can work on together in the evaluation capacity building process. I offer it to clients first as a baseline, then towards the end of our partnership together. Whether it’s a short-term or long-term project, clients like knowing their staff’s general views on evaluation, and how the process can be aligned with their respective roles and organizational mission and strategy. (At times, they may notice that what they expected to score lower on is higher than expected, and vice versa).

I recently administered the assessment for a new client, whose staff works remotely and is comprised of several departments that work individually and collectively on several programs and strategies. Leading up to sending the assessment to her colleagues, the staff member leading the project with me shared her nervousness about what the results may reveal about her staff, but is excited to see how the results can influence how they can proceed with their evaluation priorities.

The goal of program evaluation is to 1) see if your program is performing in the way it’s intended to and 2) facilitate organizational learning and improvement.  In general, people tend to experience anxiety when they’re being evaluated. It’s like when you waited in a line with your classmates to see which kickball team captain would choose you for their team. You want the best players on your team, and the captains are sizing you up. Who is known to kick the farthest? Who has the capability to catch the ball mid-air? Who can run the fastest?

And when you’re last to be picked, it makes you feel a way. “Why wasn’t I one of the earlier picks?”, you asked. More often the not, the team captains probably chose their team based on who they’re friends with, but there’s a lot of emotional reactions that may come up.

You get into the real world of adulting, and the feeling is still there. This time, it surfaces when you’re meeting with your supervisor for you mid-year or annual review. You think you’re performing well in most areas, until you discover (through someone else’s perspective) that you’re not.

In “Strategies for Managing Evaluation Anxiety: Toward a Psychology of Program Evaluation” (American Journal of Evaluation, Vol. 23 Issue 3, 2002), Stewart Donaldson, Laura E. Googler, Michael Striven dubbed the term “excessive evaluation anxiety” (or XEA). Symptoms of XEA include:

• Lack of access to important information and data
• Compliance and cooperation problems
• False reporting
• Effects on bias and validity
• Reduced utilization of evaluation findings.

This can lead to stakeholders behaving in ways that can destroy the credibility of evaluation findings (and of evaluators). When the findings of a program’s evaluation yield that it’s not performing in the way it was intended, the blame game can happen. Instead of falling victim to XEA, how can you and your staff become more open to viewing the process as a learning experience?

To give you a taste of the Evaluation Capacity Measure I give to my clients (and to help jumpstart the conversation with your staff around evaluation anxiety), here are some questions you can walk your staff through:

Here’s what you need:

• Flip chart paper, chalkboards, or walls
• Note cards
• Pens or pencils 

Designate five open areas around the room. Each area will have one question at the top of the flip chart paper, chalkboard, or wall.

Give staff enough time to answer the questions thoughtfully . When time is up, have staff place their note cards under the designated question.

Next, divide up your staff so that each question has several eyes on it. Have staff place the note cards into themes, grouping similar responses together.

Last, have staff do a “walk-around”, where they can view the themes for each question. To close out the exercise, discuss the themes with staff and create an action plan to address the themes.

The steps:

Have your staff answer the following questions alone on note cards:

In what ways do you anticipate (if any) this evaluation process impacting your organizational role?

According to Donaldson et. al (2002), when you don’t know how the evaluation process will impact what you do, it can lead to role ambiguity. This can also come up when the person evaluating your program isn’t part of your staff. How this question is answered can depend on a few factors:

• The number of roles a staff person is expected to take on in this process
• Any perceived negative consequences (from funders, staff leadership, etc.)
• Clarity around the purpose of the evaluation
• Willingness to participate to the process

In what ways do you anticipate (if any) this evaluation process impacting how your department engages with other departments within your organization?

Sometimes, organizational departments work in silos. Putting all of these departments together to work towards a common goal can be a big task. How this question is answered can depend on a few factors:

• How each department “touches” the program being evaluated (Is the program largely the responsibility of one department, or do multiple interact with it?)
• How each department generally operates (Is there a standard protocol for how each department operates, or are departments given free reign to do their own thing?)
• The extent to how well each department knows what the other departments are doing (purpose, duties, overall functioning, etc.)

In what ways can this evaluation process help generate buy-in from your organizational department? 

One of the goals of program evaluation is to facilitate organizational learning and improvement. Some staff will be more open to the process than others, especially when an outside evaluator is helping out. This question is ultimately about trust. How this can be answered depends on a few factors:

• How much do staff trust organizational leadership to foster an environment for learning?
• How well does organizational leadership reinforce open communication and ongoing feedback (Donaldson et. al, 2002)
• Perceived ability of the evaluator to revisit the goals and purpose of the evaluation process with staff (Donaldson et. al, 2002)
• Perceived expectation that the results of the process will negatively impact a staff member’s role

From your perspective, what evaluation priorities should your organization consider during this evaluation process? 

In other words, what do you hope the evaluation process will answer? The answer will depend on a few factors:

• Organizational role
• How one’s department “touches” the program being evaluated
• (Possibly) funding
• Past program evaluations
• Organizational strategic planning process

What will you need from this evaluation process in order to be effective in your role?

This question encompasses the previous questions, but ultimately it 1) helps your staff articulate “What’s in it for me?”  and 2) identifies how this process can lead to shifting towards a positive evaluation culture. The answer can depend on a few factors:

• Clarity on the purpose of the evaluation
• Reassurance that the process is worth the time and effort to do
• How the evaluation process adequately involves the perspective of each department

Key Takeaway 

Identifying your staff’s concerns and expectations for the evaluation process can determine the openness of the process and can help to reduce Excessive Evaluation Anxiety. Try this exercise and let me know how it goes (and if you’re interested in having your staff take the complete Evaluation Capacity Measure, contact me).