8 Mar, 2017

Try This: Ask Better Questions

By |2021-08-19T19:28:59-04:00March 8th, 2017|Categories: Research & Evaluation|Tags: , , |0 Comments

Starting today, we’re going to ask better questions. Questions that allow you to dig deeper to unearth richer experiences. This is crucial in gaining a better understanding of why someone keeps (or stops) coming back to your programs, products or services.

When I say “dig deeper”, what I’m getting at is being strategic in how we ask questions. There’s a difference between asking questions that allow you to truly hear what someone is saying, and asking questions because you’re searching for certain types of responses.

Digging deeper, goes beyond “I love it!” or “I wouldn’t change a thing”.  People are coming back to you for a reason, and these reasons can help you enhance what you’re offering, and can also inspire you to come up with creative and engaging solutions to address other needs that you’re currently not addressing.

Tips and examples 

Good questions are:

  • Unbiased
  • Empowering
  • Provide a safe space for the person to feel comfortable responding to
  • Stretch the person who is responding 

I’ve highlighted the last point for a reason. Here’s an example:

Back in 2015, I facilitated a few focus groups for a client, a nonprofit that provides social justice oriented feminist leadership for young women of color. The focus groups were for the organization’s 6-week summer leadership program for young women of color in the New York City area. The organization wanted to know, among  other things, how effective the program had been that summer.

Okay, sounds easy. I did a few site visits during the 5th week of the program to facilitate the focus groups. I had my questions ready based on the evaluation questions the organization sought out to explore. During the first focus group, I asked “Looking back on everything you’ve learned during the past 5 weeks, can you share something that you would change?” Some of the responses I got looked similar to “I loved everything!” or “I wouldn’t change a thing” or “Everything was good”.

Initially, I chalked it up to the participants being teenagers. Then I realized they were responding this way because of HOW I asked the question.

So, I tried a different approach for the second and third focus groups:

Looking back on everything you’ve learned during this program, if you could rebuild this program from the ground up, based on your own needs and interests, what would it look like? 

(more…)

12 Jan, 2017

Try This: The World Cafe

By |2021-08-19T19:02:13-04:00January 12th, 2017|Categories: Research & Evaluation|Tags: , , |0 Comments

What’s the first image that comes to mind when you think of a focus group?

Mostly likely a small group of people–typically between 8-12 in size–gathered around a table, responding to a facilitator’s questions while being recorded. There may be some snacks involved.

Focus groups are a qualitative data activity used to gather feedback on a product, program or service from a group of people who have some type of commonality, such as age, race, gender, work experience, etc. The information they share can help the organizations or businesses gain a better understanding of why something is working, why it’s not working, and how that something impacts their lived experience.

If you’ve ever facilitated a focus group, you might know how boring they can be. And if you’ve been in a focus group, you definitely know how boring they can be.  But what I’ve noticed is this: Focus groups are not boring, technically. It’s how they are structured that makes them boring. There are more engaging ways to conduct a focus group. One of those methods is the World Cafe.

First, some background

From The World Café: Living Knowledge through Conversations that Matter, the World Cafe is a methodology that invites large group dialogue. While in a basic focus group, participants are asked a question and discuss it openly, the World Cafe takes it a step further by allowing for a larger group of participants to be in the space. While they are discussing the question amongst themselves, the conversation flows more freely because it’s a conversation amongst the group rather than the group responding directly to the facilitator. Also, the group is helping the facilitator collect data in a more dynamic, participatory way.

Based on recommended World Cafe design principles, the focus of the World Cafe:

  • Set the context– What is the purpose for bringing people into the space, and what do you (and the group) want to achieve?
  • Create hospitable space– Pay attention to how the space is set up. Is it comfortable and inviting? If accessibility is a need, does the space function so that those with certain needs are comfortable in the space?
  • Explore questions that matter– You can explore a single question or you can develop questions that build on each other. Either way, you’ll be able to synthesize the data to explore common themes
  • Encourage everyone’s contribution– Encourage everyone to participate in ways that work for them. While some find it easier to express themselves verbally, others may find it better to draw or simply listen
  • Connect diverse perspectives– The key to the World Cafe is sharing perspectives. When participants move about the room (more on this later), they’re  meeting new people, seeing ideas and thoughts that have been generated, and sharing new insights
  • Listen together for patterns and insights– Shared listening is the key factor in the success of a World Cafe. This is the way to look for themes. Encourage people to listen for what is not being spoken along with what is being shared
  • Share collective discoveries– Once the World Cafe is complete, invite participants to reflect on patterns, themes and questions expressed and to share them with the larger group

Here’s a video explanation of the process.

Here’s what you’ll need:

(more…)

12 Mar, 2015

Ask Nicole: What’s the Difference Between Research and Evaluation?

By |2021-08-19T18:43:13-04:00March 12th, 2015|Categories: Research & Evaluation|Tags: , , |0 Comments

image

Do you have any questions related to social work, evaluation, reproductive justice? Curious about how I feel about a particular topic? Contact me and I’ll answer it!

This is probably the most common question you’ll hear about evaluation practice. Because I’m asked this question often, I would like to given my take on it.

To start, there are several differences between research and evaluation. Evaluation is a systematic way of figuring out how effective your programs and services are, and if the desired outcomes of the program/service line up with what participants are experiencing. You can do this in a variety of ways, including surveys, focus groups, interviews, and more. Evaluation can inform key stakeholders (which can include legislators, program participants, funders, nonprofit staff, etc.) how sustainable your program or service is.

In comparison, research is designed to seek new knowledge about a behavior or phenomenon and focuses on the methods of getting to that new knowledge (hypothesis, independent/dependent variables, etc.). In other words, research wants to know if a particular variable caused a particular effect (causation). Once testing is done, researchers can make research recommendations and publish their findings. However, one of the key differences between research and evaluation is that conducting an evaluation can lead to insights in what’s going well and what can be improved. In other words, evaluation shows how valuable your program or service is.

(more…)

3 Dec, 2014

Who Are The People Behind The Numbers?

By |2021-08-19T18:41:54-04:00December 3rd, 2014|Categories: Research & Evaluation|Tags: , |0 Comments

image

(Photo credit: Kaiser Family Foundation)

“Statistics are real people with the tears wiped away. When statistical data are presented, they seem sanitized and tend to distance the reader from the actual problem at hand.”  ~ Dr. B. Lee Green 

Let’s take a look at this graph, taken from the policy fact sheet “Sexual Health of Adolescents and Young Adults in the United States”, developed by the Kaiser Family Foundation.

This fact sheet provides key data on sexual activity, contraceptive use, pregnancy, prevalence of sexually transmitted infections (STIs), and access to reproductive health services among teenagers and young adults in the United States.

The chart above is taken from this fact sheet, and the data and information is listed in the 2013 Kaiser Women’s Health Survey. To list some statistics:

**70% of women 19 to 24 rated confidentiality about use of health care such as family planning or mental health services as “important”; however, the majority of girls and women were not aware that insurers may send an explanation of benefits (EOB) that documents use of medical services that have been used to the principal policy holder, who may be a parent.

**Today, 21 states and DC have policies that explicitly allow minors to consent to contraceptive services, 25 allow consent in certain circumstances, and 4 have no explicit policy;

**38 states require some level of parental involvement in a minor’s decision to have an abortion, up from 18 states in 1991. 21 states require that teens obtain parental consent for the procedure, 12 require parental notification, and 5 require both.

Of course, the correlation makes sense: the older a woman is, the higher likelihood she is aware of what a EOB is and how health insurance companies many send them by mail to her home. In fact:

One of the earliest [Affordable care Act] provisions that took effect in September 2010 was the extensions of dependent coverage to young people up to age 26, who had the highest uninsured rate of any age group at the time the law was passed. In 2013, over four in ten (45%) women ages 18 to 25 reported that they were covered on a parent’s plan as a dependent. because that are adult children, the extension of coverage has raised concerns about their ability to maintain privacy regarding the use of sensitive health services such as reproductive and sexual health care and mental health. (Kaiser Family Foundation, 2013)

I also find it interesting that the younger a woman is, the higher she is to rate confidentiality when seeking various health care services. Also the fact that only 21 states and DC allow minors complete consent to access contraceptives and that most states require some level of parental involvement in a young person’s decision to have an abortion is worth looking into, especially in states that allow young people to access contraception without parental consent.

But we’re not here to talk about completely about the statistics. And we’re not here to provide a full-on critique of policy fact sheet.

(more…)

1 Oct, 2014

10 Common Mistakes that Keep Respondents from Completing Your Survey

By |2021-08-19T18:39:34-04:00October 1st, 2014|Categories: Research & Evaluation|Tags: , |0 Comments

image

Developing survey questions is harder than it looks. Asking questions is easy, but asking direct, unbiased, and valid questions is more of an art form. There’s a lot that goes into it, including flow, how the questions tie into what your program evaluation wants to answer, and keeping your respondents engage enough to complete the survey.

Here are 10 common mistakes and my tips for avoiding them:

Not knowing your target audience: Understanding who your audience is can help you craft survey questions that are pertinent to them. Avoid using words or phrases that your respondents may not know the meaning of. Instead, use words and phrases that are tailored to your target audience. Are your surveying nurses, social workers, or teachers? It’s ok to use words or phrases that are most common to those target audiences. On the other hand, if you’re not sure if your audience will understand what you mean by “reproductive justice”, it’s best to gather insights from the program coordinator or workshop facilitator to see if this term has been discussed.

Not explaining WHY: Believe it or not, most respondents are willing to help you if you share the value in completing your survey. When a respondent knows what’s in it for them, there is likelihood that the survey gets completed. If respondents know that their responses can aid in determining pay raises or in the restructuring of an under-performing program’s activities you’re more likely to complete it. If an incentive (i.e. a gift card to the respondent’s favorite retail store, coffee shop, or to wherever Visa is accepted) is included when a respondent completes your survey, indicate that on your survey at the very beginning before respondents begin.

Including extensive demographic questions: When you ask too many demographic questions, it can result in taking up a lot of room that could have been used for other questions. Before you add in questions to gather information on a respondent’s income level, religion, socio-economical status, etc., consider if it’s appropriate and relevant to the overall survey and the basis of the evaluation. Also, unless the rest of your survey depends on these answers, consider leaving demographic questions for the end of the survey as they tend to be the more uninteresting parts of a survey for respondents to complete.

Asking too many questions: Tying into the second point, asking too many questions can be the downfall of your survey. There are a variety of question types—open-ended, multiple choice, Likert or interval (very satisfied, satisfied, neutral, dissatisfied, very dissatisfied), ratio (“How many days do you spend studying?”), and dichotomous (true/false, yes/no, agree/disagree)—but it’s more about the intent behind the question. My recommendation is to create a survey that can have up to 15 questions. Keep in mind that engagement levels wane, especially during an online survey where there are more distractions (i.e., social media, videos, online shopping, etc.) (more…)

This Is A Custom Widget

This Sliding Bar can be switched on or off in theme options, and can take any widget you throw at it or even fill it with your custom HTML Code. Its perfect for grabbing the attention of your viewers. Choose between 1, 2, 3 or 4 columns, set the background color, widget divider color, activate transparency, a top border or fully disable it on desktop and mobile.
Go to Top