Program Evaluation

But I don’t think it gets enough credit with that word. The insights from surveys with open-ended questions, interviews and book groups and case studies are extremely informative and can be hard to articulate graphically. Once you have collected your data it is time to review and analyze. Sometimes people can get confused about the difference of these, especially after so much work goes into developing an output.

Note that specific online resources are listed above in the sections in which those resources are most appropriate. A list of resources at the end leads you to more in-depth information for designing and completing your own evaluations and/or for working with professional evaluators. Help us promote nonprofits and make an even greater impact in communities. The story of the nonprofit sector, told from the nonprofit perspective for the first time. A collaborative national project calling on board members to advance their nonprofits’ missions through greater advocacy. Advocacy is essential to advance and achieve nonprofits’ missions.

To learn more about program evaluation, How to Start Measuring Your Organization’s Impact is our recorded webinar that teaches six simple steps to develop a powerful measurement system that will increase impact and impress funders. Fill in your indicators in the Framework for a Basic Outcomes-Based Evaluation Plan. Also, carry over the outcomes you identified from the example logic model to the basic evaluation plan. These are materials and resources that the program uses in its activities, or processes, to serve clients, eg, equipment, staff, volunteers, facilities, money, etc. These are often easy to identify and many of the inputs seem common to many organizations and programs.

And you might also compare your most recent results to that of previous reporting periods if it’s not your first round of data collection. So you might be comparing your current results to a baseline or an end line or a mid-line depending on your project length. Finally, consider looking at data trends such as differences between major demographics within your target population or differences between your results and those of similar programs within your agency or organization in the community. And finally, importantly, I think it’s important to make inferences and draw conclusions by looking at multiple data points when possible rather than focusing too heavily on a single data point. Sometimes different data points have conflicting results and you have to think through the impact of reporting errors or bias to decide which data points are reflecting results most accurately.

Common Myths To Get Out Of The Way Before You Start Planning

In your theory of change, you describe what change you expect and how with a series of if-then statements. If obesity prevention programs are targeted at multiple locations, including schools, communities, or within the family, then they are more likely to produce results. Another example might be if community members experience programs to increase intake of fruits and vegetables, then they’re less likely to be overweight or obese. These are two bullets in a theory of change related to obesity, but I’ll use examples throughout this webinar. And you can imagine a couple of other if-then statements that we could add on to this. And you could have a theory of change with multiple if-then statements.

Code is that some comprehensive form of program evaluation is incorporated as part of an organization’s regular activities for each of the organization’s programs. Program evaluation provides answers the critical questions that your board, staff, volunteers, funders, and supporters have about your organization and its work. Evaluations should include input from program participants and should monitor the satisfaction of participants. They should be candid and should be used by leadership to strengthen the organization’s effectiveness, and, when necessary, be used to make programmatic changes. Few evaluations of social programs have demonstrated positive results. Part of the problem may be that program teams and evaluators often conduct rigorous summative evaluations of programs that are not actually ready for that level of scrutiny.

• Impact studies, which are designed to establish whether a program is generating desired effects.
• The outcome talks about change and knowledge, skills, behavior, and stakeholder experiences benefiting from as a result of the program or an output more of a tangible value produced as a result of the program.
• Our learning objectives today, we’re going to focus on five different aspects of program evaluation.
• Dummies helps everyone be more knowledgeable and confident in applying what they know.

Each outcomes evaluation process is somewhat different, depending on the needs and nature of the nonprofit organization and its programs. Consequently, each nonprofit is the “expert” at their outcomes plan. Therefore, start simple, but start and learn as you go along in your outcomes planning and implementation. This document provides guidance toward basic planning and implementation of an outcomes-based evaluation process in nonprofit organizations.

Planning Your Outcomes Evaluation Step 5: Piloting

Imagining your organization as being the largest food distributor for the needy on the planet is not a goal; it is a vision. When setting a goal, many organizations find the “SMART” method effective. According to the University of Virginia, a SMART goal is one that is Specific, Measurable, Achievable, Results-focused, and Time-bound. Challenges foundation leaders to rethink the role of evaluation and turn it into a vital institutional tool to achieve philanthropic purpose and improve strategy. Resources target primarily grantmakers, but may be applicable to nonprofits seeking to develop evaluation methods. Scroll down for a list of actual grantmaker evaluation projects.

This one includes the core elements I showed you earlier plus a few others, which I think can be very useful to think through and include with your logic model if you have the space to do so. But I’m showing you this one because I like that the outcomes are broken up into short-term, medium-term, and long-term. When developing your logic model and even the complexity you present on a page, I’d like to remind folks to think about your audience.

Program Evaluation

Nearly 70 percent also reported using outcome measures, which focus on ultimate effects. You don’t always know what you don’t know about the needs of your clients – outcomes evaluation helps ensure that you always know the needs of your clients. Outcomes evaluation sets up structures in your organization so that you and your organization are very likely always focused on the current needs of your clients.

So exploring the role of evaluation, which we consider an organizational best practice. Evaluation is a part of a broader strategic management process. Too often, I find clients thinking of program evaluation as something sitting on its own island. In fact, program evaluation is part of a larger strategic management process which includes many organizational components. Before digging into the details of program evaluation, I want us to start with one important role of any evaluation, and that is of asking questions and getting feedback. I like to use the example of Google to introduce this idea. Google’s mission is to organize the world’s information and make it universally accessible and useful.

In this recorded webinar,Building Excellence and Impact into your Program Designs, expert Liana Downey shares four “must-do’s” when designing your program. Stay up-to-date with the latest nonprofit resources and trends by subscribing to our free e-newsletters. Learn about relief funds governments have available to invest in nonprofits and how to advocate for them.

I once worked with an organization that had a lot of programming around developing reports. The education and behavior change that comes from the report are outcomes, and measuring them may communicate how valuable the report was. So, an outcome indicates program effectiveness, while an output may indicate program efficiency. The outcome talks about change and knowledge, skills, behavior, and stakeholder experiences benefiting from as a result of the program or an output more of a tangible value produced as a result of the program. Now might be the best time to get some evaluation expertise, for example, a consultant or utilize a local nonprofit service provider to help you review your drafted outcomes and indicators.

Resources

If you don’t have a data focus culture, you might think about how to work with your organization more so that when you do the evaluation the results are valued. Second, start with the end in mind by identifying clear program outcomes. I sometimes want to invert my logic model so that the outcomes are to the left with seems like a natural starting point. Too often we think about programs you want to do, or that we think our donors like, or that just feel good but not necessarily linking to what we want to achieve.

Each brief contains a mini-case, based on one grantmaker’s experiences. However, nonprofits frequently design and implement evaluation activities after a program is up and running, making it difficult, if not impossible, for evaluators to gather the information they need to accurately measure the success of the program. Strongly consider getting evaluation expertise now to review, not only your methods of data collection mentioned above, but also how you can analyze the data that you collect and how to report results of that analyses. These are the units of service regarding your program, for example, the number of people taught, counseled, sheltered, fed, clothed, etc. These are the activities, or processes, that the program undertakes with/to the client in order to meet the clients’ needs, for example, teaching, counseling, sheltering, feeding, clothing, etc. Note that when identifying the activities in a program, the focus is still pretty much on the organization or program itself, and still is not so much on actual changes in the client.

As a nonprofit, we often ask too many questions and struggle with finding the resources to answer them. If we are strategic in the questions we ask, then evaluation does not become an overwhelming data collection and analysis process as I think many fear. For now, however, I want to focus on data collection because that is a component of this cycle that we get the most questions about. Once you have figured out which indicators you want to measure, you’ll need to create a data collection and management plan. It doesn’t have to be overly onerous and I want to emphasize that. It is often helpful to partner with organizations who house data. You can get them invested in your project by explaining the purpose, and why you would like to use their data or partner with them.

And that’s often the case that government agencies, sister organizations and community forums are interested in your outcomes too and are willing to share their data and measurement tools where appropriate. And finally, you can use formal or informal data depending on what you’re measuring. For example, sometimes at a conference you might see large pieces of paper on the board that say, “What did you learn?

Other times I’ve seen clients wanting measurement tools supported by peer review journals to give them credibility for their audience. You can be creative here, or formal, as long as you are clear about the indicators you’re measuring and the audience with whom you’re communicating your data with. I have a couple of data collection tips here, and based around the idea that don’t forget about data that you already have, we collect data all the times. You might have it stored somewhere on a drive, and we want to just reduce the burden of over collecting redundant data. For example, where our partners are already waiting around for a piece of your program or filling out another form, it might be easy to collect data at those points. Another important element of data collection I’d like to recommend is collecting both quantitative and qualitative data. While you can often get a feedback from more participants more quickly with quantitative data, the results may seem shallow and dry without qualitative elements.

So, we talked a little bit about outcomes and I specifically like to divide them into short-term, long-term, and mid-term outcomes. Our learning objectives today, we’re going to focus on five different aspects of program evaluation. One is to explore the role of evaluation at a higher level. We will talk about three different types of program evaluation.

We will define a theory of change and articulate the core elements of a logic model and when it is useful. And then finally, we will define the program evaluation cycle with specific insights on data collection, management, and analysis.

This document provides guidance toward planning and implementing an evaluation process for for-profit or nonprofit programs — there are many kinds of evaluations that can be applied to programs, for example, goals-based, process-based and outcomes-based. Instead, they are made up of data from interviews and surveys. This kind of data can effectively register change, as well.

The Evaluation Conversation: A Path To Impact For Foundation Boards And Executives

The UNM Evaluation Lab hosts an event every Spring where community organization leaders, and UNM students, faculty, and policy center researchers come together… Candid’s Online Librarian service will answer your questions within two business days. Develop a timeline to monitor the success of the program on an ongoing basis. Level and scope of information in report depends for whom the report is intended, e.g., funders, board, staff, clients, etc. To round out your knowledge of this Library topic, you may want to review some related topics, available from the link below. Each of the related topics includes free, online resources.