At nonprofit organizations impact is everything. Funders, the public, beneficiaries, and stakeholders all want to know how you are positively influencing your community and improving the world.
The question then is, how is that accomplished? What are the best strategies for measuring impact at nonprofits?
In this article, we will explain the importance and intricacies of measuring program impact and how to evaluate nonprofit programs via evaluation tools, methods, and industry best practices.
Let’s get started!
Introduction to Measuring Impact in Nonprofit Organizations
Nonprofit programming and services should all contribute to a broader impact or positive effect on the community.
Data and metrics mean very little if nonprofit organizations cannot connect them to positive results for people served by the organization.
For nonprofits, impact measurement can be complex and varied. Because nonprofits have different missions, they measure the impact of their programs on the people they serve in different ways.
For example, nonprofits can measure simple information such as
the number of participants served,
the number of people engaged, or
the number of meals provided.
They can also track the change in participant behavior or feelings.
Measuring each of these elements requires understanding which evaluation methods make the most sense for your organization.
Insights Straight To Your Inbox
Signup up to receive the next Instrumentl Insights report and access exclusive data that will give you an edge.
To effectively measure impact, a nonprofit must first make a plan for program evaluation.
In the following section, we will review this multi-step process, why it is important to implement for every nonprofit, and what types of challenges organizations face when attempting to evaluate their programs.
What Is Nonprofit Program Evaluation?
Nonprofit program evaluation is a multi-step process that involves collecting data to demonstrate the efficacy of a program and to identify potential areas for improvement.
Program evaluations are essential for nonprofit professionals to gain a better understanding of key program strengths and potential weaknesses.
Through program evaluations, nonprofit organizations can track and collect important data information such as
participant attendance,
participant improvement and changes over time,
financials and program costs, and
stories of beneficiary experiences.
Program evaluation can be either simple or complex, depending on the intricacies of the program. Many program evaluators will utilize tools like logic models to track inputs, outputs, outcomes, results, and overall impact.
The key features of a standard logic model are:
Inputs: Inputs are the resources that are to be integrated and utilized through the program to achieve its objectives. This can include but is not limited to personnel, funding, community partnerships, supplies and equipment, volunteers, and space.
Activities: Activities are a description of the primary actions taken to carry out the program’s goals such as providing services or weekly programming.
Outputs: Outputs refer to direct results of a program’s activities, such as the total number of people served or engaged.
Outcomes: Outcomes represent changes over time such as improvements in a participant’s health or an increase in their self-esteem.
Impact: This is key, and ultimately the most important element funders will be interested in learning about. Program impact makes the connection between the short term program outputs and the long term outcomes to illustrate the advancement of a broader objective or mission. For example, a program output might be “Provided housing to 500 individuals” which contributes to broader societal impact of eliminating housing insecurity.
Why Evaluate Nonprofit Programs?
Nonprofit program evaluation is essential for two key reasons.
First, nonprofit program evaluations can help staff identify inefficiencies in their initiatives. Evaluations can help determine whether programs are aligned with the overall mission of the nonprofit or if modifications need to be made to better address the broader objectives of the organization.
Secondly, evaluations help demonstrate the effectiveness of programs to key stakeholders and ensure that they are having the intended impact on the people the organization aims to serve. In fact, evaluation is often a requirement of most grants awarded to nonprofit organizations.
Program evaluation is the most effective way for nonprofits to measure and illustrate the impact of their program or services to external partners, including grant funders.
Challenges in Measuring Impact Nonprofit Organizations Face
There are a few challenges that nonprofit organizations may face when attempting to measure the impact of their programs.
Collecting data to measure impact can be a complex process. Larger nonprofits, in particular, may need dedicated teams to collect and effectively assess the data to illustrate the program’s impact.
Without enough staff capacity, the process of measuring impact can be overwhelming and difficult to manage.
Moreover, smaller nonprofits that are just starting out may not know how to begin evaluating programs in a way that would be both measurable and impactful.
It can be a challenge to:
develop enough outputs and outcomes to paint the most accurate picture of the program,
ensure those outputs and outcomes are both measurable and impactful, and
effectively track and maintain evaluation standards throughout the program.
Despite these challenges, there are many tools and resources that nonprofits can leverage to help them overcome these challenges, which we will review later on in the article.
How to Evaluate Nonprofit Programs: A Step-by-Step Guide
Evaluating nonprofit programs does not need to be a scary or difficult process. With the appropriate tools, resources, and direction, you can apply tried and true strategies to your own plan for program evaluation.
Define the Evaluation's Purpose and Scope
There should always be a purpose for a program’s evaluation. A nonprofit should not simply track any outcomes or any outputs without a clear objective in mind. Every aspect of the program’s evaluation should ladder up to the purpose of the evaluation which could be:
Identifying whether or not the program is effectively meeting its objectives
Ascertaining if the program is meeting the requirements or stipulations of a grant
Discovering if there needs to be any improvements or modifications made to the program design
Or a variety of other reasons depending on the type or purpose of the program itself
Your organization should also identify which aspects of the program are being evaluated.
Is the evaluation focusing on participant outcomes and achievements? Is it focused on the actions and activities of staff? Is it an evaluation of costs and finances? If you can narrow down the focus of the evaluation you can also identify which pieces should be analyzed.
Select Appropriate Tools and Methods
Next, the nonprofit should identify the appropriate tools and methods for evaluating the program.
There are three major methods for program evaluation and data collection, which are qualitative, quantitative, and mixed methods.
Quantitative refers to an approach to evaluation that focuses on numerical outcomes and measurable results;
Qualitative approaches refer to conceptual, non-numerical information; and
Mixedmethods are a combination of the two.
An example of a quantitative approach to program evaluation is evaluating measurable, numerical outcomes such as “Program ABC has served 100 participants” or “100 participants will improve their test scores by 80%”.
An example of a qualitative approach to program evaluation might be interviewing participants for their feedback on your program.
A mixed method approach to evaluation would include a combination of both—reporting on numerical outcomes and sharing qualitative information such as participant quotes and stories.
There is no right or wrong approach or methodology—it really comes down to the program's design, objective, and the purpose of the evaluation.
Collect Data and Analyze Results
Tools such as spreadsheets, surveys, CRM software, participant intake forms, and other data collection tools can help make program evaluation a simple process for everyone involved.
A few examples of tool you can utilize, especially for effective data collection, are:
Survey Monkey
CRMs (e.g. Salesforce)
Google Forms
Microsoft Excel
Google Sheets
Blackbaud
Once outcomes, outputs, observations, and information on the program activities have been collected, the data can be analyzed.
Typically, outcomes and outputs are analyzed against projections or forecasts from the year before.
For example, if the program anticipated that 60% of youth participants would improve their self-sufficiency and the data collected demonstrates that 65% of youth participants improved their self-sufficiency, then it can be inferred that the program was successful.
If there are areas in which the program appears to have underperformed, it does not necessarily mean that the program is ineffective. It may mean that
the program requires additional resources to carry out the work,
that there were external challenges that created barriers to effective operations, or
that participants need more time to demonstrate improvements with help from the program.
Report Findings and Apply Insights
Once all the information has been collected and assessed it is time to report out on the findings.
The best reports will not simply report the data and leave it at that. You should think critically about what that data means.
If the outcomes resulted in lessons learned about the program, communicate those lessons learned in the report and also develop proposed next steps to address any gaps or how to continue sustaining success.
Take for example, the results reported in Habitat for Humanity’s annual report from FY2019.
The report includes high-level quantitative outcomes such as the total number served, the total number of people who improved their housing conditions, and the total number of participating volunteers, among other metrics. The data accompanies insightful context and insights from the organization.
As briefly mentioned above, there are many different tools, resources, and methods for effectively measuring the impact of nonprofits and their programming.
Let’s take a look at the different methods of measuring program impact, noting their similarities, key differences, and when and how to use both for the most effective evaluation.
Quantitative Evaluation Tools
Quantitative evaluation approaches refer to evaluation that focuses on numerical data or data that can be given numeric value, counted, or measured.
An example of quantitative data in program evaluation is the number of participants served by a program.
Oftentimes, quantitative data is presented in a grant proposal or report as outcomes or as outputs.
Outcomes refer to changes in a participant or in the program over a period of time, typically the grant period (e.g. “Participants will improve their job readiness skills by 45% during the grant period”).
Outputs, on the other hand, represent straightforward numerical data such as the number of participants served by the program.
There are countless tools that nonprofits can leverage for quantitative evaluation.
As previously referenced, CRM tools such as Salesforce can help track quantitative data associated with individual participants or people engaged by the organization. Because the platform is designed to track customers, it can track detailed information about individuals and has mechanisms in place to analyze and evaluate data across programs or across organizations (such as participant demographics, outcomes, etc.).
Outcomes and outputs are important to help funders and external audiences understand the achievements and success of a program during a period of time.
Qualitative Evaluation Methods
Qualitative approaches to program evaluation refer to information that is collected via non-numerical methods, such as observations made by staff, participant interviews, survey responses, or other conceptual information.
Qualitative data can be obtained in several ways.
Case notes or other forms of cataloged observation from program staff.
Program staff communicate with participants, learning about their experiences and documenting insights.
Qualitative information is often reported by nonprofit organizations as participant or program stories.
Take, for example, the annual impact report for Girl Forward, a nonprofit that helps refugee girls in Texas and Illinois. The organization shares key information about their mentoring program and even utilizes a direct quote from a participant to highlight the impact and achievements of this program.
Sometimes programs that cannot share specific or sensitive information about participants will instead provide qualitative information or stories about program staff, illustrating how their work has a positive impact on the lives of the people they work with on a day-to-day basis.
Mixed-Methods Approach to Evaluation
Lastly, a mixed methods approach to evaluation utilizes both of the approaches mentioned above, combining qualitative and quantitative data to provide a broader, more nuanced picture of the program being assessed.
A mixed-methods approach is a good choice for evaluation when a program is able to use qualitative examples to provide context for quantitative outcomes and outputs or vice-versa.
Take, for example, an afterschool program aimed at helping students improve their learning. Maybe the students’ math scores have increased by 50%. But the program also has a story about a student getting enjoyment out of seeing increases in their overall knowledge and grade improvement.
The data is good on its own, but with the participant story it has an even greater impact—it shows that not only are students improving their scores, but they are also getting personal satisfaction out of meeting or exceeding expectations.
Wrapping Up: The Next Steps
Effectively measuring impact will help strengthen a nonprofit’s operations while also increasing transparency and accountability, making it easier to cultivate strong relationships with funders and the general public.
You know the work of your nonprofit is critical, but with the lessons learned in this article you can now communicate that impact to a wider audience.
Up next, pair this article with lessons on building relationships with funders. Likewise, watch the recording of our workshop with Grant professional Scot Scala as they explain a hyper-focus on diversity, equity, and inclusion can strengthen the duration of grant-funder relationships.
.svg)
%201.png)
