This website is part of

VetoViolence Logo

Evaluaction Logo

Importance of Evaluation

1. Add Value TO YOUR WORK

Depending on your view of evaluation, you may see it as invaluable or as not having any benefit at all. You may see it as a way to highlight your program's flaws and nothing more. However, evaluation may also uncover areas in which your program truly shines. Or you may find some areas for improvement. When you know what's not working well, you know what to fix. Whether your program is on the right track, needs some help, or needs a reboot, improving your evaluation efforts can help. It's about doing the work as well as you can, whether you're going from good to better-or from better to the best.

Learn from the experts

Howard Spivak

Director of the Division of Violence Prevention
National Center for Injury Prevention and Control
Centers for Disease Control & Prevention (transcript)

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

Phyllis Millspaugh

Program Administrator
Kentucky Cabinet for Health and Family Services
Clinical Consultation Branch (transcript)

Patricia Cook-Craig

Associate Professor of Social Work
University of Kentucky (transcript)

2. Confidence IN YOUR INVESTMENT

If you were going to invest in a product or program, you would want to be confident in your investment, right? Like those infomercials you see on TV-will you really lose 30 pounds in two days? Probably not. What about your program? You want to be able to justify using your resources well: Are your program's activities being produced with appropriate use of resources such as budget and staff time? Does the value or benefit of achieving your program's goals and objectives exceed the cost of producing them?1 Demonstrating you are on the right path and being financially responsible-with good theory supporting your efforts-will help increase your credibility. Evaluation provides checks and balances to help you and your program stay on track, with the evidence to back it up.

Learn from the experts

Valerie Spiva Collins

Training and Technical Assistance Supervisor
FRIENDS National Resource Center (transcript)

Deborah Gorman Smith

Director, Chicago Center for Youth Violence Prevention
Professor, University of Chicago (transcript)

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

3. ACCOUNTABILITY

A well-thought-out evaluation plan and engaging in evaluation may help you feel confident in your investment, and will help you remain accountable. Were your program's activities put in place as originally intended?2 It's crucial to long-term success that you know if your program is working as intended and where any weak links may be so you can address them. You not only have an obligation to ensure your organization can stand behind the work you're doing, but it's also critical for building and maintaining successful relationships. It's important that you're able to showcase your program to the people and communities you serve and who have a vested interest in your success. And if your evaluation uncovers a success story, an example others can learn from and make their own, even better.

Learn from the experts

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

Dan Whitaker

Professor of Public Health
Georgia State University (transcript)

Valerie Spiva Collins

Training and Technical Assistance Supervisor
FRIENDS National Resource Center (transcript)

Jim Hmurovich, BA, MS Ed

President & CEO, Prevent Child Abuse America (transcript)

1Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

Myths

Do you think evaluation is too hard, too expensive, too time-consuming for you or your organization to handle? Not so fast.

Myth 1: It's too complex.

Evaluations can be as different as the programs evaluated. You decide how complex your evaluation will be. It is often a good idea to start small. You have probably already collected valuable information already that could be used in your evaluation.

Myth 2: It's too expensive.

Ask yourself-can you afford not to do evaluation? Opting out of evaluation doesn't necessarily mean you're saving money. Program evaluation can show you how effectively you are using your resources, and it can be a way to 'sell' your activities or components to stakeholders. Don't forget that you can start small and scale up as you're able. You already track some information for funders and other stakeholders. To start, think about what you can do with that.

Myth 3: It's too time-consuming.

Time is another scarce resource. So wouldn't you like to know you are using it wisely? Yes-it will take some effort to create a plan and determine how much time you have to spend on evaluation. But on the other hand, as a result of doing an evaluation, you may also find out how to best use that limited time.

Myth 4: I don't have enough staff.

Program evaluation often needs to include many types of stakeholders. The good news is everyone won't need to commit the same amount of time and energy to the process. Often a small team-one that is consistently and intensively involved and engaged in the evaluation process-can be an effective way to keep the process manageable.

Myth 5: I can just tack it on at the end.

It's a common misconception—and, in some cases, an accepted practice—that evaluation is something you do at the end of programming. Although ending violence is a long-term goal of much of your programming, reaching that goal means evaluating your efforts and using that information to continuously improve performance. Even if you are still in the program planning process, you can evaluate who is involved and why.

Myth 6: My resources are better used providing activities.

No one will argue that providing activities is a bad way to use your resources. However, a program evaluation shows how well you are using those resources. Are the activities and resources working? Wonderful-now you know for sure. Do they need to be reworked? Have you been duplicating efforts without realizing it? This is also great information to have, so you use your financial resources most efficiently and can identify where to make improvements.

Myth 7: It will make me look bad!

You may find that some of what you are doing could be done better. That's not the end of the world. Learning organizations—those that learn from past experience to continually improve programming—understand that success and failure provide constructive feedback to build upon. Being flexible in this way will show your program is one that changes with the times and is committed to continuous improvement.

e.val.u.a.tion

[ih-val-yoo-ey-shuhn]
noun

Program Evaluation is a systematic examination of the worth, merit, or significance of an object or set of organized activities intended to achieve a specific result.3

By Systematic Examination, We Really Mean: Take an orderly and methodical look at what you're doing. Do you have a plan in place to examine how your program is functioning, whether you are reaching your intended audience, wether you are achieving your intend outcomes? Is your program achieving what you set out to do?

By Significance, We Really Mean: Determining if what you're doing is making a difference in the way you intended. Evaluation provides the opportunity to assess your program's impact, determine if you need to make improvements, and/or build program capacity.

By Organized Activities, We Really Mean: Knowing what you're evaluating - what is the "it" that is being evaluated? A set curriculum, strategy, policy? You want to be sure what you intended to measure and what you actually measure are one and the same. What you are measuring should be developed enough to be evaluated, meaning so of the "kinks" have been worked out.

3Framework for Program Evaluation in Public Health (MMWR 9/17/99)

People of Evaluation5

Stakeholders, as the name implies, are people invested in what happens with the program. This label covers a range of individuals and organizations. Program staff members have a professional stake in the organization's success, but many have a connection and passion for the work that make the investment much more personal. Other stakeholders may have a financial investment in the program if they provided funds to support the program, but the individuals and communities that experience the program are also stakeholders because programmatic changes could improve or diminish the components they receive.

Survivors

They provide a deeper and broader understanding of the issue. Violence ceases to be abstract when working with them: They will ensure your work is real and relevant.

Community Members

They increase the credibility of your efforts; a more connected community can easily engage families and inspire community participation.

Neighborhood Associations/Networks

They increase credibility of your efforts; a more connected community can easily engage families and inspire community participation.

State and Local Health Departments

They often implement the plans that are central to your effort. They have access to data, specific skill sets, and content knowledge that can supplement your evaluation efforts.

Schools/Educational Groups

They often implement the plans that are central to your effort; this can result in better attendance, grades, and graduation rates. This can improve your relationship with the schools.

Legislators/Policymakers

They can fund/authorize continuation or expansion of your efforts—which, in turn, can help lessen the financial and health burden on the state.

Funders

They can fund your efforts. Contributions of private or public funders can help make communities better—which can help lessen the burden on the state and taxpayers.

Program Evaluators

They may be part of your staff, public health partners, or someone you hire externally. They offer a specific skill set, as they know how to systematically gather credible information to tell the story of your program and work. If you opt to bring in an external evaluator, you may benefit from the different, unbiased perspective they will bring to your efforts.

Framework for Program Evaluation in Public Health

Standards Help you determine if you've designed your evaluation well, in order to reach its full potential.6

  1. Utility: Ensures you provide useful information
  2. Feasibility: Ensures the evaluation is doable & practical
  3. Propriety: Ensures the evaluation is appropriate
  4. Accuracy: Ensures you provide accurate findings

Engage Stakeholders

Get people on board who can help you! This doesn't necessarily mean everyone has to agree with you; your stakeholders could also have a different lens or offer another perspective. Including people with a range of viewpoints could yield great benefits in the work you do.

Describe the Program

Clearly define/explain the parts and pieces of what you're doing.

Focus the Evaluation Design

Figure out what you want to know about the program so your time and money are well spent.

Gather Credible Evidence

Collect believable, relevant information that creates a well-rounded picture of the program.

Justify Conclusions

Any judgments or decisions are warranted, based on information collected.

Ensure Use & Share Lessons Learned

Ensure Use -Share the information in a way that people will actually use.

Share Lessons Learned - Help them help you – you want your stakeholders to use the information you've so carefully collected. Collect feedback, follow up with them, share successes and lessons learned.

6http://www.cdc.gov/mmwr/pdf/rr/rr4811.pdf

What Can I Afford?7,8

Evaluation plans are not "one size fits all." How you approach evaluation will depend greatly on your organization's specific needs and resources. What are your financial limits? What about time? Do you have staff you can dedicate to this effort? Even if you answer those questions with "Zero dollars, no time, no staff," that doesn't mean evaluation can't work for you. Consider what you're already doing and use that as a starting point. Keep that in the back of your mind as you use the chart below to quickly identify what's feasible based on your situation:

Evaluation Plan Costs
STEP ACTION 0 $ $$ $$$
Take stock of current activities Could be as simple as making a list of what you're currently doing checked checked checked checked
Develop a logic model There are several free options available online checked checked checked checked
Monitor your program You can track what program funding is spent on and describe program activities. This may help to justify continued funding and build capacity. checked checked
Self-assessment Programs at this stage of development often use self-assessment or an external evaluator to observe and document program activities. You might obtain numerical counts of activities, participants, activities, or products produced. You may also assess how satisfied participants or recipients were. checked checked
More moderate evaluation efforts You might assess changes in participants' knowledge, attitudes, or behaviors and collect information about implementation of your program.

Programs at this stage of development often use self-assessment or an external evaluator to observe and document program activities.
checked checked
Moderate to higher cost evaluation efforts With additional resources, you may add a comparison group to help you assess changes in knowledge, attitudes, or behaviors among the participants in your program. Additional costs come in when you want to measure longer term changes because you need to collect additional data. Programs at this stage of development often use external evaluators to observe and document program activities. checked checked

 

7Afterschool Evaluation 101, Harvard Family Research Project, v 1.0, 2011

8Program Manager's Guide to Evaluation, ACF, 2nd ed. 2010

Do's and Don'ts9

DO

DO use evaluation to tell the story of your program's successes

DO promote the idea that evaluation is about learning from the experiences of a program

DO identify and involve as many stakeholders as possible

DO incorporate evaluation throughout program development

DO have standards: Your evaluation should be useful, feasible, ethical and accurate


DON'T

DON'T monitor and evaluate for the sake of it; your efforts should have real purpose behind them

DON'T go down the evaluation road without first having a plan in place

DON'T forget to manage the buy-in process with key stakeholders

DON'T worry if you don't get it right the first time

9Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide pp. 13-14

Recruit your Evaluaction team10

Good evaluation requires a combination of skills, best pulled together as an evaluation team instead of giving just one person the responsibility. It's likely you're probably already doing some of this, even if you're not calling it evaluation. Members of the team don't necessarily need to view themselves as "evaluators." They may have the ability to do evaluation in some form or fashion without it being their sole job function. They may already be thinking evaluatively about the program and their role—whether by asking important questions, determining what information may be needed to answer those questions, or collecting the appropriate information in a thoughtful way. This mindset and your team's ability to think strategically can have a positive impact on your overarching efforts.

Any of your colleagues who are involved with the program and critical to its success can be an asset to the team. Maybe they work in your office, are partners at a local university, or are external evaluators you hire to assist your efforts. Regardless of their job or title, building a team with people who have the following skills will help you put your plan into action.

Adaptable

You're skilled at working with a variety of stakeholders

Supporter

You understand the potential pros and cons of evaluation

Analyzer

You're comfortable with data sources and analysis

Equal Opportunist

You understand the importance of sharing all evaluation findings, positive or otherwise

Educator

You're able to educate program personnel in designing and conducting the evaluation as well as using the results

Experienced

You have experience with the type of evaluation needed

Innovator

You're able to develop innovative approaches to evaluation while considering your program's realities

Incorporator

You know how to integrate evaluation into all program activities

Achiever

You're skilled at using what you learn from the evaluation to make program improvements

10Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

Build an Evaluation Plan11,12,13

By now, you may realize that you are, in fact, already doing some evaluation of your program. Perhaps you were "thinking evaluatively" all along but didn't realize it. Or maybe it doesn't seem as daunting as it once did and you're ready to dive in. Regardless of where your current activities stand, it's not too late to begin evaluating what you're doing. Evaluation is a process, one dependent on what you're currently doing and on the direction in which you'd like go. Use our Evaluation Plan Builder to help you get started—or to better organize your efforts if they are already under way.

Keep in mind as you work your way through this tool that you may not be able to answer all questions immediately. Or perhaps you find yourself needing to pause for a moment, so you can discuss parts of the plan with colleagues who may be able to provide the answers you need. No need to worry: MyVeto enables you to save your work so you can make changes later.

Our Evaluation Plan Builder will provide you with several printable documents: a logic model and your draft evaluation plan. So perhaps you want to start with the logic model and handle the rest later? No problem! You can build your plan step-by-step, moving forward when you're ready.

But before you build your evaluation plan, you'll need to recruit your EvaluACTION team. Why don't we do that now?

What, How, & Why it Matters

So what exactly is an evaluation plan? The Centers for Disease Control and Prevention (CDC) defines one as "a written document that describes how you will monitor and evaluate your program, as well as how you intend to use evaluation results for program improvement and decision making." You can look at it as a road map of sorts. It will help you organize your efforts, ensure you're using your financial and staff resources wisely, help you plan the next steps, and assist you in staying accountable along the way. Although the plan is outlined as a series of steps, you may find that you need to go back and forth as you develop your evaluation plan. That's okay! The CDC Evaluation Framework, which we use here, was designed to be cyclical.

The What

An evaluation plan will help you lay out what your evaluation will and will not entail so you can make the best use of your evaluation resources.

The How

It's important that you prioritize your efforts and streamline your work. It's also essential to know which specific evaluation questions to ask, so you gather information that will help determine if what you're doing is truly making a difference. An evaluation plan will help you get the necessary buy-in from stakeholders and focus your evaluation efforts so you can answer the "so what?" question about your program.

Why it Matters

It's important you're able to prioritize your efforts and streamline your work. It's also essential to know which specific evaluation questions to ask, so you gather information that will help determine if what you're doing is truly making a difference. An evaluation plan will help you get the necessary buy-in from stakeholders and focus your evaluation efforts so you can answer the "so what?" question about your program.

Ready...set...let's go!

Click here to get started

11Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide http://www.cdc.gov/eval/guide/index.htm

12http://stacks.cdc.gov/view/cdc/24531

13http://stacks.cdc.gov/view/cdc/24531

Next Steps

Now that you have a better sense of what you’re currently doing and what you would still like to know, check out these resources. They will help you take your efforts to the next level!

CDC's Program Evaluation Resources

This site provides additional evaluation resources, including step-by-step manuals, assistance with specific evaluation steps and logic models, and links to key professional associations and key journals.

CDC's Evaluation for Improvement Manual

This manual provides step-by-step information on how to hire an empowerment evaluator with the goal of building an organization's capacity to conduct evaluations.

CDC's Developing an Effective Evaluation Plan Workbook

This workbook provides detailed information and worksheets for planning evaluations.

HHS's How to Hire an Evaluator

This publication offers some guiding principles for hiring an evaluator who is appropriate to your project's needs.

American Evaluation Association (AEA)

This site provides information on evaluation, as well as a searchable database for finding an evaluator.

Policies & Regulations | Accessibility Statement