Evaluation

Evaluation is often viewed as an outside expectation by funders or other stakeholders. However, evaluation can be viewed and used as an integral and important part of your interfaith service-learning effort. Evaluation can:

  • Help you examine whether and how well your program is meeting its goals for service, growth, and learning.
  • Give systematic feedback to help design and strengthen your program.
  • Provide a systematic way to tell your story to your participants, partners, and supporters.

The scope, complexity, expense, and sophistication of evaluation can vary considerably, depending on your purposes and the audiences of the evaluation. Evaluations for program improvement and reporting to partners can often be conducted internally, using existing tools and resources. As the stakes increase due to external funding or a desire to demonstrate impact, you will likely need to engage an evaluation expert.

Set Your Evaluation Goals and Articulate Your Logic Model

Before determining HOW to conduct your evaluation, clarify your goals for your evaluation and the logic model for your interfaith service-learning program. Consider the following:

  • Consider youth-led evaluation (or youth participatory evaluation) as your overall method. Young people learn basic principles and practices of evaluation, including how to formulate evaluation questions, developing skills in observing, interviewing, developing surveys, and conducting focus groups, and analyzing and interpreting what they discover. Not only is this approach consistent with with a youth-led approach to interfaith service-learning, but it also can yield powerful and unique findings that might not be observed by professional evaluators. For a resource, see the attached guide, Youth Participatory Evaluation Project (PDF), developed by the Center for Adolescent Health Promotion and Disease Prevention at the Johns Hopkins Bloomberg School of Public Health.
  • What do you want the evaluation to accomplish? Do you want it to help you strengthen future programs? Are you being required by a funding or other partner to show impact? Consider the perspectives of youth, community partners, community residents, parents, faith communities, funders, and other stakeholders in sorting through these questions.
  • What kind of evaluation is most appropriate for the current status of your interfaith service-learning effort? If you are just getting started, you likely are not ready to show a major impact in the short term. At that point in the process, an evaluation focused on improving what you’re doing is likely the most realistic and helpful.
  • Design a logic model for your program. Essentially, a logic model shows the assumptions of how your program works. It shows the connections between the program activities and short- and long-term outcomes. The most widely used guide for creating logic models is available from the W. K. Kellogg Foundation Logic Model Development Guide (PDF).
  • Use this evaluation worksheet (PDF) to help you shape your evaluation plan.

Plan Your Data Collection

  • Develop or access the evaluation tools you need to collect data. These may include surveys, focus group protocols, observation tools, or others—depending on your purpose and focus.
  • Using instruments that have already been developed and validated is efficient and gives confidence in the credibility of the data. However, you need to be sure that they answer the questions you need to have answered. Consider this guidance from Project STAR when choosing off-the-shelf evaluation instruments.
  • Remember that survey data and interviews are not the only kinds of evaluation data. Structured observation of activities, analysis of reflections on interfaith engagement and service-learning, structured story telling, and other approaches can all play roles in the evaluation—depending on your goals.
  • Keep “drilling down” the abstract concepts for agreeing on what a person would SEE if we were doing this? That’s what you want to measure.
  • Seek appropriate parental permission for collecting data from youth. (The type of permission necessary will depend on the nature and purpose of your evaluation, expectations of participating institutions, and any specific guidelines or mandates in your state.
  • Pretest survey instruments and interview questions to be sure people understand them and that they are yielding valuable information. It is very frustrating to administer a survey to a large numbe of people only to learn that the questions were confusing or were misinterpreted.

Implement Your Data Collection

  • Plan the appropriate times to collect data when you can expect widespread participation without disrupting other activities.
  • Make it a priority to collect data from quality samples. If you are surveying youth participants or community, work hard to get a high return rate of survey completions. If fewer than half of the sample complete surveys, it raises questions about whether the results are an adequate reflection on the project or program.

Analyze, Reflect on, and Share Data

  • Depending on the type of data you collected, you will analyze it in different ways with different kinds of expertise.
  • Involve multiple stakeholders in interpreting the data. How do youth, parents, community partners, faith community leaders, and others interpret the findings?
  • Share the data with others. If your focus is on formative evaluation for continuous improvement, make sure that the who leadership team teases out the implications of the evaluation for future efforts.
  • Use the evaluation findings to help you tell the story of your program and the broader interfaith service-learning story.

Repeat the Cycle

You may complete a particular part of your evaluation, but it’s important to repeat the cycle, watching for change, providing additional input into program quality and design, and asking new questions as the program and network mature.

Some Hints for Conducting Your Evaluation

  • Formative evaluation focuses on data for improvement. That’s different from monitoring a work plan. Since implementation and improvement are ongoing, so formative evaluation is never complete.
  • Ideally, work first for consistent participation, use, and quality of each key element in your service-learning program. Then focus on measuring immediate effects and end results.
  • Set targets that reflect your expectations, then work to raise these expectations over time.

More Resources