Implementation evaluation asks about the practical lessons that emerge from putting a new project into action. Rarely does a project go off without a hitch, and lessons learned during implementation help organizations identify if an approach may need to be modified and what critical next steps are required. In turn, these lessons can help others avoid the same pitfalls. Finally, they teach the Foundation important lessons that can help improve our grantmaking capacity.
Implementation evaluation is the appropriate focus for the evaluation of:
- Demonstration projects for training or service delivery where the intervention/training model is still undergoing development
- Projects that seek to replicate an existing model in one or more new settings, or with a different population
- Planning and seed grants
- Service expansion grants
- Technical assistance grants
- Advocacy and community organizing grants
The following questions may be used to develop effective implementation evaluation. These questions may help applicants create an outline of how they plan to gather information about their project. The list is intended to be illustrative—some questions may not be relevant to all projects and applicants may want to include other questions that are not listed below.
- What is your program model (goals, objectives, activities, resource inputs, short- and long-term outcomes, types of clients/participants targeted, timeframe, budget, etc.)?
- What aspects of your original program model were implemented as planned and what had to be changed?
- Why were revisions needed?
- What changes were made, and why did you select these new approaches and discard other options?
- What aspects of the program were felt to work particularly well and why?
- Is there evidence that any unintended outcomes occurred, either positive or negative, for either the program, its staff, or for participants? For example, did you receive unexpected publicity, attract new volunteers, connect to new partner organizations, or identify and meet unexpected client needs? Alternately, did the project cause stress among or between staff, divert staff from other responsibilities they have to clients, or cost more than expected?
- How do you explain unintended outcomes?
- Did you confront any barriers that were not anticipated?
- Will you do things differently now, based on lessons learned to date?
- What next steps will you take and/or do you recommend to further revise the model and why?
- Are there conditions under which you would recommend that this program or service not be used, and why?