Effectiveness & Maintenance – Organizational Implementation and Effectiveness

This indicator focuses on the planned or intentional contributions of SNAP-Ed partners and other stakeholders. It captures the dollar value and type of resources that partners and other stakeholders invested over the period assessed for implementation of discrete interventions in MT5, MT6, LT5, and LT6 in one or more domains or settings. Implementing Agencies may choose which interventions have required significant, measurable fiscal inputs by SNAP-Ed and partners and use a case study approach to document how SNAP-Ed funding and activities encouraged partners to co-invest in new, ongoing, or sustained interventions and community efforts within each domain or setting.

“Leveraged resources” is sometimes referred to as return on investment (ROI). For practical purposes in ongoing programs, the term ROI is used with the understanding that it does not have the rigor of cost-benefit or cost-effectiveness research.

This indicator is intended to demonstrate in fiscal terms that SNAP-Ed funding and staff stimulate other agencies, nonprofits, businesses, and foundations to co-fund evidence-based interventions benefitting low-income people in SNAP-Ed qualified settings. Using the RE-AIM model, co-investment may occur in association with any phase: Adoption, Implementation, Effectiveness, and Maintenance. Ultimately, it is expected that the provision of SNAP-Ed services and funding will lead other stakeholders to help out with their own resources. Eventually, this should result in the organizational changes being institutionalized within the partner organizations and requiring little or no ongoing SNAP-Ed support.

This indicator looks at/quantifies the resources for sustaining nutrition and physical activity supports or standards in selected settings. The focus is on leveraged resources that include staff, funding, and in-kind support.

LT9a.	Staff (number of full-time equivalents [# FTE])
LT9b.	Funding (total dollars spent)
LT9c.	In-kind support including the value of: Physical contributions such as dedicated space, new equipment, etc. Partnership activities and contributions Leader and/or champion activities and contributions Volunteer staffing time (hours) Communication activities designed to continue awareness and support for the program (number of activities)

SNAP-Ed Costs (pro-rated for the specific project but excluding fixed costs and SNAP-Ed infrastructure):

• Personnel, consultants providing direct assistance to the partners
• Out-of-pocket costs for materials, training, travel to this project
• Grant award to partner organization
• Other costs

In-kind resources:

• Personnel: The dollar-value equivalent of employee time for those who spend a significant amount of their on-the-job time (>10%) to help plan, conduct, report on, or evaluate the SNAP-Ed intervention, paid by a partner institution or organization. Estimated using the value of salary and benefits times the percent time spend on SNAP-Ed work.
• Volunteers: Calculated using conventional Federal calculations for volunteer time
• Physical space, equipment, or services that are redirected for dedication for SNAP-Ed purposes and provided by the institution or organization for SNAP-Ed use. Includes space only when it could otherwise be rented out.

Baseline and annual change in the amount of these resources should be monitored. An annual comparison may determine whether there is maintenance-of-effort or an increase in resources contributed by partners.

See ST6 and ST7 for more information on partnership and local champion resources and how to measure them. See LT8 for more information on media and how to measure its costs.

N/A

Reviews of administrative documents like budgets and work plans to pull information on staffing, budget, and support operations such as needs assessments, strategic plans, evaluation reports, media buys, etc. Where available, a review of the sustainability plan could determine the commitment of resources by partners, champions, and other critical stakeholders.

On-site assessments and inventories can provide information about personnel, space, physical conditions, and the equipment available to conduct ongoing programmatic activities. Alternatively, any documentation of facility size, conditions, and equipment could be reviewed.

On-site interviews with staff, partners, and champions about the nature and value of their organization’s engagement.

Staffing: The dollar-value of personnel whose job is dedicated to help plan, conduct, report on, or evaluate the SNAP-Ed intervention and not paid by SNAP-Ed. They may be paid through a partner’s budget, from grants, or from donations.

Funding: The total dollars in out-of-pocket costs paid to co-fund the SNAP-Ed intervention like rented facilities, transportation, media, consultants or contractors. Dollars may come from a partner’s budget, from grants, or from donations.

The Community Tool Box includes a checklist to assist in assessing in-kind support
https://ctb.ku.edu/en/table-of-contents/sustain/long-term-sustainability/solicit-contributions/tools

Additional evaluation tools to measure LT9 can be found in the SNAP-Ed Library.

Funding
In-kind support
Leveraged resources

Johnson K, Haysb C, Hayden C, Daley C (2004). Building capacity and sustainable prevention innovations: a sustainability planning mode. Evaluation and Program Planning, 27, 135–149.

Akerlund KM (2000). Prevention program sustainability: The state’s perspective. Journal of Community Psychology, 28, 353–362.

Foster-Fishman PG, Berkowitz SL, Lounsbury DW, Jacobson S, Allen N. (2001). Building collaborative capacity in community coalitions: a review and integrative framework. American Journal of Community Psychology, 29(2), 241–261.

Goodman RM. (2000). Bridging the gap in effective program implementation: from concept to application. Journal of Community Psychology, 28, 309–321.

Goodman RM, McLeroy KR, Steckler AB, Hoyle R. (1993). Development of level of institutionalization scales for health promotion programs. Health Education Quarterly, 20(2), 161–179.

Goodman RM, Speers MA, McLeroy K, Fawcett S, Kegler M, Parker E, Smith S, Sterling TD, Wallerstein N. (1998). Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Education and Behavior, 25(3), 256–278.

O’Loughlin J, Renaud L, Richard L, Gomez L, Paradis G. (1998).Correlates of the sustainability of community-based heart health promotion interventions. Preventive Medicine, 27, 702–712.

Rudd RE, Goldberg J, Dietz W. (1999). A five-stage model for sustaining a community campaign. Journal of Health Communication, 4, 37–48.

Schwartz R, Smith C, Speers MA, Dusenbury LJ, Bright F, Hedlund S, et al. (1993). Capacity building and resource needs of state health agencies to implement community-based cardiovascular disease programs. Journal of Public Health Policy, 480–493.

Steckler A, Goodman RM. (1989). How to institutionalize health promotion programs. American Journal of Health Promotion, 3(4), 34–44.

Zins JE, Elias MJ, Greenberg MT, Pruett MK. (2000). Promoting quality implementation in prevention programs. Journal of Educational and Psychological Consultation, 11(2), 173–174.