Page Contents
Framework Component
Indicator Description
*SNAP-Ed Priority Outcome Indicator
Background and Context
Measuring PSE adoption goes hand-in-hand with estimating reach. Reach is often based on estimation when actual counts are unavailable, considering the population that is potentially exposed to the intervention (UNC, 2013). Evaluators should also consider ways to maximize measures of reach by monitoring a policy or environmental change that can spread across sites or an entire organization. For instance, the reach of a local school wellness policy will be greater when the policy is adopted district-wide rather than one school at a time. We calculate total reach to demonstrate the broader exposure on the entire organization.
Many estimates of reach require accessing partner data, such as customers at a farmers market, clients at a food pantry, or customer estimates for shopping for food if these data are not known or publicly available. Often data-sharing agreements across public and private sectors are necessary and may require letters of agreement. Institutional Review Board (IRB) other levels of approval that may be necessary.
Outcome Measures
Takes place when SNAP-Ed sites or organizations put into effect an evidence-based PSE change, such as those included in the SNAP-Ed Strategies & Interventions: An Obesity Prevention Toolkit for States.
MT5a. Number and proportion of sites or organizations that make at least one change in writing or practice to expand access or improve appeal for healthy eating
MT5b. Total number of policy changes
MT5c. Total number of systems changes
MT5d. Total number of environmental changes
MT5e. Total number of promotional efforts for a PSE change
Potential Reach
MT5f. Reach: Total potential number of persons who encounter the improved environment or are affected by the policy change on a regular (typical) basis and are assumed to be influenced by it.
What to Measure
The documentation of change(s) adopted in the SNAP-Ed qualified site or organization. Documentation (direct observation, photographic evidence, repeated self-assessments or surveys) or interviews with key informants to confirm the uptake of the PSE change in the site or organization. Nutrition-related changes can include one or more of the following PSE changes, often including favorable procurement or meal preparation activities or others that expand access and promote healthy eating. The following list is not exhaustive; other evidence-based or practice-based changes may arise.
Policy
A. Improvements in hours of operation/time allotted for meals or food service
B. Policies for working parents
C. Rules for foods served in meetings or in classrooms
D. Standards for healthier eating across the organization
E. School wellness or child care wellness policy
Systems
F. Change in menus (variety, quality, offering lighter fares)
G. Removing sugar-sweetened beverages from children’s menus
H. Improvements in free water access, taste, quality, smell, or temperature
I. Restrictions on use of food as rewards or during celebrations
J. Change in food purchasing specification towards healthier food(s)
K. Change in vendor agreement towards healthier food(s)
L. Prioritizing farm-to-table/increase in fresh or local produce
M. Enhanced training on menu design and healthy cooking techniques
N. Use of standardized, healthy recipes
O. Collecting excess wholesome food to donate to charitable organizations
Environmental
P. Edible gardens (establish, reinvigorate or maintain food gardens)
Q. Lactation supports or dedicated lactation space
R. Healthier vending machine initiatives (e.g. access to healthier foods and beverages)
Promotion (Marketing)
S. Point-of-purchase and distribution prompts
T. Menu labeling with calorie, fat, sodium, added sugar counts
U. Vending machine labeling (e.g. calories, traffic light color coding)
V. Increased awareness of the changes by target audiences
It is important to document each change that occurs within a site. One change alone may not have enough magnitude to produce an impact. Thus, evaluators can document multiple changes that occur (e.g., signage, changes in layout and display of food and beverages). Measuring adoption may be labor-intensive; thus, it can be appropriate to choose a sample of sites for evaluation purposes (see Appendix C for details on sampling).
Measuring Reach
Estimate the total number of persons at the site who are expected to encounter the change on a regular (typical) basis and are assumed to benefit from it. The total number of persons who have the potential to benefit from the change in the site cannot exceed the total number of persons at the site. When there are multiple changes occurring in the same site, it is important to only count those persons who potentially encounter the change on a regular basis.
- For example, a local SNAP-Ed agency consults with a school cafeteria representative to adopt changes in the layout or display of food during lunch service to prompt healthier selections; report the total maximum number of students who purchase lunch from the cafeteria. According to the school records, there are 1,000 students; of these, 900 purchase lunch on a consistent basis. Potential reach = 900. Potential reach would not be 1,000 as not all students purchase lunch on a consistent basis.
- Another example: As part of a comprehensive worksite wellness program led by SNAP-Ed at a textiles plant, the worksite manager agrees to setup a hydration station offering fruit-infused water in the staff breakroom. The total number of employees at the organization, who we assume visit the breakroom on a regular basis, is 150. This would be your total potential reach.
Estimate the total number of persons at all sites affiliated with an organization who are expected to encounter the change on a regular (typical) basis and are assumed to benefit from it. The total number of persons who have the potential to benefit from the change in the organization cannot exceed the total number of persons at the organization.
- For example, a school district office (i.e., an organization) adopts a policy that all teachers of elementary and middle grades will cease using candy as a reward for performance in the classroom. There are 7 schools in the district (4 elementary and 3 middle) with a combined enrollment of 4,300 students. This would be your total potential reach since all 4,300 students would be potentially affected by this policy. At this stage of evaluation, do not measure full-scale implementation of the policy at each site; this comes later in the evaluation framework (see LT5).
- Building upon the same worksite example from above: The local SNAP-Ed program has developed a partnership with the regional corporate office of the textiles company to adopt a vending machine–labeling campaign using a traffic light color coding system. All but one of the organization’s six offices and plants has at least one vending machine. We exclude the site without a vending machine; among the five sites with vending, there are a total of 2,026 employees. This would be your total potential reach.
Reporting Reach Characteristics
When aggregating reach of PSE changes across sites and organizations, you are encouraged, but not required, to describe the characteristics of persons potentially reached. Because PSE activities are to be adopted only in SNAP-Ed qualified sites, we assume the majority of persons exposed to the change are low-income. However, reporting reach information by race/ethnicity, gender, languages spoken, household income levels, eligibility for free and reduced priced school meals, ZIP codes, or other factors will be important to address stakeholders’ concerns and questions. Reporting reach data by different socioeconomic, race, and Latino/Hispanic origin may explain to what extent PSE changes have the potential to benefit disparate populations
Population
Surveys and Data Collection Tools
Additional evaluation tools to measure MT5 can be found in the SNAP-Ed Library.
Key Glossary Terms
Additional Resources or Supporting Citations
Institute of Child Nutrition: https://theicn.org/
University of North Carolina Center for Training and Research Translation: Evaluating Policy, Systems, and Environmental Changes: https://www.centertrt.org/?p=evaluation_interactive_framework
National Farm to School Network: https://www.farmtoschool.org/