In this evaluation of 40 schools participating in the HealthMPowers program, we observed significant improvements over time in student health-related knowledge, self-efficacy, and behaviors; student fitness as measured by BMI-for-age and performance on the PACER test; as well as in school policies and practices. Improvements were stronger in the first two years of the program as compared to the third year. We hypothesize several potential explanations for this difference: there could be a threshold reached beyond which improvement is more difficult to achieve, or the small sample size in this cohort could contribute to random error which could obscure a true effect.
Although our study design, without a control group, precludes direct comparison with some other programs studied, our results are comparable to those found by similarly comprehensive programs that incorporate not only classroom health education lessons and increased opportunities for physical activity, but also involve school staff and families in promoting and reinforcing positive behaviors. One program consistent with this philosophy is the Healthy Schools Program, a four-year program that involves the formation of a “school wellness council”, completion of an assessment, action planning, and technical assistance and implementation support (online tools, “train the trainer” sessions for teachers) to enable improvements in school environment to support health (e.g., PA opportunities for students, staff wellness, availability of nutritious food) . An evaluation of that program in 2012 showed that participating schools demonstrated significant improvements in employee wellness, school meals, health education and physical activity, as well as policies and systems ; however, no assessment was made of student-level knowledge or fitness outcomes and, as in the present report, this analysis lacked a control group. Another program involving the creation of school-specific action plans was assessed using a quasi-experimental design in 18 Swedish schools; this study indicated significant improvements of school policies and practices, but did not show any significant differences between intervention and control groups in student-level outcomes such as behavior or BMI . The Alberta Project Promoting active Living and healthy Eating (APPLE) program utilizes “School Health Facilitators”, individuals placed in each school to address the specific school environment—including facilitators and barriers to healthful policies, practices, and behaviors—and engage stakeholders at all levels, from parents, to students, to staff . A 2012 evaluation of this three-year intervention, in comparison to a set of randomly selected non-intervention schools in Alberta, found not only improvements in student behaviors (e.g., increased fruit and vegetable consumption, increased physical activity), but also decreases in weight . The Action Schools! BC intervention similarly provides support to schools to customize the inclusion of health education and physical activity promotion into school curricula, while also making environmental changes (e.g., modifications to playground equipment) and engaging family and community members . When evaluated in 2011 using a pre-post design similar to our own, this program was shown to increase aerobic fitness in students over the course of seven months, though few other significant changes were seen in student-level indicators (no significant changes in physical activity or BMI-for-age Z scores, for example) . Other comprehensive school-based programs also showed significant effects across some or all measures, although they were not tailored to specific schools, as were the above examples (and the HealthMPowers initiative). Reviews suggest that, at minimum, effective programs must incorporate increased opportunities for physical activity in addition to physical and health education classes, and that behaviors are easier to change than fitness levels (rev. in [7, 13, 30]), although changes in student fitness did occur with the HealthMPowers program. (However, it must be noted that PACER results are not directly comparable to other measures of fitness such as the one-mile run). Further, a whole-school approach in line with the WHO-recommended health promoting school framework [31, 32] appears to generate better student outcomes than a focused intervention where activities are confined to the classroom or physical education class.
This analysis has at least four strengths. First, student-level data was obtained not only on knowledge and behaviors, but also on BMI and aerobic capacity (as measured by the PACER test), enabling analysis of the program’s effect on student body composition and fitness. Second, student-level data were supplemented by school-level assessments of policies and practices, providing insight into how school environments change over the course of the HealthMPowers program. Furthermore, we were able to include data from multiple cohorts of schools, enabling us to look at the effect of the program over time. Lastly, the fact that this was an effectiveness study rather than an efficacy study strengthens the applicability of the findings to everyday practice. These results represent the effect of the HealthMPowers program as implemented in a real-world setting with a diverse student body, enhancing the generalizability of these results to other public school settings.
Despite these strengths, there are at least five limitations. First, and most significantly, although baseline data was collected for each school, there were no control schools, introducing the possibility that observed differences were due to secular changes over time. However, given that significant changes were observed across cohorts (i.e., year-on-year changes were observed regardless of start year), we feel that this is not likely to explain our findings. Additionally, although changes in student height and weight are to be expected over time, utilization of a BMI-for-age Z score takes into account this natural variation by age. Furthermore, recent data suggests that prevalence of obesity (defined, for children, by BMI-for-age measures) among elementary-school-aged children is remaining steady over time , making secular changes an unlikely explanation for the decreases seen in BMI-for-age Z score. We did do a sensitivity analysis with change in BMI, and found a mix of increases and decreases in BMI among the different strata of children overweight or obese at baseline (magnitude range: −0.28 – 0.51, mean change 0.16). All changes were nonsignificant. Given the age of the children (mean age 10.8), we would expect a slight increase in BMI over the course of the school year . While student aerobic capacity will also increase naturally over time, the fitness recommendations based on PACER performance don’t change between the ages of 10 and 11 (mean age in our cohort was 10.8 years), and average student performance (upon which criteria are based) changes very little from ages 11 to 12 . A second limitation is that due to the data collection design, we were unable to follow individual students from year to year. Thirdly, not all participating schools provided data on all indicators of interest; however, schools that did not provide data were similar demographically to schools that did provide data, with the exception of the mean percentage of students receiving free or reduced lunch (p = 0.025; although the mean total number of students receiving free or reduced lunch was not significantly different by inclusion status: p = 0.878). Another limitation is that the measurement of school-level changes was performed with the CITT instrument, which has not been separately validated despite having been used by HealthMPowers for over ten years. However, a sensitivity analysis comparing student outcomes between schools with above-average improvement as measured by CITT and schools with below-average improvement did uncover substantial differences in changes in student weight status, which helps to support the validity of the CITT. Specifically, students from above-average improvement schools experienced a −0.08 point decrease in BMI-for-age Z score, as compared to a −0.04 decrease among students from schools with below-average improvement (p = 0.0016). Similarly, students from above-average schools gained only 0.17 BMI points as compared to 0.28 BMI points in students from below-average schools (p = 0.02). Assessment of inter- and intra-rater reliability of the CITT, as well as test-retest reliability, is currently being undertaken at the time of publication; validity studies are also being considered. Lastly, this study is limited by the fact that student behavior data were self-reported, while PACER and BMI data were collected by teachers as opposed to health professionals or study staff. However, HealthMPowers has trained school staff in how to correctly collect PACER, height, and weight data, and also provides annual refresher training to schools. Further, a comprehensive fitness assessment manual was developed by HealthMPowers for the Georgia Department of Education, and an electronic version is available on the Georgia Department of Education’s website. Unfortunately, budget limitations precluded use of accelerometers or pedometers to objectively measure student physical activity.
Although this study contributes to the body of knowledge on childhood obesity prevention, some gaps remain to be addressed. Future research should focus on which specific elements of school-based programs are most effective in changing student behaviors and fitness levels—for instance, what is the contribution of a School Health Team? How does family engagement factor into student-level outcomes? While some randomized trials (e.g., Williamson et al. ) have attempted to assess the effect of primary prevention (environmental modification) compared to primary plus secondary prevention (education), few studies have specifically looked at the comparative effectiveness of program elements in detail. Additionally, more studies are needed that follow children enrolled in long-term (more than six-month) interventions over the course of several years. Lastly, given the potential connections between physical fitness and academic performance [9, 10], future research should measure academic performance as an outcome of increased levels of physical activity during the school day. Future research on the HealthMPowers program specifically could include following individual children across years, as well as assessing other outcomes such as academic performance.
Considering the HealthMPowers program along the RE-AIM (Reach, Efficacy, Adoption, Implementation, Maintenance) framework also provides a helpful way to assess its overall public health impact . As discussed above, HealthMPowers reached nearly 40,000 students over the course of the 2012 – 2013 school year, representing substantial reach. Although efficacy has not been demonstrated in a randomized controlled trial, the program’s effectiveness in improving student- and school-level health indicators has been shown in the above analysis. Adoption is by definition high given that this was an evaluation of already-participating schools. Further, given that this was an evaluation of an existing program, the results demonstrated reflect the current level of implementation and that level which would be expected in similar settings with similar resources. Maintenance (extent to which a program is sustained over time) is demonstrated with HealthMPowers by the fact that many schools choose to continue working with HealthMPowers even after the three-year program has finished, and the fact that few drop out. An analysis of sustainability conducted early in HealthMPowers’s history demonstrated that two years after its programming ceased in 12 schools, all of the schools continued to provide daily physical activity, annual fitness testing of students, and worksite health promotion activities for school staff. Eleven of the 12 schools continued to engage students in goal-setting and implementing a plan to improve their health-related fitness, to involve families in supporting their child’s self-improvement plan, and to make environmental and policy changes necessary to align health programming and practice .