Adding effect sizes to a systematic review on interventions for promoting physical activity among European teenagers
International Journal of Behavioral Nutrition and Physical Activity volume 7, Article number: 29 (2010)
This commentary adds effect sizes to the recently published systematic review by De Meester and colleagues and provides a more detailed insight into the effectiveness of interventions to promote physical activity among European teenagers. The main findings based on this evidence were: (1) school-based interventions generally lead to short term improvement in physical activity levels, but there were large differences between interventions with regard to effect sizes; (2) a multi-component approach (including environmental components) generally resulted in larger effect sizes, thereby providing evidence for the assumption that a multi-component approach should produce synergistic results; and (3) if an intervention aimed to affect more health behaviours besides physical activity, then the intervention appeared to be less effective in favour of physical activity.
The recently published systematic review by De Meester and colleagues  provides an excellent summary of the effectiveness of interventions to promote physical activity among European teenagers. The authors, however, based their conclusions merely on p-values while effect sizes are more appropriate to derive conclusions with regard to differences in the effectiveness of interventions. Therefore, in addition to the highly appreciated work done by De Meester and colleagues, I added effect sizes and provided an even more detailed insight into the effectiveness of interventions to promote physical activity among European teenagers.
Traditionally, null hypothesis-testing resulted in reporting of p-values as the key result of studies in behavioural sciences. Nowadays, however, reporting of effect sizes is deemed increasingly important . Volker  argues on why to report effect sizes. In short, p-values are not particularly informative to determine whether a statistically significant effect is meaningful and substantive. They are merely "a conditional probability indicative of the probability of a result at least as extreme as the obtained difference assuming that the null hypothesis is true."  Therefore, effect sizes (i.e, Cohen's d) are reported in this commentary to gain a more detailed insight into the meaningfulness and substantiveness of the results of the studies included by De Meester and colleagues . As correctly argued by De Meester and colleagues , performing a meta-analysis is problematic because of the heterogeneity of the outcome measures of the included studies. Nevertheless, effect sizes still provide the opportunity to quantify the intervention outcomes instead of having only a conditional probability. It needs to be stressed, however, that an effect size on its own is not equal to the public health impact of an intervention. According to the RE-AIM model, the public health impact of an intervention can only be evaluated by the assessment of five dimensions: Reach, Efficacy, Adoption, Implementation and Maintenance (hence the acronym RE-AIM) . Thus, the public health impact of an intervention that has a small effect, but reaches a large group of people, can still be high. Furthermore, it is important, when looking at the potential of interventions, not to lose sight of the quality assessment of the studies (irrespective of the intervention outcomes).
Table 1 gives an overview of the effect sizes of intervention outcomes. Effect sizes were calculated by using the formulas described by Lipsey and Wilson . As recommended by Morris , effect sizes were based on the pooled pre- and post-test standard deviation to obtain a more precise effect size estimate. Odds ratios were converted into Cohen's d (as described by Chinn ). Three studies did not report sufficient information to calculate effect sizes; one of the authors of these studies was able to provide the required information. In line with Cohen's classification [8, 9], effect sizes were divided into five levels: trivial (Cohen's d ≤ .2), small (> .2), moderate (> .5), large (> .8), and very large (> 1.3).
The effects sizes in Table 1 clearly demonstrate the differences between effect sizes and p-values. For example, the p-value of the difference between the intervention group and the control group in the study by Christodoulos and colleagues , with regard to total moderate to vigorous physical activity, did not reach significance. The effect size for this outcome measure, however, was very large (d = 2.79). This demonstrates the potential of this intervention with regard to behaviour change, apart from the small sample size in the specific study. Another example, the study of Hill and colleagues , shows that although the conditions in this study did not differ significantly, the effect sizes indicate a clear difference between the impact of these interventions (ranging from 0.18 to 0.45).
Taking into account the effect sizes reported in Table 1, the conclusion of De Meester and colleagues  that school-based interventions generally lead to short term improvement in physical activity levels still holds. There were, however, large differences between interventions with regard to effect sizes. These differences should be taken into account when judging the potential of intervention solutions. Therefore, the evidence with regard to involvement of family is inconclusive and recommendations regarding family involvement should be interpreted with caution as they are premature. In contrast to De Meester and colleagues , the evidence provided by effect sizes is not inconclusive with regard to a multi-component approach. Interventions that included environmental components (as identified in the published review) generally resulted in larger effect sizes. This provides evidence for the assumption that a multi-component approach should produce synergistic results. With regard to interventions aimed at multiple behaviours, it can be concluded that these interventions resulted in smaller effects with regard to physical activity. There appeared to be no differences in effect sizes related to quality assessment of the studies (as assessed in the published review). Nevertheless, when homogeneous outcome measures are available for future studies, meta-analyses are needed to fully warrant strong conclusions with regard to potential moderators of effect sizes (e.g., methodological quality).
Based on the evidence identified by the review of De Meester and colleagues  and the effect sizes reported in this commentary, a detailed insight into the effectiveness of interventions to promote physical activity among European teenagers is provided. In summary, the main findings based on this evidence:
(1) School-based interventions generally lead to short term improvement in physical activity levels, but there were large differences between interventions with regard to effect sizes.
(2) A multi-component approach (including environmental components) generally resulted in larger effect sizes, thereby providing evidence for the assumption that a multi-component approach should produce synergistic results.
(3) If an intervention aimed to affect more health behaviours besides physical activity, then the intervention appeared to be less effective in favour of physical activity.
De Meester F, Van Lenthe FJ, Spittaels H, Lien N, De Bourdeaudhuij I: Interventions promoting physical activity among European teenagers: a systematic review. International Journal of Behavioral Nutrition and Physical Activity. 2009, 6: 82-10.1186/1479-5868-6-82.
Kline R: Beyond significance testing: Reforming data analysis methods in behavioral research. 2004, Washington, DC: APA
Volker MA: Reporting effect size estimates in school psychology research. Psychology in the Schools. 2006, 43: 653-672. 10.1002/pits.20176.
Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. American Journal of Public Health. 1999, 89: 1322-1327. 10.2105/AJPH.89.9.1322.
Lipsey MW, Wilson DB: Practical meta-analysis. 2001, Thousand Oaks, CA: Sage publications
Morris SB: Estimating effect sizes from pretest-posttest control group designs. Organizational Research Methods. 2008, 11: 364-386. 10.1177/1094428106291059.
Chinn S: A simple method for converting an odds ration to effect size for use in meta-analysis. Statistics in Medicine. 2000, 19: 3127-3131. 10.1002/1097-0258(20001130)19:22<3127::AID-SIM784>3.0.CO;2-M.
Cohen J: Statistical power analysis for the behavioral sciences. 1988, Hillsdale, NJ: Lawrence Erlbaum associates, 2
Cohen J: A power primer. Psychological Bulletin. 1992, 112: 155-159. 10.1037/0033-2909.112.1.155.
Christodoulos AD, Douda HT, Polykratis M, Tokmakidis SP: Attitudes towards exercise and physical activity behaviours in Greek schoolchildren after a year long health education intervention. British Journal of Sports Medicine. 2006, 40: 367-371. 10.1136/bjsm.2005.024521.
Hill C, Abraham C, Wright DB: Can theory-based messages in combination with cognitive prompts promote exercise in classroom settings?. Social Science & Medicine. 2007, 65: 1049-1058.
Haerens L, Deforche B, Vandelanotte C, Maes L, De Bourdeaudhuij I: Acceptability, feasibility and effectiveness of a computer-tailored physical activity intervention in adolescents. Patient Education and Counseling. 2007, 66: 303-310. 10.1016/j.pec.2007.01.003.
Verstraete SJM, Cardon GM, De Clercq DLR, De Bourdeaudhuij IMM: Increasing children's physical activity levels during recess periods in elementary schools: the effects of providing game equipment. European Journal of Public Health. 2006, 16: 415-419. 10.1093/eurpub/ckl008.
Tsorbatzoudis H: Evaluation of a school-based intervention programme to promote physical activity: an application of the theory of planned behavior. Perceptual and Motor Skills. 2005, 101: 787-802. 10.2466/PMS.101.7.787-802.
Lubans D, Sylva K: Controlled evaluation of a physical activity intervention for senior school students: effects of the lifetime activity program. Journal of Sport & Exercise Psychology. 2006, 28: 252-268.
Murphy NM, Ni Dhuinn M, Browne PA, ÓRathaille MM: Physical activity for bone health in inactive teenage girls: is a supervised, teacher-led program or self-led program best?. Journal of Adolescent Health. 2006, 39: 508-514. 10.1016/j.jadohealth.2006.01.008.
Lindberg LC, Ståhle A, Rydén L: Long-term influence of a health education programme on knowledge and health behaviour in children. European Journal of Cardiovascular Prevention and Rehabilitation. 2006, 13: 91-97. 10.1097/00149831-200602000-00014.
Chatzisarantis NLD, Hagger MS: Effects of a brief intervention based on the Theory of Planned Behavior on leisure-time physical activity participation. Journal of Sports & Exercise Psychology. 2005, 27: 470-487.
Digelidis N, Papaioannou A, Laparidis K, Christodoulidis T: A one-year intervention in 7th grade physical education classes aiming to change motivational climate and attitudes towards exercise. Psychology of Sport and Exercise. 2003, 4: 195-210. 10.1016/S1469-0292(02)00002-X.
Harrison M, Burns CF, McGuinness M, Heslin J, Murphy NM: Influence of a health education intervention on physical activity and screen time in primary school children: 'Switch Off-Get Active'. Journal of Science and Medicine in Sport. 2006, 9: 388-394. 10.1016/j.jsams.2006.06.012.
Haerens L, De Bourdeaudhuij I, Maes L, Cardon G, Deforche B: School-based randomized controlled trial of a physical activity intervention among adolescents. Journal of Adolescent Health. 2007, 40: 258-265. 10.1016/j.jadohealth.2006.09.028.
Simon C, Wagner A, Platat C, Arveiler D, Schweitzer B, Schlienger JL, Triby E: ICAPS: a multilevel program to improve physical activity in adolescents. Diabetes & Metabolism. 2006, 32: 41-49.
Jurg ME, Kremers SPJ, Candel MJJM, wal Van der MF, De Meij JSB: A controlled trial of a school-based environmental intervention to improve physical activity in Dutch children: JUMP-in, kids in motion. Health Promotion International. 2006, 21: 320-330. 10.1093/heapro/dal032.
Moon AM, Mullee MA, Rogers L, Thompson RL, Speller V, Roderick P: Helping schools to become health-promoting environments - an evaluation of the Wessex Healthy Schools Award. Health Promotion International. 1999, 14: 111-122. 10.1093/heapro/14.2.111.
Baxter AP, Milner PC, Hawkins S, Leaf M, Simpson C, Wilson KV, Owen T, Higginbottom G, Nicholl J, Cooper N: The impact of heart health promotion on coronary heart disease lifestyle risk factors in schoolchildren: lessons learnt from a community-based project. Public Health. 1997, 111: 231-237. 10.1016/S0033-3506(97)00040-1.
Ortega-Sanchez R, Jimenez-Mena C, Cordoba-Garcia R, Muñoz-Lopez J, Garcia-Machado ML, Vilaseca-Canals J: The effect of office-based physician's advice on adolescent exercise behavior. 2004, 38.
Walker Z, Townsend J, Oakley L, Donovan C, Smith H, Hurst Z, Bell J, Marshall S: Health promotion for adolescents in primary care: randomised controlled trial. British Medical Journal. 2002, 325: 524-527. 10.1136/bmj.325.7363.524.
Kelleher CC, Fallon UB, McCarthy E, Dineen BD, O'Donnell M, Killian M, Hope A, Bluett D, Varley O, McDonagh G: Feasibility of a lifestyle cardiovascular health promotion programma for 8-15-years-olds in Irish general practice: results of the Galway Health Project. Health Promotion International. 1999, 14: 221-229. 10.1093/heapro/14.3.221.
Woods C, Mutrie N, Scott M: Physical activity intervention: a Transtheoretical Model-based intervention designed to help sedentary young adults become active. Health Education Research. 2002, 17: 451-460. 10.1093/her/17.4.451.
The author declares that they have no competing interests.
RC synthesized the data and calculated the effect sizes, interpreted the findings, and wrote the manuscript.
About this article
Cite this article
Crutzen, R. Adding effect sizes to a systematic review on interventions for promoting physical activity among European teenagers. Int J Behav Nutr Phys Act 7, 29 (2010). https://doi.org/10.1186/1479-5868-7-29