Treatment adherence and facilitator characteristics in a community based pediatric weight control intervention
© Jelalian et al.; licensee BioMed Central Ltd. 2014
Received: 10 April 2013
Accepted: 10 January 2014
Published: 13 February 2014
There is a pressing need to develop effective and broadly accessible interventions to address pediatric obesity. An important dimension in translating interventions to community settings is evaluating the fidelity with which the intended treatment is delivered and the level of facilitator needed to deliver the intervention with efficacy.
The primary objectives of this study were to: 1) provide descriptive information regarding adherence to protocol and non-specific facilitator characteristics (e.g. interpersonal characteristics, group management skills) within the context of a community based pediatric weight control intervention delivered by paraprofessionals; and 2) examine the relationships among facilitator adherence and characteristics and rate of change in percent overweight demonstrated by youth over the course of the 24-week intervention.
The intervention was conducted between February and September of 2011. Children (6–16 years) and parents completed primary outcome measures at baseline, 12, and 24 weeks (i.e. end of treatment). A 2-part rating form was developed to assess facilitator adherence to weekly content and general provider characteristics at two different time points during the intervention.
Youth participating in this study were on average 11.3 years old (SD = 2.8), with most being under the age of 13 years (74.2%). Over half were female (54.8%) and over two-thirds were White (68.4%). On average, facilitators adhered to 96.0% (SD = 5.2%) of the session content at Time 1 and 92.6% (SD = 6.8%) at Time 2. Higher Content Adherence at Time 1 and Time 2 were associated with greater loss in percent overweight.
Our data suggest that paraprofessionals without prior expertise in pediatric weight control can be trained to successfully deliver an intervention that is evidence based and incorporates behavioral and educational components. These findings need to be considered in light of some limitations, including the fact that facilitator domains were assessed with a modification of a standardized tool and we did not obtain inter-rater reliability of observations. These limitations not withstanding, investing time in training facilitators to adhere to a given protocol is critical and may be of higher priority than focusing on more general facilitator characteristics.
Pediatric obesity is an epidemic in the United States, with approximately one third of youth identified as overweight or obese [1, 2]. Obesity in youth is associated with a number of health risks, including elevations in cholesterol, diastolic and systolic blood pressure, triglycerides, and fasting insulin levels as well as increased risk for type 2 diabetes [3, 4]. Furthermore, obesity in children tracks into adulthood, which in turn is related to increased rates of morbidity and mortality . In addition to health risks, youth who struggle with excess weight are at greater risk for a range of psychosocial difficulties including decreased self-esteem, challenges in peer relationships [6, 7], and impairments in health related quality of life (HRQOL) [7, 8].
Family-based behavioral weight control programs have demonstrated efficacy in the treatment of overweight children between the ages of 8 and 12 years, with some promise regarding long-term outcomes. While effective, these interventions have been administered through carefully controlled clinical trials [9, 10] or delivered in specialized/tertiary clinical settings affiliated with academic medical centers. As a result, they are available for only a small percentage of children and adolescents struggling with excess weight. The need for identifying alternative, more broadly accessible delivery systems for these interventions increased with the USPSTF recommendation that primary care providers refer youth to behavioral intervention programs .
There are very few examples of interventions to assist children and adolescents who are already overweight/obese that are delivered in community settings and available to a greater segment of the population. We are aware of only three studies explicitly designed to examine effectiveness of pediatric obesity interventions in a community setting. Project STORY  examined the utility of Cooperative Extension Service offices in rural settings for delivery of a weight control intervention for children 8–12 years. Children assigned to both a family based and parent only condition demonstrated greater decrease in BMI z-score than those assigned to wait list control at 10-month assessment. A second effectiveness study involved examination of the MEND Program developed in the United Kingdom . Children between 8–12 years randomized to receive the intervention demonstrated significant decreases in BMI (-1.2 kg/m2) and waist circumference (-4.1 cm) compared to those assigned to a delayed treatment control. More recently, the JOIN program, developed by UnitedHealth Group and delivered through the YMCA, was shown to significantly decrease percent overweight among school age children .
While these interventions have been effective, little is known regarding the fidelity of treatment implementation, including adherence to treatment protocols and facilitator characteristics. Such information is critical for scalability of health change interventions, as it has direct implications for interpretation and potential generalizability of study findings. In a recent review, only 5% of more than 75 weight control studies reported on the extent to which the intervention content was delivered as intended . Attention to treatment fidelity is also highlighted by the NIH Behavior Change Workgroup, which includes recommendations for defining, evaluating, and enhancing treatment fidelity in behavioral interventions . In addition to examining the extent to which the intervention was delivered as intended, the Workgroup further emphasized the importance of assessing provider characteristics (e.g. warmth) .
The importance of treatment fidelity is further highlighted in the RE-AIM framework , a model helping to increase the external validity and public health impact of health interventions. Within this framework, “Implementation” is a key domain that includes the extent to which project staff delivered an intervention as intended . Historically, both weight control prevention and intervention studies have provided minimal information regarding the consistency with which interventions are delivered and potential differences of treatment efficacy across interventionists [15, 19]. We are not aware of any previous research that has examined the combination of treatment fidelity and provider characteristics as related to outcomes for pediatric behavioral weight control interventions. Assessment of fidelity is particularly important in the context of development and implementation of a scalable intervention, given the absence of a tightly controlled environment and the broad scale implications for dissemination.
We had the opportunity to examine treatment fidelity and facilitator characteristics in the context of a recently delivered 24-week community based pediatric weight control program for youth ages 6–17 years old. The JOIN program  involved a collaboration between UnitedHealth Center for Health Reform and Modernization and the YMCA of Greater Providence to deliver an empirically supported family based behavioral weight control intervention. Intervention components from evidence based treatments were modified to reduce cost and increase scalability. Specific modifications included: intervention delivery by YMCA facilitators with no previous training in pediatric obesity, combined child and parent groups, and 12 in-person sessions (efficacy trials typically include a greater number of sessions), with 12 additional sessions conducted by parents at home. Findings from the initial pilot were recently published and indicated a 4.3% reduction in percent overweight for children between 6–12 years, with a smaller decrease for adolescents . These results have significant implications for continued development of scalable interventions to address pediatric obesity that can be delivered in community settings. To be effective, such interventions will require replicable models for training paraprofessional facilitators as well as identification of key facilitator characteristics that are related to positive outcomes.
The purpose of this study was to examine two key components of treatment fidelity: facilitator adherence to treatment content and non-specific facilitator characteristics in the context of a community based pediatric weight control intervention. Primary study objectives were to: 1) provide descriptive information regarding adherence to the treatment protocol and non-specific facilitator characteristics (e.g. interpersonal characteristics, group management skills); and 2) examine the relationships among facilitator adherence and characteristics and rate of change in percent overweight demonstrated by youth over the course of the 24-week intervention.
Children and parents completed primary outcome measures at three evaluation points, baseline, 12 weeks, and 24 weeks (i.e. end of treatment).
Body weight was measured on calibrated scales (Detecto Model 6129, Cardinal Scale Manufacturing Company, Webb City, MO) with participants wearing light clothing and no shoes. Height was measured using a stadiometer (Seca 217 Mobile Stadiometer, Seca, Hamburg, Germany). Body mass index was calculated as weight (kg) divided by height (m)2. Percentage overweight was calculated as the percentage over the median BMI for age and gender.
A total of 16 groups (12 child, 4 adolescent) were conducted at eight different YMCA sites in the Providence, RI area. Nine of the groups were based in urban locations, with seven in suburban/rural settings. Sessions were led by nine YMCA-based facilitators, 7 of whom had Bachelor’s degrees and 2 of whom had Masters degrees. All of the facilitators had some experience supervising groups and 7 had experience with behavioral coaching. None of the facilitators had experience with pediatric obesity, although 3 of them had prior experience with adult weight control. Three of the nine facilitators led both child and adolescent groups.
YMCA facilitators received a total of 4.5 days of in-person training with experienced behavioral weight control interventionists (Drs. Foster and Jelalian), including 2 ½ days prior to start of the program and 2 1-day sessions during the course of the intervention. The initial training session included an orientation to the program, review of behavioral weight control strategies with children and adolescents, recommendations for managing group interactions, review of the first several lesson plans, and information regarding involvement in research and administrative oversight. Approximately 1 ½ days were focused on clinical content. The additional 2 full-day trainings focused exclusively on review of treatment materials and role-plays of lesson content. Throughout treatment delivery, facilitators also participated in 1-hour weekly telephone based supervision meetings to facilitate adherence. Approximately 75% of each hour long call was devoted to treatment related issues, including review of upcoming content, a question and answer period, and concerns regarding specific participants. The remaining time was dedicated to logistical issues such as use of the program tracking tool to document weekly attendance and weight. This study was approved by the New England Institutional Review Board.
The intervention included a total of 12 face-to-face sessions conducted at the YMCA and 12 briefer (i.e. 10–15 minute) home-based sessions led by parents. Home sessions were designed to last 10–15 minutes and followed the general format of a parent and child reviewing progress with behavioral goals and the parent briefly introducing new content provided to them by the YMCA facilitator. Face to-face sessions were held across the 24-week intervention at weeks: 1, 2, 3, 4, 8, 10, 12, 14, 16, 18, 20, and 24. Each facilitator was observed twice during the 24-week intervention. The “time 1” observation was conducted during session 1, 2, 3, or 4 of the face-to-face sessions. The “time 2” observation was conducted during session 8, 10, 12, 14, 16, 18, or 20 of the face-to-face sessions. All facilitators except for one were observed once at time 1 and once at time 2. This facilitator was observed only once because the group was discontinued. For the three facilitators who conducted both child and teen groups, observations were made for one of the child and adolescent groups, resulting in a total of 12 observations at each of two time points. All observations were conducted by one of two observers, a postdoctoral fellow in pediatric psychology or a clinical psychologist with expertise in weight control.
A 2-part rating form was developed to assess facilitator adherence to weekly content and general provider characteristics. Weekly content adherence was measured as the extent to which the facilitator covered each of the prescribed topics, with ratings of Yes, No, and Partial. Nine-items evaluated more general provider characteristics such as interpersonal skills and ability to effectively manage group interactions. These items were adapted from a well validated psychotherapy assessment tool  and rated on a 5-point likert scale. A final item included assessment of overall expertise (1–3 = “Novice,” 4–6 = “Intermediate,” and 7–9 = “Advanced”).
Parameter estimates and inferential statistics for the growth models (N = 155)
Model 1: unconditional lLinear
Initial Percent Overweight (POW) with slope (rate of change)
Slope (rate of change)
POW week 12
POW week 24
Model 2: predictors of rate of change
Age in months
Facilitator characteristics (time 1)
Facilitator characteristics (time 2)
Expert rating (time 1)
Expert rating (time 2)
Percent content adherence (time 1)
Percent content adherence (time 2)
Participants included 155 children and one parent/guardian per child, and were referred to the program through area pediatricians, school nurses, and postings at local YMCA facilities. Participating youth were on average 11.3 years old (SD = 2.8), with more children under the age of 13 years (74.2%) than adolescents ages 13 and older (25.8%). Over half of children participating in this study were female (54.8%) and over two-thirds were White (68.4%). At baseline, participants had an average BMI of 30.5 (SD = 7.3), a BMI z-score of 2.23 (SD = .41), and percentage overweight of 72.5 (SD = 34.0). The vast majority (91.6%) of youth were obese (≥ 95th BMI%ile) and nearly half (46.5%) met the 99th BMI%ile for age and gender.
Weight change outcomes
Eighty four percent of the sample (n = 130) completed the 24 week assessment, this included 86% of children (99/130) and 78% of adolescents (31/40). Completers were defined as participants who attended at least one group session and were present for the 24-week assessment. Completers and non-completers did not differ on baseline characteristics. We also examined attendance by age group and found that children attended a mean of 8.62 face-to-face sessions and 8.16 home-based sessions, while adolescents attended an average of 9.39 face-to-face sessions and 9.80 home-based sessions. Attendance rates did not differ significantly for children <13 and those ≥ 13 years.
For the overall sample, there was a significant reduction in percent overweight (M = 3.5%) from baseline to 24 weeks. Children (<13 years old) experienced a 4.5% (<.001) reduction in percent overweight from baseline to 24 weeks, while adolescents (≥ 13 years old) did not show a significant reduction (0.2% reduction in percent overweight, p > .05. This may reflect the fact that a number of adolescents had BMIs in excess of 45 and may have been less well suited for a lifestyle intervention. No significant gender effects emerged for change in percent overweight at 24 weeks.
Intervention content adherence
An observation of the extent to which group facilitators adhered to intervention content was conducted twice for each of the nine facilitators. On average, facilitators were observed to adhere to 96.0% (SD = 5.2%) of the session content at time 1. Adherence to session content at time 2 was slightly lower, at 92.6% (SD = 6.8%). There were no significant differences on any domains for facilitators who led one group (n = 5) compared to those who led more than one group (n = 4).
Facilitator characteristics and expert facilitator rating
Facilitator characteristics at Time 1 and Time 2
Time 1 ratings
Time 2 ratings
n = 12
n = 11
Item 1: Transition smoothly between activities
Item 2: Use open-ended questions as a means of encouraging participants to talk
Item 3: Encourage children and parents involvement with discussion and activities
Item 4: Encourage group members to develop their own solutions to problems and prompt use of problem-solving when appropriate
Item 5: Manage participant behavior effectively to minimize disruptions to group process (e.g. maintain focus, help get participants back on track)
Item 6: Demonstrate strong interpersonal skills by expressing warmth, genuineness, and enthusiasm while delivering content
Item 7: Reinforce participant efforts by using praise and minimal encouragements (e.g. “uh-huh”, “okay”)
Item 8: Support participants by acknowledging progress toward weekly behavioral goals
Item 9: Deliver intervention without relying too much on written manuals (e.g. not reading)
Total sum of facilitator characteristics (sum across nine items)
Facilitator characteristics (average across nine items)
Expert facilitator rating
Results from the first growth model are presented in Table 1 (model 1). Youth (on average) began at 72.10% overweight (intercept) and their percent overweight decreased significantly by 0.14 per week over the 24 week program. Youth’s rate of change was unrelated to their initial percent overweight. Table 1 also reveals significant individual differences in youth’s initial percent overweight and rate of change, which permitted further investigation of predictors of change.
Correlations among study variables, (off diagonals), and means and (Standard Deviations) corrected for non-normality and nesting
1. Age (months)
2. Facilitator characteristics (time 1)
3. Facilitator characteristics (time 2)
4. Expert rating (time 1)
5. Expert rating (time 2)
6. Percent content adherence (time 1)
7. Percent content adherence (time 2)
8. POW (week 1)
9. POW (week 12)
10. POW (week 24)
Facilitator training for this community based pediatric weight control intervention, which consisted of approximately 2 ½ days of in person didactics followed by two 1-day booster sessions, and weekly phone supervision, resulted in a high level of adherence in delivering weekly content as intended. It is noteworthy that during both the first and second observations, facilitators exceeded 90% in their coverage of prescribed material. These findings clearly document that paraprofessionals without prior experience in pediatric weight control can be effectively trained to deliver a complex evidenced based weight control intervention.
Overall, YMCA staff also scored highly on ratings of facilitator characteristics (e.g., expressing warmth, genuineness, and enthusiasm) traditionally associated with positive outcomes in the psychotherapy literature . Relatively lower scores were observed on use of open ended questions and attention to behavioral goals, while higher ratings were seen on less specific domains of interpersonal skills and managing group behavior.
To our knowledge, this is the first community based obesity treatment study to include examination of intervention implementation. Previous research suggests variability in treatment fidelity for programs targeting youth health practices (e.g. substance abuse) in a real-world setting . Although not specific to obesity, assessment of fidelity within the context of school-based nutrition education curriculum demonstrates that fidelity and facilitator rapport with youth are associated with increases in student knowledge .
Contrary to our expectation, higher ratings of general facilitator characteristics at time 2 were associated with participants achieving less change in BMI. One possibility for this finding is that facilitators recognized that children in their group were struggling and subsequently increased their efforts with regard to the non-specific facilitator characteristics measured in this rating. Alternatively, this finding may have resulted from measurement error and/or facilitator reactivity to in-vivo observation. In particular, facilitators with group members who were not doing as well with regard to weight loss may have increased their efforts in response to the presence of observers.
A number of limitations need to be taken into account in considering the study findings. First, the nature of the study prohibited use of videotaped observations. As a result, facilitators were aware of observers within the group meetings and we were unable to calculate inter-rater reliabilities for observations. To alleviate potential reactivity to in-vivo observation, facilitators were assured that ratings would not be shared with YMCA staff and that ratings would be anonymous at the data entry phase. In addition, observers remained in the background to minimize any impact on group participants. Despite these limitations, differences in adherence were related to weight change, suggesting that reactivity of observations may not have been a major concern. A second limitation is that we used a modified measure of intervention fidelity. While standardized measures of intervention fidelity for certain interventions (e.g. motivational interviewing) do exist , we did not utilize a standardized measure because this type of measure does not exist for community-based behavioral weight control. Inclusion of a standardized measure with evidence of reliability and validity would have strengthened this study. Finally, this study did not include pre- and post-tests related to facilitator content knowledge included in the training. Examining changes in facilitator knowledge would be helpful for informing refinement of facilitator training practices. It is also important to recognize that adherence to content reflects only one aspect of a complex process through which content delivery, and participant knowledge acquisition and learning, are related to behavior change .
Despite these limitations, this study provides important information. Our data suggest that paraprofessionals without prior expertise in pediatric weight control can be trained to successfully deliver an intervention that is evidenced based and incorporates behavioral and educational components. The observed adherence was even more impressive given the fact that facilitators had to manage the challenges of parents and children attending the same group session. Given the need for continued development and implementation of weight control interventions in community settings, these findings have important implications. Second, our findings suggest that investing time in training facilitators to adhere to a given protocol is critical and of higher priority than focusing on more general facilitator characteristics. It is noteworthy that even with high rates of content adherence, differences in adherence were related to changes in weight status. Just as models of learning  emphasize the importance of analyzing and applying knowledge in the process of acquiring knowledge, it is important to consider how paraprofessionals trained to deliver the intervention may have utilized additional strategies past simply verbalizing the curriculum (e.g., applying intervention content to the unique children/parents in their groups). Future research should include videotaped or audiotaped session observations to allow for assessment of observer reliability as well as consider the potential for identifying less intensive training models that result in equally high levels of facilitator adherence. In addition, it would be valuable to move beyond evaluation of content adherence to examine both participant and facilitator understanding of content (and ability to apply content to real-life situations) as related to behavioral outcomes.
The authors would like to thank the study participants, UnitedHealth Group Center for Health Reform and Modernization, YMCA of the USA, YMCA of Greater Providence and United Health Group’s Scientific Advisory Board on Childhood Obesity (Robert I. Berkowitz, MD, Leonard H. Epstein, Ph.D., Elissa Jelalian, Ph.D., Marsha D. Marcus, PhD and Denise Wilfley, PhD).
- Ogden CL, et al: Prevalence of obesity in the United States, 2009–2010. NCHS Data Brief. 2012, 82: 1-8.Google Scholar
- Ogden CL, et al: Prevalence of obesity and trends in body mass index among US children and adolescents, 1999–2010. JAMA. 2012, 307 (5): 483-490. 10.1001/jama.2012.40.View ArticleGoogle Scholar
- Daniels SR: The consequences of childhood overweight and obesity. Future Child. 2006, 16 (1): 47-67. 10.1353/foc.2006.0004.View ArticleGoogle Scholar
- Daniels SR, et al: Overweight in children and adolescents: pathophysiology, consequences, prevention, and treatment. Circulation. 2005, 111 (15): 1999-2012. 10.1161/01.CIR.0000161369.71722.10.View ArticleGoogle Scholar
- Fontaine KR, et al: Years of life lost due to obesity. JAMA. 2003, 289 (2): 187-193. 10.1001/jama.289.2.187.View ArticleGoogle Scholar
- Janssen I, et al: Associations between overweight and obesity with bullying behaviors in school-aged children. Pediatrics. 2004, 113 (5): 1187-1194. 10.1542/peds.113.5.1187.View ArticleGoogle Scholar
- Zeller MH, et al: Health-related quality of life and depressive symptoms in adolescents with extreme obesity presenting for bariatric surgery. Pediatrics. 2006, 117 (4): 1155-1161. 10.1542/peds.2005-1141.View ArticleGoogle Scholar
- Schwimmer JB, Burwinkle TM, Varni JW: Health-related quality of life of severely obese children and adolescents. Jama. 2003, 289 (14): 1813-1819. 10.1001/jama.289.14.1813.View ArticleGoogle Scholar
- Epstein LH, et al: Five-year follow-up of family-based behavioral treatments for childhood obesity. J Consult Clin Psychol. 1990, 58: 661-664.View ArticleGoogle Scholar
- Epstein LH, et al: Ten-year outcomes of behavioral family-based treatment for childhood obesity. Healthy Psychol. 1994, 13: 373-383.View ArticleGoogle Scholar
- Barton M, United States Preventive Services Task Force: Screening for obesity in children and adolescents: US preventive services task force recommendation statement. Pediatrics. 2010, 125 (2): 361-367.View ArticleGoogle Scholar
- Janicke DM, et al: Sensible treatment of obesity in rural youth (STORY): design and methods. Contemp Clin Trials. 2008, 29 (2): 270-280. 10.1016/j.cct.2007.05.005.View ArticleGoogle Scholar
- Sacher PM, et al: Randomized controlled trial of the MEND program: a family-based community intervention for childhood obesity. Obesity (Silver Spring). 2010, 18 (Suppl 1): S62-S68.View ArticleGoogle Scholar
- Foster GD, et al: Feasibility and preliminary outcomes of a scalable. Community-based treatment of childhood obesity. Pediatrics. 2012, 130 (4): 652-659. 10.1542/peds.2012-0344.View ArticleGoogle Scholar
- Klesges LM, et al: External validity reporting in behavioral treatment of childhood obesity: a systematic review. Am J Prev Med. 2012, 42 (2): 185-192. 10.1016/j.amepre.2011.10.014.View ArticleGoogle Scholar
- Bellg AJ, et al: Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004, 23 (5): 443-451.View ArticleGoogle Scholar
- Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999, 89 (9): 1322-1327. 10.2105/AJPH.89.9.1322.View ArticleGoogle Scholar
- Estabrook B, Zapka J, Lemon SC: Evaluating the implementation of a hospital work-site obesity prevention intervention: applying the RE-AIM framework. Health Promot Pract. 2012, 13 (2): 190-197. 10.1177/1524839910385897.View ArticleGoogle Scholar
- Klesges LM, Dzewaltowski DA, Glasgow RE: Review of external validity reporting in childhood obesity prevention research. Am J Prev Med. 2008, 34 (3): 216-223. 10.1016/j.amepre.2007.11.019.View ArticleGoogle Scholar
- Dobson KS, Shaw BF, Vallis TM: Reliability of a measure of the quality of cognitive therapy. British Journal of Clinical Psychology. 1985, 24 (4): 295-300. 10.1111/j.2044-8260.1985.tb00662.x.View ArticleGoogle Scholar
- Muthén LK, Muthén BO: Mplus statistical analysis with latent variables User’s guide. 1998–2010, Los Angeles, CA: Muthén & Muthén, 6Google Scholar
- Norcross JC: Psychotherapy relationships that work: Therapist contributions and responsiveness to patients. 2002, New York, New York: Oxford University Press, IncGoogle Scholar
- Ennett ST, et al: Evidence-based practice in school substance use prevention: fidelity of implementation under real-world conditions. Health Educ Res. 2011, 26 (2): 361-371. 10.1093/her/cyr013.View ArticleGoogle Scholar
- Resnicow K, et al: How best to measure implementation of school health curricula: a comparison of three measures. Health Educ Res. 1998, 13 (2): 239-250. 10.1093/her/13.2.239.View ArticleGoogle Scholar
- Faulkner MS: Intervention fidelity: ensuring application to practice for youth and families. J Spec Pediatr Nurs. 2012, 17 (1): 33-40. 10.1111/j.1744-6155.2011.00305.x.View ArticleGoogle Scholar
- Bloom BS: Taxonomy of educational objectives, handbook I: the cognitive domain. 1956, New York: David McKay Co Inc.Google Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.