Skip to main content
Download PDF
Download ePub

Is level of implementation linked with intervention outcomes? Process evaluation of the TransformUs intervention to increase children’s physical activity and reduce sedentary behaviour

International Journal of Behavioral Nutrition and Physical Activity volume 19, Article number: 122 (2022) Cite this article

Abstract

Background

TransformUs was a four-arm school-based intervention to increase physical activity and reduce sedentary behaviour among primary school children. Pedagogical and environmental strategies targeted the classroom, school grounds and family setting. The aims of this study were to evaluate program fidelity, dose, appropriateness, satisfaction and sustainability, and associations between implementation level and outcomes among the three intervention arms.

Methods

At baseline, 18-months (mid-intervention) and 30-months (post-intervention), teachers, parents and children completed surveys, and children wore GT3X ActiGraph accelerometers for 8 days at each time point to determine physical activity and sedentary time. Implementation data were pooled across the three intervention groups and teachers were categorised by level of implementation: (i) ‘Low’ (< 33% delivered); (ii) ‘Moderate’ (33–67% delivered); and (iii) ‘High’ (> 67% delivered). Linear and logistic mixed models examined between group differences in implementation, and the association with children’s physical activity and sedentary time outcomes. Qualitative survey data were analysed thematically.

Results

Among intervention recipients, 52% (n = 85) of teachers, 29% (n = 331) of parents and 92% (n = 407) of children completed baseline evaluation surveys. At 18-months, teachers delivered on average 70% of the key messages, 65% set active/standing homework, 30% reported delivering > 1 standing lesson/day, and 56% delivered active breaks per day. The majority of teachers (96%) made activity/sports equipment available during recess and lunch, and also used this equipment in class (81%). Fidelity and dose of key messages and active homework reduced over time, whilst fidelity of standing lessons, active breaks and equipment use increased. TransformUs was deemed appropriate for the school setting and positively received. Implementation level and child behavioural outcomes were not associated. Integration of TransformUs into existing practices, children’s enjoyment, and teachers’ awareness of program benefits all facilitated delivery and sustainability.

Conclusions

This study demonstrated that intervention dose and fidelity increased over time, and that children’s enjoyment, senior school leadership and effective integration of interventions into school practices facilitated improved intervention delivery and sustainability. Teacher implementation level and child behavioural outcomes were unrelated, suggesting intervention efficacy was achieved irrespective of implementation variability. The potential translatability of TransformUs into practice contexts may therefore be increased. Findings have informed scale-up of TransformUs across Victoria, Australia.

Trial registration

International Standard Randomized Controlled Trial Number ISRCTN83725066; Australian New Zealand Clinical Trials Registry Number ACTRN12609000715279. Registered 19 August 2009. Available at: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=308387&isReview=true

Background

The positive relationships between higher levels of physical activity in children and improved cardiometabolic health risk factors, musculoskeletal health, mental health and wellbeing, cardiorespiratory fitness, and a reduced risk of unhealthy weight gain are well established [1]. More recently, time spent in prolonged sedentary behaviour (expending < 1.5 metabolic equivalents [METs] whilst in a sitting or reclining posture; e.g. computer use, TV viewing [2]) has been suggested as an independent risk factor for cardiometabolic diseases in children and youth [3,4,5]. However, evidence for the prospective negative association between prolonged sitting and biomedical health indicators in children remains inconclusive [6, 7]. Global physical activity guidelines recommend that children aged 5–17 years should accumulate, on average, 60 minutes of daily moderate- to vigorous-intensity physical activity (MVPA) [8]; however, the majority of children in high-income countries do not achieve this [9]. Only 22% of children from the United States (aged 6–19 years) [10] and England (aged 5-15 years) [11] achieve this guideline. In 2011–12, less than 40% of Australian children aged 9–13 years achieved the recommended 60 minutes of MVPA per day, and only 7% met the Australian sedentary behaviour guidelines of less than 2 h screen time per day [12]. Since physical activity and sedentary behaviour have been shown to track from childhood into adolescence and adulthood [13, 14], early intervention to optimise children’s physical activity and sedentary behaviour levels in childhood is preferable.

Increasingly, sedentary behaviour has been targeted in interventions alongside physical activity [15], and targeting both behaviours simultaneously may achieve the greatest health benefits [15]. Schools are ideal settings for physical activity and sedentary behaviour interventions, due to mandated schooling [16], and the majority of children spend large proportions of their waking hours sedentary at school [17, 18]. Further, making small changes to the school environment, such as installation of playground line-markings [19], equipment provision [20] and reducing playground density [21] can increase physical activity in large numbers of children, while introducing height-adjustable standing desks or classroom equipment (e.g. balls and bean bags to facilitate movement integration in the classroom) [22, 23] can reduce sedentary time. However, there remains a paucity of successful interventions combining physical activity promotion and sedentary behaviour reduction in children [24].

The lack of robust evidence for successful interventions targeting these behaviours in the school setting led to the development of a novel whole-of-school intervention to promote physical activity and reduce sedentary behaviours among 8–9 year old children, referred to as TransformUs [25]. The intervention incorporated behavioural and environmental strategies in the classroom, school grounds and home setting. It was based on social cognitive theory [26], behavioural choice theory [27] and ecological systems theory [28]. Alignment of these theories to the TransformUs intervention components and objectives is shown in Additional File 1. The efficacy of TransformUs was tested in a four-arm cluster-randomised controlled trial (RCT) involving 20 primary schools, 226 teachers and over 1600 children in Melbourne, Australia (2010–13). The three intervention arms targeted reductions in sedentary behaviour (SB-I group), increases in physical activity (PA-I group), and a combination of both (SB + PA-I group). These groups were compared to a usual practice control group. Six-month intervention effects of TransformUs showed significant increases in MVPA in the SB-I and PA-I groups during recess [29], and the SB + PA-I group spent 13.3 min/day less in weekday sedentary time compared to the control group [30]. Compared to the control group, children in the PA-I group spent 27 minutes less time sedentary at 18-months and those who received the sedentary intervention spent 5 minutes more in MVPA (at 18-months) and 33 minutes less time sedentary (at 30-months) [31].

Although the program demonstrated efficacy, the effectiveness varied by group at different time points, and the process and challenges of the implementation remain unexplored. Outcome data in isolation informs intervention effectiveness, while process evaluation determines whether an intervention is delivered or received as intended, and identifies influences on effectiveness and the potential for sustainability in routine practice [32,33,34]. Assessment of implementation elucidates the implications of intervention fidelity, dose and adaptation on program outcomes [35, 36]. Fidelity refers to the degree to which an intervention is implemented as it is prescribed in the original protocol [37], adaptation is the degree to which an intervention is changed or modified by a user during adoption and implementation [38], and dose refers to the amount of the intervention delivered [39]. Program fidelity is a particularly contested area of implementation research, as there is tension between the extent that an intervention remains ‘true’ to the program protocol to maximise the potential for positive impact, versus the reality of implementation in practice where adaptation is expected and may be encouraged for quality improvement [40]. For improved research-practice translation, there is increasing acknowledgement that interventions, and their implementation, may require ongoing adaptation for contextual relevance [41]. Yet adaptation can lead to both positive and negative outcomes on program impact. Despite that. Fidelity and adaptation ‘co-exist’ [42]. Nonetheless, the extent that adaptation impacts on program effectiveness is likely to be program, setting and population specific. The aims of this paper are therefore twofold: firstly, to assess differences in fidelity, dose, appropriateness, satisfaction and sustainability between Transform Us! intervention groups and over time; and secondly, to examine the associations between overall teacher implementation level (dose and fidelity across intervention groups combined) and child physical activity and sedentary behaviour outcomes. Findings from this evaluation will contribute to knowledge on effective implementation of school-based physical activity and sedentary behaviour interventions and the association between levels of implementation and intervention outcomes.

Methods

Overview of TransformUs

A detailed description of TransformUs has been published elsewhere [25], and Additional File 2 contains the TIDieR checklist. Briefly, the objectives of the intervention were to: (i) provide a whole-of-school environment that increased opportunities and support for physical activity and reducing sedentary behaviour; (ii) modify curriculum content and classroom lesson delivery to incorporate physical activity and sedentary behaviour messages, and provide an environment conducive to active behaviours; and (iii) increase awareness and opportunities for physical activity and reduced sedentary behaviours in the home setting. The underlying hypothesis was that the intervention would lead to behaviour change by impacting targeted behavioural and environmental mediators.

Complete delivery of TransformUs involved implementation of six behavioural and environmental components, which varied by intervention arm (SB-I, PA-I and SB + PA-I) [25]. Figure 1 presents the program logic model, Additional File 1 illustrates how the intervention components correspond to the intervention arms. The SB-I arm aimed to reduce prolonged sitting in the classroom and reduce overall sedentary time at home. The PA-I arm aimed to increase or maintain MVPA during morning recess and lunch time within school hours, and increase physical activity at home. The SB + PA-I group combined strategies from the SB and PA arms, with the aim of simultaneously reducing prolonged sedentary time and increasing physical activity at school and at home. In the first year of the intervention (2010), all Grade 3 teachers attended a 2-hour face-to-face professional development (PD) training session led by the research team. This was repeated in the second year (2011) with all Grade 4 teachers, which included teachers previously trained in Grade 3 and new teachers within Grade 4. The teacher training covered study requirements, the intervention strategies and aims of the project. A mid-year morning tea in 2011 facilitated a problem solving question and answer session. In year three (2012), when the cohort of children were in Grade 5, no face-to-face training was provided to those teachers, they only received provision of the resources. No additional implementation support was provided to teachers or schools, nor was there any ongoing implementation support provided between the data collection time points.

Fig. 1
figure 1

TransformUs Logic Model. PD: Professional Development; PA: Physical Activity; SB: Sedentary Behaviour

Full size image

Process evaluation design

A mixed method post-hoc study was adopted based on the UK Medical Research Council (MRC) recommendations for process evaluation of complex interventions [32], as it explores the unanticipated barriers and facilitators to implementation. Specifically, we used a concurrent triangulation design, whereby the qualitative and quantitative data were used in equal weighting to interpret the study findings [43]. The design was Six process indicators were included based on definitions and priority areas identified in evaluation research [44,45,46,47,48]. Indicators included: (i) Fidelity (teacher adherence to delivery of components as intended and adaptations to key message delivery); (ii) Dose delivered (proportion, frequency and duration of components delivered by teachers); (iii) Dose received (proportion, frequency and duration of components parents and children were exposed to); (iv) Appropriateness (teacher perceived fit, compatibility and ease of delivery of classroom components); (v) Satisfaction (teacher and parent planned continuation and/or support for TransformUs, and children’s enjoyment of the program); and (vi) Sustainability (facilitators and barriers to sustained implementation and integration of TransformUs into school policy).

Participants

Primary schools within a 50 km radius of Melbourne, with an enrolment > 300 children and at least two Grade 3 classes, were eligible to participate in the TransformUs RCT [25]. Schools were stratified by low, mid and high socio-economic status (SES) based on the Australian Bureau of Statistics’ Socio-Economic Indexes for Areas (SEIFA) [49]. Of the then 1159 Government primary schools in Victoria [50], 148 (12.8%) schools were invited to participate (via fax or email) and 29 (20%) responded. A target recruitment of 20 schools (15 intervention and 5 control schools) was achieved from the 29 responders. Schools represented low (n = 8), mid (n = 11) and high (n = 1) SEIFA areas. Due to challenges recruiting schools representing high SES areas, the high and mid SES strata were combined for randomisation. Schools within the two SES strata were randomised by a statistician (not involved in the trial) into one of the four groups (PA-I, SB-I, SB + PA-I or control current practice [C]). All Grade 3 children (n = 1606) in intervention and control schools were invited to participate in the study evaluation. Grade 3 children were recruited for two reasons. Firstly, children aged 8–9 years are a target population for increases in physical activity [51], and secondly, it facilitated follow-up as children would have remained at primary school throughout the study duration [25]. Active informed parental consent was required on behalf of themselves and their child. Parents could consent for their child to take part in any combination of the evaluation assessments as involvement in all assessments was not compulsory. For the purposes of this paper, only data from children participating in the intervention were used for analyses relating to child outcomes.

Data collection procedure

Process-level quantitative and qualitative data were collected via parent, teacher and child surveys at baseline (T1; Feb-June 2010), 6-months (T2; Nov-Dec 2010), 18-months (T3; Nov-Dec 2011) and 30-months (T4; Nov-Dec 2012). Due to low participant response rates at T2 and that the study purpose was to assess intervention implementation after substantial delivery of program components; only data collected at T1, T3 and T4 are included as part of this evaluation. Table 1 presents application of the process and outcome data against the six evaluation indicators.

Table 1 Application of process-level data against evaluation indicators
Full size table

Measures

Baseline evaluation surveys captured teacher, parent and child socio-demographic characteristics.

Process evaluation indicators

Quantitative data provided information on environmental context (school characteristics), teacher delivery context (teaching grades), and attendance at TransformUs training. Surveys at T3 and T4 assessed teacher fidelity, dose delivered, appropriateness, satisfaction and sustainability, and parent and child satisfaction and dose received (example survey items in Table 1 and an example teacher survey is shown in Additional File 3). Lesson evaluations in year one (2010) and year two (2011) were used to assess teacher fidelity and adaptations to the key messages. Each key message included three components: a class discussion, class activity and individual child worksheet. For each of the nine key messages, teachers were asked to record in lesson evaluations any barriers and facilitators to implementation (free text response), and how they delivered the key messages (response options: ‘I used the materials as they are’, ‘I modified the materials’ or ‘I created new materials’). Fidelity to the intervention protocol was assessed based on the extent that teachers reported delivery of the key messages using the ‘materials as they are’. Qualitative open-ended survey responses contributed to assessing barriers and facilitators to intervention adoption, delivery, sustainability and integration into school policy (example survey items in Table 1). Survey questions were refined between the T3 and T4 data collections to improve comprehension and reduce participant burden. Some questions were reworded or removed as a result.

Physical activity, sedentary time and sedentary breaks

Children wore a hip-mounted ActiGraph GT3X (Pensacola, FL) accelerometer for 8 consecutive days during waking hours (excluding water-based activities) to obtain behaviour data. Time (mins) spent in sedentary, light-intensity physical activity (LPA) and MVPA, and frequency of breaks in sedentary time, on an average weekday, average weekend day and average day were calculated. Accelerometry data were considered valid if there was a minimum of 4 days, including one weekend day, with at least 8 h per day of wear time (or at least 50% wear time within periods of the day i.e. class time) [52]. A 15-second epoch length was used and sedentary time was defined as < 25 counts per 15-second epoch [53]. The number of breaks in sedentary time was defined as the frequency of occasions that the accelerometer data exceeded 100 counts.min-1 [54]. The Freedson age-adjusted cutpoints [55] were used to calculate time spent in light- (1.5–3.9 METs), moderate- (4.0–5.9 METs) and vigorous-intensity (≥6.0 METs) physical activity. Average wear time for valid periods and valid days were calculated and non-wear time was defined as a period of ≥20 minutes of consecutive zeros [56]. A detailed description of the accelerometry measurement protocol and data management has been previously published [25].

Data analyses

As the study purpose was to evaluate process-related factors amongst participants delivering or receiving TransformUs, no control group comparisons were made. Comparisons between intervention groups (groups PA-I, SB-I and SB + PA-I) were examined separately at T3 and T4, using linear mixed-effects models for continuous variables and logistic mixed-effects models for dichotomous variables, with random intercepts for school, and adjusted for school-level SES (mid/high vs. low). In the models, study intervention arm was the independent variable and the different process evaluation indicators (i.e., dose delivered) were the dependent variables. Due to small cell sizes for binary teacher outcomes, inferential models were not conducted, while group comparisons of continuous teacher outcomes were made using unadjusted linear regression models. Bias-corrected bootstrapping (1000 resamples) was used to produce robust standard errors. An implementation score for each teacher was generated by summing their scores for fidelity and dose delivered, corresponding to the number of intervention components within each intervention group. The maximum score attainable (indicating complete program delivery), varied by group: PA-I (max score 6), SB-I (max score 4) and SB + PA-I (max score 8). For data to be considered valid, responses to a minimum of 5 of the 6 components was required for the PA-I group and a minimum of 7 of the 8 items for the SB + PA-I group. Due to the small number of intervention components in the SB-I group (n = 4), complete data for all 4 components was required. In instances of missing data for one component in the PA-I and SB + PA-I groups, the mean score of the remaining components was imputed. At T3, 46 of the 60 teachers (77%) and at T4, 28 of the 92 teachers (30%) provided valid data.

Teachers were grouped by level of implementation based on the proportion of the entire intervention delivered (dose delivered and fidelity). Consistent with previous research [57, 58], implementation levels corresponded to: (i) ‘low’ (< 33% of the entire intervention delivered); (ii) ‘moderate’ (33–67% delivered); and (iii) ‘high’ (> 67% delivered). To examine associations between teacher implementation level and child behavioural outcomes, linear mixed models with random intercepts for school were fitted. Due to small samples, teacher implementation data were pooled across intervention groups, to calculate a standardised mean implementation score. Analyses were adjusted for potential confounders of school socioeconomic position (SEP), average accelerometry wear time, baseline values of the outcome variables and intervention group. In the models where sedentary breaks were included as the outcome variable, analyses were also adjusted for average sedentary time. All statistical analyses were conducted using Stata (SEv17) and statistical significance was set at p < 0.05.

Qualitative survey data, from open-ended survey responses and lesson evaluations, were entered into NVivo10 and analysed thematically. Thematic analysis was conducted by HK, who has expertise in qualitative analysis, and involved preliminary data familiarisation, coding and tabulation of raw themes. Raw themes were then grouped into major themes based on patterns of emergence and overlapping relevance. Coding and theme development was initially deductive, based on the study aims and underlying theory, followed by an inductive process directed by the content of the data [59]. Barriers and facilitators were grouped according to their level of impact (organisational or individual).

Results

Across the 15 intervention schools recruited, 1134 children were eligible to take part in the trial. Of these, 443 (39%) provided consent to participate in at least one evaluation assessment. Survey and valid accelerometry data were obtained from 253 children in the PA-I (n = 97), SB-I (n = 81) and SB + PA-I (n = 75) groups, and survey consent was obtained from 163 teachers and 1141 parents. Participant response rates at T3 and T4 are shown in an a (see Additional File 4).

Sample characteristics

Baseline surveys were completed by 52% (n = 85) of teachers, 92% (n = 407) of children and 29% (n = 331) of parents in the intervention groups. The mean (SD) age of teachers was 37 (12.5) years, the majority were female (83%) and employed full time (93%), and on average had taught for 12.4 (12.1) years. The mean age of children was 8.2 (0.5) years and girls represented 58% of the sample. Most parents (88%) who completed a survey were female and the mean age was 39.4 (5.1) years. Among responding teachers at T3, 20% reported having attended TransformUs training in 2010, 30% in 2011, and 25% attended the mid-morning (problem-solving) tea in 2011.

Fidelity and dose delivered (Teachers)

Key messages and active/standing homework (PA-I, SB-I, SB + PA-I)

Tables 2 and 3 present teacher fidelity and dose delivered by intervention group. Responding teachers delivered approximately 70% of the nine key messages at T3 (M = 6.3, SD = 2.9), reducing to 48% by T4 (M = 4.4, SD = 3.5). The dose of key message delivery was statistically significantly greater in the PA-I group compared to the SB-I and SB + PA-I group at T3 and T4 (Table 3). Only 31% of teachers delivered all nine key messages at T3, which reduced over time, and very few teachers delivered all nine key messages without any adaptation in either 2010 (15%) or 2011 (9%) (i.e., delivered one or two rather than all three key message components) (Table 2). Fidelity of key message delivery was low across all intervention groups, although consistently higher in the PA-I group; significant at T3. The majority (65%) of teachers set active/standing homework at T3 (fidelity). Homework in general was set once or more per week by the majority (54%) of teachers, whereas only 19% set active/standing homework once or more per week. Fidelity to delivery of both components declined over time. The frequency (dose) of active/standing homework delivery was low across all groups, although was consistently higher in the PA-I group over time (Table 3).

Table 2 Teacher intervention fidelity and dose delivered (dichotomous outcomes) at T3 and T4
Full size table
Table 3 Teacher between-group comparison of teacher intervention dose delivered (continuous outcomes) at T3 and T4
Full size table

Standing lessons and active breaks (SB-I, SB + PA-I)

At T3, 30% of teachers delivered one standing lesson per day and 56% delivered active breaks (Table 2). Teacher fidelity to standing lesson and active break delivery increased over time, remaining consistently higher in the SB + PA-I group. The dose of active break delivery was consistent across both groups at T4; however, the teacher reported duration was significantly greater in the SB + PA-I group, compared to the SB-I group, at T3 (Table 3). The dose of standing lesson and active break delivery was not measured at T3.

Physical activity/sports equipment, line markings and promotional signage use (PA-I, SB + PA-I)

The frequency (dose) of weekly physical activity/sports equipment, signage and line marking use was consistent across the PA-I and SB + PA-I groups (Table 2). Almost all teachers made physical activity/sports equipment available during recess and lunch, and used physical activity/sports equipment in class. Over half of all teachers reported using physical activity/sports equipment in class once or more per week, and this increased over time (Table 2). Teacher fidelity for physical activity/sports equipment use in class remained consistently greater in the SB + PA-I group over time (Table 2). Teachers were only encouraged to use line markings during recess and lunch, yet reported using line markings during class time and this increased over time. Over half of all teachers consistently used line markings once or more per week in class, particularly in the SB + PA-I group (Table 2). Teacher fidelity to promotional signage use reduced by over half of all teachers between T3 and T4.

Fidelity and dose delivered (Teachers Qualitative data)

Qualitative data from open-ended questions in teacher surveys and lesson evaluations captured facilitators and barriers to intervention delivery and adherence (dose and fidelity) (Table 4). Themes were broadly consistent at T3 and T4 and across intervention groups, although response rates to the open-ended questions were low amongst all intervention groups at both time points.

Table 4 Barriers and facilitators to teacher intervention delivery, sustained implementation and integration into school policy
Full size table

Facilitators to program delivery

At T3, the most frequently reported theme was children’s enjoyment of TransformUs as a facilitator to implementation. Teachers described children’s pleasure in participating in the program’s physically active elements and the children’s ability to refocus quickly after active breaks. This included an awareness and understanding of program values and benefits as influencing their decision to integrate the program:

“Has many benefits and makes the classroom much more fun and I think, even more relaxed!” [T3, Grade 3/4 Teacher, SB + PA-I]

Implementation was also supported by the teachers’ freedom and flexibility to deliver aspects of the program when required, although identified only at T4. At an organisational level, successful integration of the program into, and expansion of, existing teaching practices was the most frequently reported enabling factor for implementation. One teacher referred to a supporting school ethos and infrastructure to facilitate program implementation, and one other described the program as expanding their existing teaching practices:

“The concept of more active participation, lessons, standing, etc has been successfully integrated. TransformUs made us think beyond what we were already doing” [T4, Grade 3/4 Teacher, PA-I]

Barriers to program delivery

At both time points, the most consistently reported barrier to program delivery at an individual level was time constraints. Teachers referred to competing demands and conflicts between timetable content and delivery. Some teachers associated active breaks and standing lessons with disruptions or distractions to the class. Specifically, these were described in reference to differences in children’s learning styles and classroom behaviour, and the potential inappropriateness of these components among children with behavioural difficulties or additional learning needs. The third most commonly reported theme by teachers, was forgetting to implement. This was reported as a barrier to key message, standing lesson and active break delivery:

“At times we forgot to plan this [standing lessons] into our program” [T3, Grade 3/4Teacher, SB + PA-I]

Organisational level barriers included the absence of ‘top-down’ school promotion, resulting in some teachers lacking an awareness of participation, in particular at T4. This was attributed to insufficient leadership and promotion of the program at the school level, including weaknesses in communication leading to program discontinuation. A ‘crowded curriculum’ was the second most common theme, followed by classroom infrastructure (such as room size), which hindered successful delivery of standing lessons for some teachers. At both time points, a small number of teachers reported either having a ‘no homework’ policy or identified a lack of parental support as a barrier to active/standing homework delivery:

“Parents didn’t want it [active/standing homework], they wanted homework that develops skills in coming home, planning time sitting and concentrating, as this will prepare them for high school etc.” [T3, Grade 4 Teacher, SB + PA-I]

Dose received (Parents and Children)

Parents

Key messages and newsletters (PA-I, SB-I, SB + PA-I)

Tables 5 and 6 present dose received by parents and children. At T3, parents reported receiving on average 3.42 (SD 1.9) of the nine newsletters and reported trialling an average 2.27 (SD 1.9) of the nine key messages at home. Parents in the SB-I group received a significantly greater number of newsletters compared to the PA-I group at T3 (Table 6). At T4, 62% of parents reported their child mentioning more than one key message at home (Table 5).

Table 5 Parent and child dose received (dichotomous outcomes) at T3 and T4
Full size table
Table 6 Parent and child between-group comparison of intervention dose received (continuous outcomes) at T3 and T4
Full size table

Children

Key messages (PA-I, SB-I, SB + PA-I)

Children in the PA-I group recalled being taught significantly more key messages than any other group at T3 and T4 (Table 6).

Standing lessons and active breaks (SB-I, SB + PA-I)

More than half of all children at both time points received a standing lesson at least once per week, and this was significantly greater in the SB + PA-I compared with the SB-I group at T3 (Table 5). Consistent across intervention groups, children’s reported exposure to teacher strategies to reduce sitting was consistently greater at T3 than T4. Some 73% of children at T3 reported that their teacher ‘sometimes’ or ‘always’ did not let them sit down for too long, and over half reported doing ‘lots of class activities standing up’ and that the teacher gets them to ‘move around a lot in class’ (Table 5). Most children (66%) reported having an active break at least once per week at T3, which reduced over time.

Encouragement of physical activity, promotional signage and physical activity/sports equipment (PA-I, SB + PA-I)

Consistent across intervention groups, 55% of the children noticed that their teacher encouraged physical activity at T3, reducing to 48% at T4. Thirty-six percent of children noticed more signs promoting physical activity in the school grounds at T3 and this increased to 41% at T4 (Table 5). Almost all children at both time points reported being allowed to use physical activity/sports equipment during recess and lunch breaks (Table 5).

Active/standing homework (PA-I, SB-I, SB + PA-I)

Active homework was reported by half of all children a T3. This proportion decreased at T4, but was significantly higher in the PA-I group, compared to the other two groups, at both time points (Table 5). At T3, the PA-I group also had higher odds of completing the active homework than the other two groups.

Appropriateness (Teachers)

Table 7 presents teacher perceptions of the appropriateness of the program. Overall, the program was reported as appropriate for the classroom setting. At T3, the majority of teachers reported that low amounts of preparation were required to implement the key messages (76%), standing lessons (72%), active breaks (82%) and active/standing homework (88%). The key messages, standing lessons and active breaks were perceived as easy to deliver. The proportion of teachers reporting high levels of appropriateness for standing lessons (amount of preparation required for delivery, ease of implementation, feasible integration into their current learning theme and feasible integration into the classroom) increased over time. The proportion of teachers reporting high levels of appropriateness for active breaks also increased over time (Table 7). The item scoring least favourably was integration of the key messages into the current learning theme. At T3, only 34% of teachers reported that key messages could be easily integrated, which was consistent across all intervention groups.

Table 7 Teacher-reported appropriateness and satisfaction with TransformUs components (dichotomous outcomes) at T3 and T4
Full size table

Satisfaction (Teachers, Parents and Children)

Table 7 also presents teacher reported satisfaction with the program, and Table 8 presents parent and child reported satisfaction with the program. TransformUs was positively received by teachers, parents and children. In terms of teacher perceived impact of the program, at T3 and T4, teachers perceived that children had greater concentration after an active break (71 and 85%, respectively), and after a standing lesson (52 and 78%, respectively). Teachers also perceived that children had greater time-on-task after an active break (68% at T3; 82% at T4), and after a standing lesson (48% at T3; 79% at T4). At T3, 86% of teachers planned to continue TransformUs strategies and 77% stated they would recommend TransformUs to other teachers (Table 7). The majority of parents supported their child’s continued participation in the program, which increased over time, and 69% perceived that TransformUs benefitted their child’s learning at T4 (Table 8). At both time points, the majority of children enjoyed standing lessons, active breaks and active/standing homework, indicating that they would like more standing lessons and active breaks in the future (Table 8). At T3, a quarter of the children reported that it was easier to listen to the teacher/complete work when standing, and over half reported it was easier to listen/complete work after an active break (Table 8).

Table 8 Parent- and child-reported satisfaction with TransformUs components (dichotomous outcomes) at T3 and T4
Full size table

Sustainability (Teachers Qualitative Data)

Qualitative data from T3 and T4 teacher surveys and lesson evaluations captured program sustainability and integration of standing lessons and active breaks into school policy. Themes were broadly consistent at T3 and T4 and across intervention groups (Table 4).

Facilitators to program sustainability

At T3, teacher awareness of program benefits to the children and teaching practice, was the dominant theme associated with program sustainability:

“We are all used to it and it doesn’t impact on the classroom time etc but helps to transition to new lessons or refocus on a task! We all need the break!” [T3, Grade 3/4 Teacher, SB + PA-I]

This was followed by children’s enjoyment, which encouraged teachers to continue implementation. At T4, five teachers inferred that additional ideas and program materials to support delivery would facilitate sustained implementation. At the organisational level, effective program integration into existing teaching practices was the key theme associated with sustainability at the school level. Specifically, teachers referred to the importance of program integration into other school and curriculum areas, including regular professional development and demonstrations of implementation. At T4 in particular, school leadership and management support was perceived as integral to long-term integration of the program as a whole-of-school approach:

“School leadership should ensure that all students across the school are involved in active learning tasks every day!” [T4, Grade 6 Teacher, PA-I]

One teacher at T3, and two teachers at T4, referred to prioritising and raising the profile of physical activity within schools, as integral to program sustainability.

Facilitators to integration of standing lessons and active breaks into school policy

Integrating and prioritising inclusion of TransformUs into school and curriculum planning meetings, including prioritising the program within a supportive planning strategy, was identified as the main facilitators to school policy integration at an organisational level:

“Needs [standing lessons] to be integrated in the curriculum from the beginning of the year and done all the way through the school.” [T4, Grade 3 Teacher, SB-I]

At the individual level, promoting and raising awareness of the program values and benefits among teachers was the primary theme associated with school policy integration. This included re-framing TransformUs outcomes to include health and educational gains for the School Council. Incorporating the program into teacher training and professional development sessions was described as facilitating whole-of-school adoption. Six teachers (one at T3 and five at T4) described school reinforcement of teacher commitment to, and staff support for program delivery, was necessary for policy integration. Specifically this included a cultural and environmental shift in classroom management:

“Again, all PLT [Professional Learning Team] members must attempt to integrate it [active breaks] into the culture/everyday environment of the classroom.” [T4, Teaching Grade unknown, SB + PA-I]

Barriers to program sustainability

At an organisational level, time was the most frequently occurring barrier to long-term program implementation, followed by the perception that the nine key messages were insufficiently integrated across the broader school curriculum. One teacher referred to a lack of consistent reinforcement and awareness of the program at the organisational level, which inhibited continued delivery:

“Great to have more physical activity, however unless it’s discussed and encouraged regularly, it gets put on the back burner. So much happens each day that programs get pushed back. Unfortunately this happened with TransformUs at [name of school].” [T4, Grade 5 Teacher, SB-I]

Far fewer barriers were reported at the individual teacher level. Perceived demands of complete program delivery and perceived challenges of effective integration into existing practices hindered sustainability amongst nine teachers at T4, and one teacher at T4.

Barriers to integration of standing lessons and active breaks into school policy

The most frequently occurring barrier to policy integration, was the perception that a mandated policy relating to TransformUs was not required. Reasons included the practical challenges with classroom infrastructure (e.g. size), time constraints with the crowded curriculum, and difficulties gaining whole of school and committee support for a new policy:

“For teachers it [active breaks] would be easy. As for policy, getting things made into “school policy“ is difficult as it goes through council.” [T4, Grade 5/6 Teacher, PA-I]

Associations between implementation level and outcomes

At T3, 48% of teachers were significantly more likely to report moderate implementation (delivering approximately two-thirds of the entire intervention) than low (26%, p = 0.03) and high (26%, p = 0.03) implementation (Additional File 5). At T4, 46% of teachers delivered approximately one-third of the entire intervention (low level of implementation). At T4, teachers were significantly less likely to report a level of implementation classified as high (11%), than both low (46%, p = 0.003) and moderate (43%, p = 0.007) implementation (Additional File 5). There were no statistically significant associations between intervention implementation level and children’s physical activity and sedentary time on an average day, average weekday or average weekend day (Table 9).

Table 9 Linear mixed models of associations between teacher implementation score (continuous outcomes) and child physical activity and sedentary behaviour outcomes (combined intervention groups)
Full size table

Discussion

This process evaluation provides unique insights into factors which may have influenced the implementation and sustained delivery of TransformUs; a school-based intervention to increase children’s physical activity and reduce sedentary behaviour. The results indicate that on the whole, TransformUs was appropriate for the school setting, and teachers, parents and children were satisfied with the strategies involved. The evaluation involved approximately half of all teachers and almost all children recruited as part of the RCT. There was, however, substantial variability in implementation of the program among responding teachers, which varied both by intervention group and intervention component. For example, the environmental aspects of TransformUs were delivered more consistently and frequently compared to the curriculum-related components.

Dose and fidelity of some of the curriculum-related aspects of the program (i.e., delivery of key messages and active/standing homework) declined over time, yet fidelity to pedagogical aspects of the program (i.e., integration of standing lessons and active breaks in the classroom) increased. Consistently, the availability and use of environmental strategies (physical activity/sports equipment in the classroom and playground line markings) remained high throughout the trial. The substantial variability in teacher dose and fidelity in relation to the TransformUs protocol is perhaps not unexpected, given the flexibility and autonomy of teaching styles and curriculum delivery that exists within schools. These differences may also simply reflect the necessary conditions for teachers to deliver TransformUs. School-based intervention research suggests that variations in teacher implementation are likely [60], and dose and fidelity may change over time [60, 61]. Without adaptation, interventions are more likely to face resistance by the user and require active engagement for delivery [62].

The majority of responding teachers delivered approximately two-thirds of the entire intervention over the course of the trial. This falls within the range of other school-based interventions reporting teacher delivery of between half [63] and three-quarters [64] of the intervention. Level of implementation was highest amongst teachers in the combined SB + PA-I group. Teachers in this group reported greater fidelity to the pedagogical aspects (standing lesson and active break delivery) and increased dosage of environmental components. The SB + PA-I group included the greatest variety of intervention components for delivery. The increased choice of strategies to implement may have enhanced the schools’ and teachers’ autonomy for delivery [65], facilitating implementation overall. Teacher implementation level was, however, unrelated to the time children spent in physical activity or sedentary behaviour. In general, higher levels of implementation are linked to improved behavioural outcomes [60, 66], even though positive outcomes can still be achieved when an intervention is not necessarily delivered as intended [60, 63]. There are several potential interpretations for this finding. It could suggest that there is no minimum threshold for implementation of TransformUs strategies for behavioural outcomes to occur; any changes or promotion of movement in schools is beneficial for child physical activity. Alternatively, as we calculated level of implementation based only on aspects of TransformUs implementation that related to teacher level delivery (i.e., dose delivered by teachers), we do not know the extent that factors may have influenced intervention impact. For example, effective implementation also includes factors at an organisational level, such as Principal buy-in and support for implementation, yet we were not able to account for this as part of the implementation score and thus do not know if factors at this level were also associated with intervention impact. Lastly, due to the small sample of responding teachers, we were unable to track individual changes in teachers’ implementation of TransformUs over time and how this may relate to child outcomes. It is unknown therefore if the lack of associations between implementation levels and outcomes resulted from the lack of sensitivity of the standardised score. Nonetheless, since variations in teacher dose and fidelity were unrelated to child behavioural outcomes in this study, yet the intervention has demonstrated positive effects on children’s’ MVPA and sedentary time [29, 30], the potential translatability of TransformUs into a practice context may be increased.

The TransformUs key messages and active/standing homework topics were designed in accordance with the existing curriculum, and adaptation was encouraged to suit planned lessons. However, the notable differences in teacher implementation of these curriculum-related components, compared to the pedagogical and environmental components, raises questions about the feasibility of uniformly integrating curriculum-based strategies within the school setting. Previous intervention research in schools has highlighted that intervention fidelity can be compromised due to challenges that teachers face adapting intervention lessons for contextual relevance [67]. Evaluation of the CATCH-ON study in the United States showed that the classroom curriculum and family-based components had the lowest levels of program institutionalisation [68]. Since adaptability is one element of sustainability, the goal for intervention evaluation may be to ascertain what degree of fidelity is needed to achieve outcomes [61], and how much variation is necessary to align with the delivery context [69]. Consistent with previous interventions targeting school settings [70], and a systematic review of factors required for sustainable implementation in schools [71]; the need for school Principals and senior school leaders to champion and embed initiatives within routine school practices, was integral to TransformUs uptake, delivery and sustainability. Innovative ways of building schools’ capacity to support teachers to align and integrate curriculum modifications into their existing teaching practices, may improve the feasibility and delivery of programs such as TransformUs.

Overall, TransformUs was perceived as being appropriate according to teachers, based on low amounts of preparation required to implement, ease of delivery and integration into teachers’ current learning themes. Qualitative data also highlighted that effective integration of TransformUs into existing practices was the main facilitator to consistent delivery and sustained implementation, and would facilitate successful integration of all aspects of the program into school policy. Compatibility of an intervention to the context [32], and effective integration into existing organisational routines, including setting appropriateness, are well-documented precursors to more effective implementation [46, 62, 72]. Understanding the delivery context can help explain the conditions necessary for optimal implementation, and the intervention’s generalisability to other settings and the reasons why variances in fidelity and adaptation may occur [32].

Variations in program implementation over time may also reflect differences in teacher exposure to training and implementation support between baseline and T4. During intervention years one (baseline) and two (corresponding to the T3 data collection), TransformUs teachers received a half-day training session and a follow-up mid-year morning tea to discuss and solve challenges to implementation. In the final year of the trial (corresponding to T4 data collection), teachers received training materials at the beginning of the year, but no face-to-face training. In addition, some teachers may have been exposed to training during only one school year, whereas others may have been exposed to training over multiple years. This would have depended on which grades they taught over the three school years the intervention was implemented. Increased participation in training has been associated with improved implementation of a school-based physical activity intervention [73], and a supporting infrastructure that builds capacity in individuals is central to implementation [46, 74].

Consistent with previous literature on factors influencing implementation of school-based physical activity interventions [75], qualitative data suggested that teacher awareness of TransformUs values and benefits, and children’s enjoyment were related to implementation. Perceived barriers included a lack of school-level awareness and promotion, organisational level endorsement and whole-of-school support, time constraints and a lack of program integration into existing school practices. Organisational climate and level of institutionalisation have previously been linked to the degree of implementation of school-based interventions [73]. Consistent with this literature, TransformUs teachers identified whole-of-school leadership, prioritisation, support and commitment to delivery, as facilitators to school policy integration. If interventions are to have a more substantial impact on children’s behaviours, implementation strategies need to target changes in the organisational infrastructure and culture of schools, in equal measure to changing observable behaviours.

In terms of the dose received among children and parents, consistently across all intervention components, reported receipt and/or awareness of TransformUs was lower at T4 than T3, although the dose received among children varied by intervention component. Consistent with previous research exploring dose received of a school-based obesity prevention intervention [76], potentially, some TransformUs components may have been easier to adopt by the children than others, explaining the differences in dose received. For example, some TransformUs elements were delivered as a whole-of-class activity increasing the likelihood of participation (e.g., taking part in a standing lesson), compared to other program elements that required individual choice to adopt (e.g., using physical activity/sports equipment during recess and lunchtime) or environmental awareness (e.g., seeing the promotional signage). Nonetheless, whilst teachers reported increased implementation of the pedagogical aspects of TransformUs over time, children’s recall of these aspects reduced. There is the potential that changes in children’s recall may reflect that the classroom components increasingly became part of routine practice and thus less ‘novel’ than when implemented in earlier phases of the trial. Previous studies of school-based interventions have suggested that students’ characteristics, such as their attitudes, motivation and engagement towards an intervention, can influence dose received [75, 77, 78], due to the extent that they participate. Whilst TransformUs was positively received by parents and children, future studies of TransformUs could explore any relationships between child and parent satisfaction and reported dose received.

A core aim of this evaluation was to understand teacher-led delivery of TransformUs and whether this was associated with outcomes. Teacher ‘implementation level’ was based on teacher adherence to intended delivery of TransformUs (fidelity) and the proportion and frequency of components implemented (dose delivered). This is consistent with previous conceptualisations of implementation in process evaluation [47], and studies which have associated implementation level with outcomes [57]. There are, however, many multi-level factors at the individual, organisational and macro level (such as school culture and characteristics) which are associated with effective implementation of interventions in the school setting [35]. Level of implementation can also include the optimal ‘dose received’ by participants [79], in addition to primarily dose delivered that was used in this study. Future research which explores the interaction of multi-level implementation factors on outcomes of school-based interventions, would improve our understanding of the conditions required for effective implementation and the extent that these factors mediate outcomes.

Strengths and limitations

Strengths of this study include assessment of numerous aspects of implementation at multiple time points to understand when optimal implementation may occur. Implementation processes are not static, rather they change to reflect the context within which implementation occurs. Implementation of some components of TransformUs (e.g., key messages and active/standing homework) was greater at T3 than T4, but the reverse was true for other components (e.g., integration of standing lessons and active breaks in the classroom). Had the changes in delivery over time not been captured, the conclusions may have over- or under-estimated the level of implementation delivered and received. The collection of both quantitative and qualitative survey data provided a more in-depth understanding of participants’ experiences. The qualitative data in particular provided insight into reasons for the variations in implementation; and have directly informed strategies to support delivery and sustainability of the program for future roll-out.

This study is not without limitations. Due to small numbers of teachers, models in Table 3 were not adjusted for school clustering or SES, and thus the results should be interpreted with caution. The small sample of teachers also meant exploration of the associations between implementation levels and outcomes was limited by use of a standardised score, and tracking of individual changes was not possible. The standardised implementation score may not have been sufficiently sensitive to detect any differences in implementation between groups. Strategies to maximise participant retention in the evaluation would have enabled more in-depth assessment of the relationship between implementation quality and program impact. Assessment of teacher implementation was based on teacher self-report of the previous school term, which is subject to recall bias. As only a sample of all participating teachers provided this self-report data, we do not know the level of implementation from non-responding teachers. Direct observations of implementation dose and fidelity may have improved accuracy of implementation assessment [67]. Lastly, our assessment of parent dose received included the number of program newsletters parents reported receiving. We were unable to assess whether parents read or acted on recommendations in these newsletters. This is an additional, important layer of assessing intervention and implementation effectiveness of programs such as TransformUs and others. Future evaluations of interventions that wish to ascertain dose received, would benefit from adopting multiple different indicators for this construct.

Conclusions

This study demonstrated that intervention dose and fidelity increased over time, and that children’s enjoyment, senior school leadership and effective integration of interventions into school practices facilitated improved intervention delivery and sustainability. Teacher level of implementation and child behavioural outcomes were unrelated, suggesting intervention efficacy was achieved irrespective of implementation variability. The potential translatability of TransformUs into practice contexts may therefore be increased. Findings have informed the scale up of TransformUs across Victoria, Australia.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

C-:

Control group

CI:

Confidence interval

METs:

Metabolic equivalents

MVPA:

Moderate- to vigorous-intensity physical activity

PA:

Physical activity

PA-I:

Physical activity intervention group; SB

Sedentary behaviour

SB-I:

Sedentary behaviour intervention group

SB + PA-I:

Sedentary behaviour and physical activity intervention group

SEIFA:

Australian bureau of statistics socio-economic indexes for area

SEP:

Socioeconomic position

References

  1. Okely AD, Salmon J, Vella SA, Cliff D, Timperio A, Tremblay M, et al. A systematic review to update the australian physical activity guidelines for children and young people. In: Report prepared for the Australian Government Department of Health; 2012.

    Google Scholar 

  2. Sedentary Behaviour Research Network. Letter to the editor: standardized use of the terms “sedentary” and “sedentary behaviours”. Appl Physiol Nutr Metab. 2012;37(3):540–2.

    Article  Google Scholar 

  3. Ekelund U, Brage S, Froberg K, Harro M, Anderssen SA, Sardinha LB, et al. TV viewing and physical activity are independently associated with metabolic risk in children: the European Youth Heart Study. Plos Med. 2006;3(12):e488.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Danielsen YS, Juliusson PB, Nordhus IH, Kleiven M, Meltzer HM, Olsson SJ, et al. The relationship between life-style and cardio-metabolic risk indicators in children: the importance of screen time. Acta Paediatr. 2011;100(2):253–9.

    Article  CAS  PubMed  Google Scholar 

  5. Saunders TJ, Chaput JP, Tremblay MS. Sedentary behaviour as an emerging risk factor for cardiometabolic diseases in children and youth. Can J Diabetes. 2014;38(1):53–61.

    Article  PubMed  Google Scholar 

  6. van Ekris E, Altenburg TM, Singh AS, Proper KI, Heymans MW, Chinapaw MJM. An evidence-update on the prospective relationship between childhood sedentary behaviour and biomedical health indicators: a systematic review and meta-analysis. Obes Rev. 2016;17(9):833–49.

    Article  PubMed  Google Scholar 

  7. Chinapaw MJ, Proper KI, Brug J, van Mechelen W, Singh AS. Relationship between young peoples’ sedentary behaviour and biomedical health indicators: a systematic review of prospective studies. Obes Rev. 2011;12(7):e621–32.

    Article  CAS  PubMed  Google Scholar 

  8. World Health Organization. Global Action Plan on Physical Activity 2018–2030: more active people for a healthier world. Geneva: World Health Organization; 2018.

    Google Scholar 

  9. Hallal PC, Andersen LB, Bull FC, Guthold R, Haskell W, Ekelund U. Global physical activity levels: surveillance progress, pitfalls, and prospects. Lancet. 2012;380(9838):247–57.

    Article  PubMed  Google Scholar 

  10. National Physical Activity Plan Alliance. The 2016 United States Report Card on Physical Activity for Children and Youth 2016.

    Google Scholar 

  11. Fuller E, Mindell J, Prior G. Health Survey for England 2015. London: NHS Digital; 2016.

    Google Scholar 

  12. Department of Health. In: Do H, editor. Make your move - Sit less - Be active for life! Canberra: Australian Government; 2014.

    Google Scholar 

  13. Biddle S, Pearson N, Ross G, Braithwaite R. Tracking of sedentary behaviours of young people: a systematic review. Prev Med. 2010;51(5):345–51.

    Article  PubMed  Google Scholar 

  14. Telama R, Yang X, Viikari J, Valimaki I, Wanne O, Raitakari O. Physical activity from childhood to adulthood - A 21-year tracking study. Am J Prev Med. 2005;28(3):267–73.

    Article  PubMed  Google Scholar 

  15. Tremblay MS, LeBlanc AG, Kho ME, Saunders TJ, Larouche R, Colley RC, et al. Systematic review of sedentary behaviour and health indicators in school-aged children and youth. Int J Behav Nutr Phys Act. 2011;8(1):1–22.

    Article  Google Scholar 

  16. Langford R, Bonell C, Jones H, Pouliou T, Murphy S, Waters E, et al. The World Health Organization’s Health Promoting Schools framework: a Cochrane systematic review and meta-analysis. BMC Public Health. 2015;15:130.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Naylor P, McKay H. Prevention in the first place: schools a setting for action on physical inactivity. Br J Sports Med. 2009;43(1):10–3.

    Article  PubMed  Google Scholar 

  18. Kriemler S, Meyer U, Martin E, van Sluijs EMF, Andersen LB, Martin BW. Effect of school-based interventions on physical activity and fitness in children and adolescents: a review of reviews and systematic update. Br J Sports Med. 2011;45(11):923–30.

    Article  CAS  PubMed  Google Scholar 

  19. Ridgers ND, Stratton G, Fairclough SJ, Twisk JW. Long-term effects of a playground markings and physical structures on children's recess physical activity levels. Prev Med. 2007;44(5):393–7.

    Article  PubMed  Google Scholar 

  20. Hyndman BP, Benson AC, Ullah S, Telford A. Evaluating the effects of the Lunchtime Enjoyment Activity and Play (LEAP) school playground intervention on children’s quality of life, enjoyment and participation in physical activity. BMC Public Health. 2014;14(1):164.

    Article  PubMed  PubMed Central  Google Scholar 

  21. D’Haese S, Van Dyck D, De Bourdeaudhuij I, Cardon G. Effectiveness and feasibility of lowering playground density during recess to promote physical activity and decrease sedentary time at primary school. BMC Public Health. 2013;13(1):1–10.

    Article  Google Scholar 

  22. Hinckson E, Salmon J, Benden M, Clemes SA, Sudholz B, Barber SE, et al. Standing classrooms: research and lessons learned from around the world. Sports Med. 2016;46(7):977–87.

    Article  PubMed  Google Scholar 

  23. Aminian S, Hinckson EA, Stewart T. Modifying the classroom environment to increase standing and reduce sitting. Build Res Inform. 2015;43(5):631–45.

    Article  Google Scholar 

  24. Leung MM, Agaronov A, Grytsenko K, Yeh M-C. Intervening to reduce sedentary behaviors and childhood obesity among school-age youth: a systematic review of randomized trials. J Obes. 2012;2012:685430.

  25. Salmon J, Arundell L, Hume C, Brown H, Hesketh K, Dunstan DW, et al. A cluster-randomized controlled trial to reduce sedentary behavior and promote physical activity and health of 8-9 year olds: The Transform-Us! Study. BMC Public Health. 2011;11(1):759.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Bandura A. Social foundations of thought and action: a social cognitive theory. Englewood Cliffs: Prentice Hall; 1986.

    Google Scholar 

  27. Rachlin H. Judgement, decision, and choice: a cognitive/behavioral synthesis. New York: WH Freeman; 1989.

    Google Scholar 

  28. Bronfenbrenner U. Ecological Systems Theory. In: Vasta R, editor. Six theories of child development: revised formulations and current issues. London: Jessica Kingsley Publishers; 1992. p. 187–249.

    Google Scholar 

  29. Yildirim M, Arundell L, Cerin E, Carson V, Brown H, Crawford D, et al. What helps children to move more at school recess and lunchtime? Mid-intervention results from Transform-Us! cluster-randomised controlled trial. Br J Sports Med. 2014;48(3):271–7.

    Article  PubMed  Google Scholar 

  30. Carson V, Salmon J, Arundell L, Ridgers ND, Cerin E, Brown H, et al. Examination of mid-intervention mediating effects on objectively assessed sedentary time among children in the Transform-Us! cluster-randomized controlled trial. Int J Behav Nutr Phys Act. 2013;10:62.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Salmon J, Arundell L, Cerin E, Ridgers ND, Hesketh KD, Daly RM, et al. The Transform-Us! cluster RCT: 18- and 30-month effects on children’s physical activity, sedentary time and cardiometabolic risk markers. Under Review. 2021.

    Google Scholar 

  32. Moore G, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Linnan L, Steckler A. Process evaluations for public health interventions: an overview. In: Steckler A, Linnan L, editors. Process Evaluation for Public Health Interventions and Research. San Francisco: Jossey Bass; 2002.

    Google Scholar 

  34. Oakley A, Strange V, Bonell C, Allen E, Stephenson J. Process evaluation in randomised controlled trials of complex interventions. BMJ. 2006;332(7538):413–6.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Domitrovich CE, Bradshaw C, Poduska J, Hoagwood K, Buckley JA, Olin S, et al. Maximizing the implementation quality of evidence-based preventive interventions in schools: a conceptual framework. Adv Sch Ment Health Promot. 2008;1(3):6–28.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Saunders R, Ward D, Felton G, Dowda M, Pate R. Examining the link between program implementation and behavior outcomes in the lifestyle education for activity program (LEAP). Eval Program Plan. 2006;29(4):352–64.

    Article  Google Scholar 

  37. Sussman S, Valente T, Rohrbach L, Skara S, Pentz M. Translation in the health professions: converting science into action. Eval Health Prof. 2006;29(1):7–32.

    Article  PubMed  Google Scholar 

  38. Rogers E. Diffusion of Innovations. 5th ed. New York: Free Press; 2003.

    Google Scholar 

  39. Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL. A glossary for dissemination and implementation research in health. J Public Health Manag Pract. 2008;14(2):117–23.

    Article  PubMed  Google Scholar 

  40. Chambers D, Glasgow R, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8(117).

  41. Koorts H, Eakin E, Estabrooks P, Timperio A, Salmon J, Bauman A. Implementation and scale up of population physical activity interventions for clinical and community settings: the PRACTIS guide. Int J Behav Nutr Phys Act. 2018;15(1):51.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Pérez D, Van der Stuyft P, Zabala MC, Castro M, Lefèvre P. A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implement Sci. 2016;11(1):91.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Creswell J, Plano CV. Designing and conducting mixed methods research. Thousand Oaks: Sage Publications, Inc; 2007.

    Google Scholar 

  44. Steckler A, Linnan L. Process evaluation for public health interventions and research. San Francisco: Jossey-Bass; 2002.

    Google Scholar 

  45. Saunders RP, Evans MH, Joshi P. Developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract. 2005;6(2):134–47.

    Article  PubMed  Google Scholar 

  46. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3–4):327–50.

    Article  PubMed  Google Scholar 

  47. Baranowski T, Stables G. Process Evaluations of the 5-a-Day Projects. Health Educ Behav. 2000;27(2):157–66.

    Article  CAS  PubMed  Google Scholar 

  48. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38:65–76.

    Article  Google Scholar 

  49. Australian Bureau of Statistics. Information paper: an introduction to Socio-Economic Indexes for Areas (SEIFA) 2006. Canberra: Australian Bureau of Statistics; 2006.

    Google Scholar 

  50. Department of Education and Early Childhood Development. Summary Statistics Victorian Schools, February 2010. Victoria: Data, Outcomes and Evaluation Division; 2010.

    Google Scholar 

  51. Neil-Sztramko SE, Caldwell H, Dobbins M. School-based physical activity programs for promoting physical activity and fitness in children and adolescents aged 6 to 18. Cochrane Database Syst Rev. 2021;9(9):Cd007651.

    PubMed  Google Scholar 

  52. Arundell L, Ridgers ND, Veitch J, Salmon J, Hinkley T, Timperio A. 5-year changes in afterschool physical activity and sedentary behavior. Am J Prev Med. 2013;44(6):605–11.

    Article  PubMed  Google Scholar 

  53. Treuth MS, Schmitz K, Catellier DJ, McMurray RG, Murray DM, Almeida MJ, et al. Defining accelerometer thresholds for activity intensities in adolescent girls. Med Sci Sports Exerc. 2004;36(7):1259–66.

    PubMed  PubMed Central  Google Scholar 

  54. Gabel L, Ridgers ND, Della Gatta PA, Arundell L, Cerin E, Robinson S, et al. Associations of sedentary time patterns and TV viewing time with inflammatory and endothelial function biomarkers in children. Pediatr Obes. 2016;11(3):194–201.

    Article  CAS  PubMed  Google Scholar 

  55. Freedson P, Pober D, Janz KF. Calibration of accelerometer output for children. Med Sci Sports Exerc. 2005;37(11 Suppl):S523–30.

    Article  PubMed  Google Scholar 

  56. Cain KL, Sallis JF, Conway TL, Van Dyck D, Calhoon L. Using accelerometers in youth physical activity studies: a review of methods. J Phys Act Health. 2013;10(3):437–50.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Gray HL, Contento IR, Koch PA. Linking implementation process to intervention outcomes in a middle school obesity prevention curriculum, ‘Choice. Control Change’ Health Educ Res. 2015;30(2):248–61.

    Article  PubMed  Google Scholar 

  58. Lee H, Contento IR, Koch P. Using a systematic conceptual model for a process evaluation of a middle school obesity risk-reduction nutrition curriculum intervention: choice, control & change. J Nutr Educ Behav. 2013;45(2):126–36.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Joffe H, Yardley L. Content and Thematic Analysis. In: Marks D, Yardley L, editors. Research Methods for Clinical and Health Psychology: Sage Publications; 2004. p. 56–66.

    Google Scholar 

  60. Durlak JA. The importance of doing well in whatever you do: a commentary on the special section, “Implementation research in early childhood education”. Early Child Res Q. 2010;25(3):348–57.

    Article  Google Scholar 

  61. Wiltsey Stirman S, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. 2012;7:17.

    Article  PubMed  PubMed Central  Google Scholar 

  62. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15.

    Article  Google Scholar 

  63. Van Nassau F, Singh AS, Hoekstra T, Van Mechelen W, Brug J, Chinapaw M. Implemented or not implemented? Process evaluation of the school-based obesity prevention program DOiT and associations with program effectiveness. Health Educ Res. 2016;31(2):220–33.

    Article  PubMed  Google Scholar 

  64. Ward D, Saunders R, Felton G, Williams E, Epping J, Pate R. Implementation of a school environment intervention to increase physical activity in high school girls. Health Educ Res. 2006;21(6):896–910.

    Article  CAS  PubMed  Google Scholar 

  65. Christian D, Todd C, Davies H, Rance J, Stratton G, Rapport F, et al. Community led active schools programme (CLASP) exploring the implementation of health interventions in primary schools: headteachers’ perspectives. BMC Public Health. 2015;15:238.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:40.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Campbell R, Rawlins E, Wells S, Kipping RR, Chittleborough CR, Peters TJ, et al. Intervention fidelity in a school-based diet and physical activity intervention in the UK: active for life year 5. Int J Behav Nutr Phys Act. 2015;12:141.

    Article  PubMed  PubMed Central  Google Scholar 

  68. Lytle LA, Ward J, Nader PR, Pedersen S, Williston BJ. Maintenance of a health promotion program in elementary schools: results from the CATCH-ON study key informant interviews. Health Educ Behav. 2003;30(4):503–18.

    Article  PubMed  Google Scholar 

  69. Goulet M, Archambault I, Janosz M, Christenson SL. Evaluating the implementation of check & connect in various school settings: is intervention fidelity necessarily associated with positive outcomes? Eval Program Plan. 2018;68:34–46.

    Article  Google Scholar 

  70. Densley B, Calvert HG, Boedeker P, Turner L. Implementation of physical activity in us elementary schools: the role of administrative support, financial resources, and champions. Int J Environ Res Public Health. 2021;18(9):4476.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Herlitz L, MacIntyre H, Osborn T, Bonell C. The sustainability of public health interventions in schools: a systematic review. Implement Sci. 2020;15(1):4.

    Article  PubMed  PubMed Central  Google Scholar 

  72. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: a synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute: National Implementation Research Network; 2005.

    Google Scholar 

  73. Mâsse LC, McKay H, Valente M, Brant R, Naylor PJ. Physical activity implementation in schools: a 4-year follow-Up. Am J Prev Med. 2012;43(4):369–77.

    Article  PubMed  Google Scholar 

  74. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol. 2008;41(3–4):171–81.

    Article  PubMed  Google Scholar 

  75. Naylor PJ, Nettlefold L, Race D, Hoy C, Ashe MC, Wharf Higgins J, et al. Implementation of school based physical activity interventions: a systematic review. Prev Med. 2015;72:95–115.

    Article  PubMed  Google Scholar 

  76. Gebremariam MK, Arah OA, Bergh IH, Andersen LF, Bjelland M, Grydeland M, et al. Factors affecting the dose of intervention received and the participant satisfaction in a school-based obesity prevention intervention. Prev Med Rep. 2019;15:10090.

    Google Scholar 

  77. Berkel C, Mauricio AM, Schoenfelder E, Sandler IN. Putting the pieces together: an integrated model of program implementation. Prev Sci. 2011;12(1):23–33.

    Article  PubMed  PubMed Central  Google Scholar 

  78. Åvitsland A, Ohna SE, Dyrstad SM, Tjomsland HE, Lerum Ø, Leibinger E. The process evaluation of a school-based physical activity intervention: influencing factors and potential consequences of implementation. Health Educ. 2020;120(2):121–39.

    Article  Google Scholar 

  79. Ferm L, Rasmussen CDN, Jørgensen MB. Operationalizing a model to quantify implementation of a multi-component intervention in a stepped-wedge trial. Implement Sci. 2018;13(1):26.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

TransformUs was funded by a National Health and Medical Research Council of Australia Project Grant (No.533815); Diabetes Australia Research Trust. The funding body had no role in the design of the study and collection, analysis and interpretation of data, or in writing the manuscript. LA is supported by an Australian Research Council Discovery Early Career Researcher Award (DE220100847). NDR is supported by a Future Leader Fellowship from the National Heart Foundation of Australia (Award ID 101895). EC is supported by an Australian Research Council Future Fellowship (FT140100085). DWD is supported by a NHMRC Senior Research Fellowship (APP 1078360) and by the Victorian Government’s Operational Infrastructure Support Program. KDH is supported by an Australian Research Council Future Fellowship (FT130100637) & Honorary National Heart Foundation of Australia Future Leader Fellowship (100370). JS is supported by a NHMRC Leadership Level 2 Fellowship (APP 1176885).

Author information

Authors and Affiliations

  1. Institute for Physical Activity and Nutrition (IPAN), School of Exercise and Nutrition Sciences, Deakin University, Geelong, VIC, Australia

    Harriet Koorts, Anna Timperio, Gavin Abbott, Lauren Arundell, Nicola D. Ridgers, Robin M. Daly, David W. Dunstan, Kylie D. Hesketh & Jo Salmon

  2. Deakin University, 221 Burwood Highway, Burwood, VIC, 3125, Australia

    Harriet Koorts

  3. Alliance for Research in Exercise, Nutrition and Activity (ARENA), Allied Health and Human Performance, University of South Australia, Adelaide, South Australia

    Nicola D. Ridgers

  4. Mary McKillop Institute for Health Research, Australian Catholic University, Melbourne, Australia

    Ester Cerin

  5. Centre for Sport Research, Deakin University, Geelong, Australia

    Helen Brown

  6. Baker Heart and Diabetes Institute, Melbourne, Australia

    David W. Dunstan

  7. School of Public Health, University of Adelaide, Adelaide, SA, Australia

    Clare Hume

  8. Amsterdam UMC, Vrije Universiteit Amsterdam, Department of Public and Occupational Health, Amsterdam Public Health Research Institute, Amsterdam, the Netherlands

    Mai J. M. Chinapaw

  9. Deakin Health Economics, Institute for Health Transformation, Deakin University, Geelong, VIC, Australia

    Marj Moodie

Authors
  1. Harriet Koorts
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Timperio
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gavin Abbott
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lauren Arundell
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicola D. Ridgers
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ester Cerin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Helen Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Robin M. Daly
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. David W. Dunstan
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Clare Hume
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Mai J. M. Chinapaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Marj Moodie
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Kylie D. Hesketh
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jo Salmon
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HK led the design of the evaluation and writing of the manuscript. GA conducted the statistical analyses with HK. AT assisted with interpreting analyses and drafting the manuscript. JS conceptualised the study, assisted with interpreting analyses and drafting the manuscript. All authors critically reviewed and revised the manuscript for intellectual content, and approved the final version.

Corresponding author

Correspondence to Harriet Koorts.

Ethics declarations

Ethics approval and consent to participate

Approvals were obtained from the Deakin University Human Research Ethics Committee (EC 141–2009), the Victorian Department of Education and Early Childhood Development (2009_000344) and the Catholic Education Office (Project Number 1545). Active informed parental consent was required on behalf of themselves and their child. Parents could consent for their child to take part in any combination of the evaluation assessments as involvement in all assessments was not compulsory.

Consent for publication

Not applicable.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1. 

Intervention components and corresponding intervention arms. Theoretical basis of TransformUs intervention components and objectives. Alignment of TranformUs intervention components to the three intervention arms, and how the social cognitive theory, behavioral choice theory and ecological systems theory correspond to the intervention components.

Additional file 2. 

TransformUs TIDieR Checklist.

Additional file 3. 

TransformUs teacher survey SB + PA-I at T3. Copy of the TransformUs teacher survey for the SB + PA-I group at data collection time point 3.

Additional file 4.

Participant response rates at T3 and T4. Teacher, parent and child response rates to evaluation surveys at T3 and T4 and teacher response rates to lesson evaluations during 2010 and 2011.

Additional file 5. 

Distribution of teachers and children by level of implementation by intervention group. Distribution of teachers and children based on the percentage of the intervention delivered (low < 33%, moderate > 33 < 67% or high level > 67%of implementation).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Koorts, H., Timperio, A., Abbott, G. et al. Is level of implementation linked with intervention outcomes? Process evaluation of the TransformUs intervention to increase children’s physical activity and reduce sedentary behaviour. Int J Behav Nutr Phys Act 19, 122 (2022). https://doi.org/10.1186/s12966-022-01354-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12966-022-01354-5

Keywords

  • Implementation
  • process evaluation
  • physical activity
  • sedentary behaviour
  • school-based intervention

International Journal of Behavioral Nutrition and Physical Activity

ISSN: 1479-5868

Contact us