Optimising a multi-strategy implementation intervention to improve the delivery of a school physical activity policy at scale: findings from a randomised noninferiority trial

Abstract

Background

To maximise their potential health benefits, school-based physical activity policies need to be implemented at scale. This paper describes the third in a sequence of trials that sought to optimise an effective strategy (PACE) to assist schools’ implementation of a physical activity policy. Specifically, it aimed to determine the probability that a multi-strategy intervention adapted to reduce in-person contact (Adapted PACE) was “as good as” the original intervention (PACE) in increasing the weekly minutes of structured physical activity implemented by classroom teachers.

Methods

A noninferiority cluster randomised controlled trial was undertaken with 48 primary schools in New South Wales, Australia. Schools were randomised to receive PACE or a model with adaptations made to the delivery modes (Adapted PACE). Teachers’ scheduled minutes of weekly physical activity was assessed at baseline (Oct 2018-Feb 2019) and 12-month follow-up (Oct-Dec 2019). The noninferiority margin was set at − 16.4 minutes based on previous data and decision panel consensus. A linear mixed model analysed within a Bayesian framework was used to explore noninferiority between the two PACE models. A cost minimisation analysis was conducted from the health service provider perspective, using the Australian dollar (AUD).

Results

The posterior estimate for the between group difference at follow-up was − 2.3 minutes (95% credible interval = − 18.02, 14.45 minutes). There was an estimated 96% probability of Adapted PACE being considered noninferior (only 4% of the posterior samples crossed the noninferiority margin of − 16.4 minutes). That is, the minutes of physical activity implemented by teachers at Adapted PACE schools was not meaningfully less than the minutes of physical activity implemented by teachers at PACE schools. The mean total cost was AUD$25,375 (95% uncertainty interval = $21,499, $29,106) for PACE and AUD$16,421 (95% uncertainty interval = $13,974, $19,656) for Adapted PACE; an estimated reduction of AUD$373 (95% uncertainty interval = $173, $560) per school.

Conclusions

It is highly probable that Adapted PACE is noninferior to the original model. It is a cost-efficient alternative also likely to be a more suitable approach to supporting large scale implementation of school physical activity policies.

Trial registration

Retrospectively registered with the Australian New Zealand Clinical Trials Registry (ACTRN12619001229167).

Background

To achieve the World Health Organization’s (WHO) goal of a 15% relative reduction in the global prevalence of physical inactivity by 2030, they have recommended the implementation of school physical activity policies [1]. Accordingly, a number of countries including Australia [2], Canada [3], the United States (U.S.) [4], Denmark [5], China [6], and England [7] have policies mandating a minimum time that schools are to provide students with structured physical activity across the school week [2,3,4, 8]. Despite this, school physical activity policies are often poorly implemented [3, 4, 8,9,10,11,12,13]. For example, studies in Canadian elementary schools found less than half of teachers were implementing the mandatory provincial physical activity policy [14, 15], and an Australian study found that only 30% of teachers were scheduling the state-required 150 minutes of physical activity across the school week [16].

To achieve population-wide implementation, and maximise the public health benefits of school physical activity policies, support to assist schools in overcoming barriers to their implementation is required [17]. Comprehensive multi-strategy interventions have been found effective in improving school’s physical activity policy implementation. For example, recent randomised controlled trials (RCTs) in Australia tested strategies which have largely relied on in-person teacher training, technical assistance, educational outreach, and ongoing support [16, 18]. These strategies substantially improved policy implementation [16, 18] however they were resource-intensive, requiring significant investment (absolute cost) and workforce infrastructure to deliver [19]. While effective, such approaches may be cost prohibitive and unsuitable for use by government health and education agencies responsible for supporting large scale implementation of physical activity policies [20].

Optimisation is an emerging innovative concept in public health that may offer a means of improving the impact of physical activity policy implementation strategies by enhancing their capacity to be delivered at scale [21]. Optimisation is a data driven process that seeks to maximise the impact of implementation strategies, within resource constraints, through repeated testing and strategy refinement [21]. In the case of school-based physical activity policies, optimisation processes could be used to improve population-level impacts through adapting implementation strategies in a manner that they achieve similar effects but can be delivered on a greater scale. For example, via adaptations to the modality of delivery of implementation strategies [20] such that more expensive (e.g., in-person) modes of support are replaced with modes that afford greater reach at lower costs [22].

While we are not aware of optimisation processes being applied to any school physical activity policy or program implementation research, the approach has previously demonstrated potential merit in this setting [23, 24]. For example, the incremental cost-effectiveness ratio of a multi-strategy intervention to assist schools’ implementation of a nutrition policy were substantially reduced over a series of sequential RCTs by removing strategies or adapting their mode of delivery [23]. The ‘optimised’ implementation intervention was then adopted as part of a jurisdictional wide scale-up by health services in Australia [23, 24]. Given the potential benefits, we conducted a sequence of trials to optimise our multi-strategy intervention which supports schools’ implementation of a mandatory physical activity policy (Physically Active Children in Education [PACE] [16, 18, 25]) for delivery at scale. This paper describes the third in that sequence of trials, that aimed to determine if a multi-strategy intervention with mode of delivery adaptations (to reduce in-person contact; Adapted PACE), was “as good as” [26] (noninferior) the original intervention (which relied on more in-person delivery of implementation strategies; PACE), in increasing the weekly minutes of structured physical activity implemented by classroom teachers. We also explored whether adapting the more expensive modes of delivery maintained a meaningful effect but at a lower cost.

Methods

This study is reported in accordance with the CONSORT statements for noninferiority and equivalence randomised trials [27], cluster RCT [28], and the Standards for Reporting Implementation Studies (StaRI) statement [29]. Ethics approval was obtained from the Hunter New England Human Research Ethics Committee (2019/ETH12353), The University of Newcastle Human Research Ethics Committee (H-2008-0343), as well as the New South Wales (NSW) Department of Education (SERAP no.2017184) and the relevant Catholic School Offices. Assay sensitivity (i.e., plausible reason to believe that the comparator would be more effective than no intervention [30,31,32]) was ascertained through emulating the conditions (e.g., school type, participants, study region, delivery personnel and processes) of previous trials that established efficacy of PACE [16, 18, 25].

Study design and setting

A two-arm, cluster randomised controlled noninferiority trial was conducted in 48 schools from the Central Coast (CC; n = 8) and Hunter New England (HNE; n = 40) regions of NSW. Collectively these regions have approximately 572 primary schools [33] across a geographic area of more than 130,500 km². They consist of a socioeconomically and demographically diverse population of approximately 158,000 children aged 5–14 years [34].

Participants and recruitment

The sampling pool for this trial consisted of all government, independent and Catholic schools in the study region, excluding those that had participated (past or current) in another physical activity intervention (including a previous PACE trial), or catered exclusively for children with special needs. A study information package was emailed to school principals and those interested were asked to provide written informed consent. Teachers at consenting schools were provided with a brief overview of the study purpose, invited to participate, and informed that completion of an evaluation survey represented their consent to participate.

Randomisation and blinding

Following baseline data collection, an independent statistician used a computerised random number function to randomise schools (1:1) to receive the original PACE or the Adapted PACE multi-strategy intervention. Allocation was stratified by region (CC and HNE) and geographic (rural versus urban) location of each school given evidence of an association between these factors and the trial outcome [3]. Schools were not informed of their allocation. At follow-up, data collectors were not blind to group allocation.

Multi-strategy implementation interventions

Group 1 – PACE

PACE was designed to assist school’s implementation of the NSW Department of Education sport and physical activity policy which requires schools to schedule 150 minutes of physical activity across the school week [35]. This may include time in: physical education (PE), sport, or other structured activities such as energisers (3–5 minute classroom physical activity breaks) and active lessons (physical activity integrated into literacy lessons) [36]. Table 1 outlines the eight original PACE implementation strategies. PACE was designed using the Behaviour Change Wheel (BCW) [37] and Theoretical Domains Framework (TDF) [38], with strategies purposefully selected to overcome school-level barriers to the scheduling of physical activity. Barriers were identified following extensive formative research which included (i) literature reviews; (ii) interviews with primary school teachers and (iii) observations of teachers’ delivery of PE, sport and the school environment. Implementation strategies were selected following the recommended process described by Michie et al. [37], mapping identified barriers to the BCW and TDF. An advisory group consisting of implementation and health behaviour scientists, physical activity experts, teachers, principals and senior government policy makers then assessed each strategy for its Affordability, Practicality, Effectiveness and cost-effectiveness, Acceptability, Side-effects/safety and Equity (APEASE). This helped reduce the number of strategies and or make them relevant to the context. A complete description of intervention development is published elsewhere [18, 25].

Table 1 Description of PACE implementation strategies, including a comparison between PACE and Adapted PACE

Group 2 – ‘Adapted PACE’

The multi-strategy PACE intervention was adapted for delivery at scale by the local health service [39]. No PACE strategy was considered discretionary (non-core) as each was theoretically derived and evidence-informed to address specific barriers of the target behaviour. Drawing on evidence from studies of scaled-up health interventions in schools [20, 23, 40,41,42], it was hypothesised that adaptation of implementation delivery would not substantively reduce effects. In doing so, this would reduce costs to the health service to deliver the implementation support. Table 1 includes an overview of the mode of delivery adaptations that were made to several PACE implementation strategies (1a, 3 and 5) following a rigorous decision making process (Additional file 1). Additional file 2 provides a report of the adaptations in accordance with the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) [43]. The final Adapted PACE met the needs of stakeholders and fit within resource constraints of the local health service. Briefly, strategy 1 (centralised technical assistance and ongoing support) and strategy 2a (mandated support via engagement of school principal) delivered via email/telephone rather than in-person, and strategy 5 (educational outreach) delivered by an in-school champion rather than an external project officer.

Data Collection

Primary outcome: weekly minutes of structured physical activity implemented by classroom teachers

Robust noninferiority testing requires use of the same outcome measure that was used in a previous trial in which the comparator was proven effective. Consistent with the previous PACE trials [16, 18], teacher’s mean minutes of scheduled physical activity (total of PE, sport, energisers and/or active lessons) was measured using a daily log-book completed during a one-week period at baseline (October 2018–February 2019) and 12-month follow-up (October–December 2019). Data from log books were considered valid and included for analysis if teachers had recorded no more than 250 minutes of physical activity across the full five-day school week.

Secondary outcomes: weekly minutes of energisers, active lessons and PE implemented by classroom teachers

Secondary outcomes included the mean weekly minutes scheduled by teachers for each of PE, energisers and active lessons in the daily activity logbook. Sport was excluded as a secondary outcome as no significant effect has been established for sport in previous trials [16, 18].

Program delivery costs

Program implementation costs were calculated from the health service provider perspective in Australian dollars (AUD) using 2019 as the base-year value (AUD$1 = approximately $0.69 U.S. dollars). Costs incurred for research or program development were excluded as they were not associated with any difference between groups. Cost-related activities were extracted from project officer records, coded by strategy, and transferred into an economic spreadsheet. Relevant costs included (i) project officer salaries; (ii) in-school champion training workshop expenses such as venue hire (actual rates charged), catering and staff reimbursement to attend; and (iii) the development and distribution of PACE resources such as physical activity equipment packs and in-school champion manuals. Consumable costs were measured directly using project records.

School and participant characteristics

Details of each schools’ type, size (number of students), relative socio-economic position, and geolocation at baseline were retrieved online via the Australian Curriculum, Assessment and Reporting Authority (ACARA) [44]. Surveys of school principals and classroom teachers were used to collect demographic information (sex, age, employment status, years teaching, and if they were a specialist PE teacher) as well as details regarding the operational characteristics of schools, school participation in other physical activity programs, and PACE implementation activity. Any differences in baseline characteristics of those participants who completed primary outcome data and those who dropped out from the study were investigated. Items were sourced from previous surveys of school principals conducted by the research team which have achieved participation rates of between 70 and 96% [45].

Analysis

Noninferiority analysis

An intention to treat approach was applied, with all available valid data included in all analyses. Mixed models within a Bayesian framework were used to compare, between PACE and Adapted PACE, the minutes of implemented weekly physical activity in terms of total physical activity (primary outcome) and PE, energisers and active lessons (secondary outcomes). All models included a random level intercept for school, to account for clustering, and a fixed effect for experimental group. A linear mixed model was employed for the primary outcome, with a fixed effect included for the baseline value of the outcome. Missing baseline and follow-up data were imputed within this model, where missing outcome values were drawn from their posterior predictive distribution using the one-step approach [46]. Gamma-hurdle models were used to analyse the secondary outcomes, as a linear distribution did not fit these outcomes. Each of these models included a gamma distribution with a log link function to model the non-zero values, and a logit link function to model the zero values of the outcome. A longitudinal model rather than a baseline adjusted model was employed due to difficulties with model convergence when attempting to impute missing data. Therefore, the models for the secondary outcomes also included a fixed effect for time and a time-by-experimental group interaction term. All models applied an uninformative prior distribution, and were then replicated using an informative prior distribution based on knowledge of the distribution of the PACE strategy obtained from the previous effectiveness trial [16]. The No-U-Turn sampler (NUTS) was used to obtain estimates from the posterior distribution for all parameters, with a burn in period of 10,000, 10,000 post-burn in samples, and 4 chains. Convergence was assessed by inspecting the trace plots of the parameters, the reported effective sample size, and the Gelman-Rubin statistic. The mean posterior estimate and 95% credible intervals are reported for all outcomes. Furthermore, the posterior probability distribution was used to determine the probability that adapted PACE was noninferior to the original model for the primary and secondary outcomes, relative to their pre-specified noninferiority margin (see below). A variance ratio, which is the ratio of the variance of the posterior predictive distributions not conditioning on group level terms, to the variance of the posterior predictive distribution conditioning on all random effects (a Bayesian equivalent to ICC), was calculated for each model [47].

Sample size and noninferiority margin

This trial had a fixed sample size of 48 schools. The noninferiority margin (∆) for the primary trial outcome was set at − 16.4 minutes of scheduled physical activity. The ∆ is an “acceptable” between-group difference based on previous trials of the reference treatment combined with clinical judgement [48]. The ∆ for the current study was informed by usual practice for determining noninferiority margins in clinical trials (often set to maintain 50% of the lower confidence interval from a trial in which the comparator was proven effective) [30], as well as input from an expert decision panel as to what value would be considered beneficial. It was calculated as per recommendations [30, 32, 48]:

∆ = (1-‘acceptable proportion of the effect size retained’) × (lower bound confidence interval) ∆ = (1–0.50) × 32.8 ∆ = 16.4* *A favourable result in the outcome is expressed as an increase in minutes therefore, our noninferiority margin is set as − 16.4 minutes, meaning that for Adapted PACE to be deemed noninferior it should not be any lower than 16.4 minutes than PACE.

The ∆ decision panel, consisting of the PACE research team and stakeholders, implementation scientists, and experts in schools and physical activity, deemed that 50% was an acceptable proportion of the effect size retained. Panel discourse was informed by: reviews of similar studies with comparable outcomes [40, 49,50,51]; quantified attenuation in effect sizes associated with physical activity interventions adapted for scale-up [20]; evidence of the inverse dose-response relationship between physical activity and health outcomes [52], supporting an ‘anything is better than nothing’ notion [53] and; with consideration that full policy implementation may occur overtime, and in a nonlinear manner, as practices normalise within schools [54, 55]. In line with the CONSORT extension for reporting of randomised noninferiority trials [27], noninferiority was considered to be demonstrated if the difference between Adapted PACE and original PACE was no more than the ∆ of 16.4 minutes, based on examination of the 95% credible interval (as per the estimate provided by the posterior probability distribution [56, 57]. As a favourable result in this study’s outcome is an increase in minutes of total physical activity implemented, the noninferiority margin was set as − 16.4 minutes; with noninferiority of Adapted PACE determined if the lower 95% credible interval does not cross the ∆.. The same process informed ∆ for the secondary trial outcomes: − 8.25 minutes for energisers, − 1.58 minutes for active lessons and − 0.95 minutes for PE.

Cost-minimisation analysis

The total delivery cost and average cost per school overall and by strategy were calculated for both groups. The between-group difference in costs were calculated for overall total cost and average cost per school. Non-parametric bootstrapping analysis with 1000 replications was used to calculate uncertainty intervals to account for sampling variability.

Results

School and participant characteristics

Figure 1 provides a flow diagram of schools and participants through the study. A total of 48 schools with 446 classes were eligible and consented to participate. The characteristics of schools at baseline were similar between groups (Table 2). Of the remaining 48 consenting schools, 6 provided invalid data (i.e., surveys with ≥250 minutes scheduled across 5 days), leaving a total of 42 schools contributing valid data at 12-month follow-up, from a total of 104 teachers. The characteristics of teachers at baseline and follow-up were similar between groups (Table 3).

Fig. 1

Flowchart of school enrolment and data collection throughout the study

Table 2 School characteristics at baseline by experimental group

Table 3 Teacher characteristics at baseline and follow-up by experimental group

Primary outcome: weekly minutes of structured physical activity implemented by classroom teachers

Figure 2 displays the distribution of the posterior estimated differences between groups in teacher's total scheduled minutes of physical activity using the uninformative prior. The posterior estimate for the baseline adjusted difference was − 2.23 minutes; with a 95% probability that the true difference lies between − 18.02 and 14.45 minutes. Only 4% of the posterior samples crossed the ∆ of − 16.4 minutes, resulting in a 96% probability of Adapted PACE being considered noninferior to PACE (i.e., Adapted PACE was no more than 16.4 minutes less than the original model) (see Table 4). The results were identical when the informative prior distribution was used.

Fig. 2

Distribution of the posterior estimated differences in teacher's total scheduled minutes of physical activity between groups (uninformative prior)

Table 4 The mean weekly minutes of physical activity implemented by teachers at baseline and 12-month follow-up with intention-to-treat noninferiority analyses results

Secondary outcomes: weekly minutes of energisers, active lessons and PE implemented by classroom teachers

Table 4 provides the between group differences in the change from baseline to follow-up for the secondary outcomes. The following posterior estimates and 95% credible intervals for the secondary outcomes have been exponentiated to represent a percentage difference in the minutes of scheduled physical activity between groups. Compared with teachers who received the original PACE model, teachers who received Adapted PACE scheduled 4% higher mean minutes of energisers (exponent of posterior estimate = 1.04; 95% credible interval 0.78, 1.38; noninferiority probability = 99.6%), 8% lower mean minutes of PE (exponent of posterior estimate = 0.92; 95% credible interval 0.77, 1.12; noninferiority probability = 16.4%), and 1% lower mean minutes of active lessons (exponent of posterior estimate = 0.99; 95% credible interval 0.58, 1.75; noninferiority probability = 56%).

Program delivery costs

The cost of the 1–2 hour educational session for staff was unique to the original PACE model, resulting in a cost of $287 per school. Of the strategies received by both groups. The most costly was the full-day training workshop for in-school champions (mean = $484 per school) followed by the ongoing support provided by project officers (mean = $86 per school) and the physical activity equipment pack (mean = $85 per school).

From the health service provider perspective, the total cost to deliver PACE was $25,375 (95% Uncertainty Interval (UI) = $21,499, $29,106), equating to approximately $1057 per school (95% UI = $896, $1213). The total cost to deliver Adapted PACE was $16,421 (95% UI = $13,974, $19,656), equating to approximately $684 (95% UI = $582, $819) per school. Adapted PACE was associated with cost-savings of approximately $8954 (95% UI = $4161, $13,432) in total delivery costs or $373 (95% UI = $173, $560) per school.

Discussion

As one study in a sequence of optimisation research, we conducted a robust noninferiority trial to explore the potential of an adapted model of PACE with reduced in-person contact. The findings showed a high probability (96%) that Adapted PACE was noninferior to the original PACE model in assisting teachers to implement weekly school day physical activity. Simply put, there is a high likelihood that Adapted PACE is “as good as” [26] original PACE in terms of effectiveness. An in-depth exploration of program implementation, provided in the published mixed methods process evaluation of this trial [58], corroborates these findings. In the current study, cost-minimisation showed substantial savings with the adapted model. The adapted, multi-strategy PACE intervention maintained a meaningful effect at a reduced cost; therefore, it is a more attractive option for achieving policy implementation, and one that may be more amenable to implementing at scale. The results are discussed in light of the literature following.

In a frequentist analysis, if the lower 95% confidence limit crosses the ∆ (when a favourable result is reflected as an increase in the outcome), this suggests, as per the CONSORT extension for noninferiority trials [27], that the findings are ‘inconclusive’ and any claim of noninferiority invalid. Importantly, the CONSORT reporting guidelines [27] are for 95% confidence limits (i.e., a frequentist paradigm), whereas we have presented 95% credible intervals from Bayesian analyses which does not correspond to a frequentist 95% confidence limit as they are not probability distributions. Thus, frequentist approaches cannot produce estimates of the probability of noninferiority. Through Bayesian analyses we have estimated this quantity of interest – i.e., the probability estimate for noninferiority which was a decidedly high 96% for scheduled physical activity and an even higher 99.6% for the scheduling of energisers. It is therefore highly probable that Adapted PACE is noninferior to PACE. Together with the substantially reduced cost, we are satisfied that Adapted PACE is a valuable approach that can achieve program objectives. Under a frequentist approach, we would have been unable able to draw any informative conclusions from the null-effects. As an example, a recent study of a public health program for reducing the risk of falls in older adults assessed whether a group-delivered format was noninferior to an individually delivered format [59]. Similar to our study, the upper bound of the two-sided 95% confidence interval crossed the predefined noninferiority margin, indicating inconclusive results [59]. Whilst the research design was robust, and the authors propose that the actual difference may have been close, the use of a Frequentist analytic approach did not enable interpretation of the null findings or meaningful conclusions.

When examining the individual components of physical activity scheduled, energisers were the only component scheduled at a higher rate by the Adapted PACE group compared to the original PACE group, with an almost 100% probability of being noninferior (99.1%). Energisers are short and do not require dedicated space, extensive resources, additional curriculum time or specialised training in physical education to deliver [60]. Consequently, compared to the other secondary outcomes in this trial (active lessons and PE), energisers may represent an opportune strategy that requires minimal external, in-person support to increase the amount of physical activity delivered by teachers across the school week. Energisers were the prominent area of change in previous trials of PACE [16, 18] also sustained at longer-term follow-up [16]. Our findings emphasize the benefit and ease of energisers as the focus for improving scheduled school day physical activity, and suggest that this is possible via Adapted PACE.

Little is known about adaptations made to implementation interventions, including the processes behind these decisions [51]. Using a rigorous, evidence informed and co-created adaptation process, this study demonstrated that the positive impact of PACE was not lost when delivered with reduced in-person support. This is important as in-person delivery typically requires more from agencies aiming to implement health interventions (e.g., transportation, money and time) [22, 61] and may be an impediment to implementation support particularly for remote or rural areas. Adapted PACE may address inequalities in health services by providing a model of support that enables more equitable access. Additionally, the costs of delivering Adapted PACE per school was nearly two thirds of the original model, effectively enabling a 35% increase in the number of schools that could receive implementation support given a fixed health service budget. Such findings are consistent with similar research of distance-delivered health interventions compared with in-person [61, 62] including the previously optimised school-based nutrition policy implementation strategy [23, 24]. Nonetheless, improvements in efficiency of this magnitude are considerable from a health service perspective and may yield substantial improvements in community health if adopted at scale. Specifically, the AUD $373 cost-savings per school represents sizable savings for scale-up to the remaining 400+ primary schools within the service region (>$149,200) or to the 1600+ primary schools across NSW [46] (>$596,800). The findings provide support for the application of mode of delivery adaptations including those in the school setting [20, 42] to support scale-up [63]. Future research should explore the impact of Adapted PACE delivered at scale to expand on the emerging evidence base of adaptations made to scale-up health interventions [20, 42].

A prominent adaptation made for the current study was the use of in-school champions (i.e., an existing teacher at the school – an extant PACE strategy) instead of the health service, to deliver the training to school teachers. Champions are common in school-based health implementation interventions [40, 41, 64, 65] and may be opportune to assume responsibility for peer-education or similar strategies. In a 2006 three-arm cluster RCT, Naylor and colleagues [40] compared the effectiveness of a multi-strategy school physical activity intervention using two delivery approaches with different cost implications: in-school champions and external liaisons. No significant difference was found between the groups in daily physical activity scheduled by teachers, although both showed improvements compared to a no-intervention control group. The current study substantiates this evidence base and adds explicit cost comparisons. The use of an in-school champion to deliver staff training, rather than an external project officer, resulted in cost-savings of approximately $206 per school. In-school champions may assist with physical activity program delivery (particularly for peer-education) at a reduced cost.

This study is the third in a sequence of RCTs undertaken to optimise PACE. Data from preceding trials were used to establish the priors that enabled the calculation of a robust noninferiority margin and efficient Bayesian analytical methods [66]. It also provided the basis to support the selection of adaptations. In this context, the findings of this study provide support for iterative, data-driven and co-ordinated processes in achieving improvement in implementation approaches. Specifically it provides one means of addressing the challenges to the implementation of physical activity policies that have beset schools internationally for over a decade. It is also consistent with achievements in implementation where similar methods have been employed, such as the use of implementation laboratories to improve diabetes management or prescribing behaviour [67]. The broader application of these processes has tremendous potential to support the implementation of other physical activity policies and indeed, other preventive health initiatives in schools and similar settings.

A number of limitations should be considered when interpreting the trial findings. Measurement of outcomes via teacher's self-report in daily logbooks is at risk of social desirability and recall bias, which may lead to teacher's over-estimation of scheduled physical activity. However, as this was a noninferiority trial, and both arms received an active PACE intervention, any data collection limitations are associated with both trial arms and should not obscure the noninferiority analyses. Teacher logbooks have also been use for all evaluations of PACE to-date (originally chosen based on use in other school-based studies [40, 51, 68] and pragmatics [16, 18]), and a consistent outcome measure may, in future, enable the assessment of any ‘scale-up penalty’ [20]. In addition, the generalisability of the findings are limited as the trial was undertaken within one health service region. Future research should explore the impact of Adapted PACE delivered within other contexts. This may be particularly important to improve physical activity policy implementation where it remains an issue in other Australian states [69] and worldwide [3, 4, 8, 10, 11]. Finally, due to time constraints, the result of a real-world health service delivery context, there is no published protocol. Whilst this increases the risk of reporting bias, the main study methods including the outcome measures and data collection procedures are identical to the other PACE trials, which have previously been published [16, 18]. Moreover, the analysis was planned a-priori in consult with independent senior statisticians who have training and expertise in Bayesian methods.

Conclusions

This study used a unique research design and analytic approach and showed that the use of scalable delivery modalities substantially reduced the cost of PACE without compromising its effectiveness. The study supports the use of mode of delivery adaptations as a strategy to minimise the relative costs of implementing health interventions, without adversely impacting on their effects. The findings should be of particular interest to health and education policy makers and practitioners interested in maximising the benefits to student health from supporting the large scale implementation of physical activity polices in schools.