Skip to main content

Table 1 RE-AIM operationalization, data sources and outcome detail

From: Evaluating the reach, effectiveness, adoption, implementation and maintenance of the Resistance Training for Teens program

 

Reach

Effectiveness

Adoption

Implementation

Maintenance

Operationalization

The estimated number and representativeness of students who may have been exposed to the program.

The impact of the program on students’ fitness, physical activity behaviors psychological constructs.

The number and representativeness of schools with at least one teacher trained (via the professional learning workshop) to deliver RT for Teens.

The extent to which the program was delivered as intended.

The extent to which the program was sustained in schools.

Data sources

i) RT for Teens teacher workshop enrollment data (to estimate student numbers).

ii) School student enrollment data (from MySchoola website; (https://www.myschool.edu.au) to evaluate dissemination cohort student characteristics.

i) Fitness tests and survey completion in a sub-sample of students (n = 750; from varied year levels) pre- (week 1) and post-program (week 10).

Note: there was no control group

A sub-sample of schools (n = 17) were selected through teacher interest following workshop completion. These teachers were asked to deliver the program for 10-weeks and were also asked to facilitate one lesson observation during the 10-week period (see Implementation for observation data).

i) RT for Teens teacher workshop enrollment data.

ii) School data (from MySchoola website) to evaluate dissemination and effectiveness sub-sample school characteristics.

iii) Teacher demographics questionnaire (n = 429)

Teacher questionnaire sent to all teachers prior to workshop attendance, with n = 429 (90%) completing it.

iv) RT for Teens app.

i) RT for Teens session observations.

ii) RT for Teens app usage

Session observations were conducted at schools involved in the effectiveness sub-sample (n = 17; see Effectiveness for sub-sample information) to measure program implementation. One observation per class (n = 22 classes from the 17 schools) was conducted at the approximate mid-point of the 10-week (~ week 5) program period. Members of the research team, all of whom held a tertiary PE teaching qualification, and had been involved in the adaptation of the SAAFE principles to RT, conducted the lesson observations using a structured observation checklist. This measure has been used extensively in past school-based studies [36, 44], however it has not been validated.

Number of workouts and fitness tests conducted using the app were also assessed to evaluate implementation of this program resource.

i) RT for Teens workshop enrollment data.

Outcome detail/s

Using teacher enrollment data (n = 468) from the workshops, we assumed that each teacher delivered the program to at least one class of 23 students (the mean class size in grades 7–10 in NSW). The characteristics of students enrolled in schools with at least one trained teacher (dissemination cohort) were collected from the MySchoola database. Characteristics included: gender distribution, SES, Indigenous status, and language background other than English

These characteristics were also collected and reported separately for schools involved in the effectiveness sub-sample (see Effectiveness for sub-sample information).

Sub-sample student-level data were collected pre- (week 1) and post-program (week 10). These assessments included fitness testing and survey completion. Fitness testing was completed by the research team, or by the classroom teacher who recorded results for the research team. Surveys were provided to schools as an online link, or as printed hard copies (only one school requested this due to internet unavailability).

Measures

Muscular fitness was assessed using the 90° push-up test (upper body muscular endurance) [45] and the standing long jump test (lower body power) [46]. Perceived fitness was reported using the IFIS, a 5-item instrument reporting perceptions on general fitness, and then separately for individual fitness components: CRF, muscular strength, speed/agility and flexibility [47]. RT self-efficacy was evaluated using a 5-item scale developed for use with adolescents [48]. Motivation to participate in RT was assessed using an adapted version of the BREQ-2 [49]. The intrinsic and identified subscales from the BREQ-2 [49] were utilized to evaluate autonomous motivation for RT using a 5-item scale. Participants self-reported their total physical activity and participation in muscle-strengthening physical activity using validated measures [50, 51].

Characteristics of schools (dissemination and effectiveness sub-sample) were collected via the MySchoola database, and included selective status, sector, location, type, and Index of Community Socio-Educational Advantage based on school locality (providing a measure of schools’ area-level SES)

Characteristics of teachers were also collected via survey prior to workshop attendance, including: age, sex, years of teaching experience, area of teaching specialty, other qualifications related to health-related fitness, and other recent professional development related to health and PE

Adoption of the RT for Teens smartphone app was also operationalized, at the teacher and school level.

During these observations, researchers collected data on: i) fidelity and ii) adherence to SAAFE teaching principles [52]

Fidelity

Measured as the compliance with the proposed physical activity session structure (see Supplementary Table 1 and Supplementary Fig. 1a). This structure recommended 10 components to include within the session. During the observation, researchers assessed what components were being delivered to students during sessions, as well as whether teachers utilized any RT for Teens resources (i.e., circuit cards, smartphone app)

Adherence to SAAFE principles

The SAAFE principles, and strategies aligned with each principle were explained to teachers during the RT for Teens Professional Learning workshop. Adherence to the SAAFE principles was determined using a 16-item checklist, with items recorded on a 5-point scale (i.e., 1 = Not at all true to 5 = Very true), with a value assigned to each of the 16 specific strategies covered during teacher training (see Supplementary Fig. 1b). Based on these scores, a percentage was calculated by summing the mean for each of the strategies and dividing by the maximum possible score, for each SAAFE principle. Means for each strategy were also calculated.

Long-term follow-up of individual-level data were not collected during the dissemination trial. Workshop enrollment data were used to determine potential institutionalization of RT for Teens, where schools sent a new teacher/s, to an additional workshop after initial training.

Analysis

Descriptive statistics (mean, standard deviations, range) were utilized to report these data.

Linear mixed models were used to analyze outcomes using IBM SPSS Statistics for Windows, Version 20.0 (2010 SPSS Inc., IBM Company Armonk, NY), with significance set at p < 0.05. Models assessed the impact of time on the reported variable. Mixed models are consistent with the intention-to-treat principle, assuming data are missing at random.

Descriptive statistics (mean, standard deviations, range) were utilized to report these data.

Descriptive statistics (mean, standard deviations, range) were utilized to report these data.

Descriptive statistics (mean, standard deviations, range) were utilized to report these data.

  1. Abbreviations: BREQ-2 Behavioral Regulations in Exercise Questionnaire-2, CRF Cardiorespiratory fitness, IFIS International Fitness Scale, NSW New South Wales, PE Physical education, RT Resistance training, SAAFE Supportive, Active, Autonomous, Fair, and Enjoyable, SES Socio-economic status