Meta-analysis of internet-delivered interventions to increase physical activity levels

Abstract

Many internet-delivered physical activity behaviour change programs have been developed and evaluated. However, further evidence is required to ascertain the overall effectiveness of such interventions. The objective of the present review was to evaluate the effectiveness of internet-delivered interventions to increase physical activity, whilst also examining the effect of intervention moderators. A systematic search strategy identified relevant studies published in the English-language from Pubmed, Proquest, Scopus, PsychINFO, CINHAL, and Sport Discuss (January 1990 – June 2011). Eligible studies were required to include an internet-delivered intervention, target an adult population, measure and target physical activity as an outcome variable, and include a comparison group that did not receive internet-delivered materials. Studies were coded independently by two investigators. Overall effect sizes were combined based on the fixed effect model. Homogeneity and subsequent exploratory moderator analysis was undertaken. A total of 34 articles were identified for inclusion. The overall mean effect of internet-delivered interventions on physical activity was d = 0.14 (p = 0.00). Fixed-effect analysis revealed significant heterogeneity across studies (Q = 73.75; p = 0.00). Moderating variables such as larger sample size, screening for baseline physical activity levels and the inclusion of educational components significantly increased intervention effectiveness. Results of the meta-analysis support the delivery of internet-delivered interventions in producing positive changes in physical activity, however effect sizes were small. The ability of internet-delivered interventions to produce meaningful change in long-term physical activity remains unclear.

Introduction

Estimates from the World Health Organisation [1] suggest that approximately 60% of the world’s population are classified as inactive or insufficiently active to receive health benefits. With the increasing burden caused by physical inactivity and chronic disease, new ways for delivering behaviour change programs to large numbers of people at low cost are needed. In particular, the internet offers an innovative medium to produce health behaviour change in terms of reach, availability and opportunities for interactive approaches [2]. Statistics demonstrate more than a 300% increase in internet usage since 2000, with over 1.5 billion internet users worldwide, representing approximately 23.5% of the world’s population [3]. Most important, internet access provides an alternate means to health care promotion for individuals who cannot access standard care due to physical disability or living in remote areas [2]. Already a large number of individuals are utilising the internet to access health-related information [4], creating an opportunity to develop and deliver health-related behaviour change interventions via the internet. Furthermore, internet-delivered behaviour change interventions are becoming increasingly common for physical activity [2, 5, 6] particularly over the last 10 years [7].

Several reviews have examined the effectiveness of internet-delivered interventions to produce health related behaviour change among adults [2, 6, 7] and one has specifically examined the components and operationalisation of computer tailored programs across all ages [8]. Further, two meta-analysis were conducted on general health behaviours in both children and adults; one focusing on computer delivery [9] and another comparing web-based to non-web-based delivery [5]. The main findings indicate that: a) short-term behaviour change is more often reported than long-term behaviour change; [2, 7–9] b) specific intervention elements such as website components (e.g., tailored content, theoretical design) and interactive features need to be further evaluated and explored for their role in both short-and long-term behaviour change and increasing website usage [2, 6, 7], and, c) internet-delivered physical activity interventions are more effective than true control groups [6]. This meta-analysis expands upon what is currently known through comprehensively synthesizing the effect of internet interventions on physical activity levels and variations in physical activity outcomes due to potential moderating variables. These findings will be useful to determine the current standing of internet interventions and to identify future directions for these types of interventions.

Methods

Inclusion criteria

To be included in the review, studies were required to provide an internet-delivered intervention with a focus on increasing physical activity. More specific, studies were included if they met the following criteria: a) participants were ≥ 18 years of age; b) the main form of intervention delivery was via the internet with either the use of a web page for the delivery and/or exchange of information, or in the form of email communication; c) physical activity and sample size measures were reported for both intervention and comparison groups; d) studies comprised of an experimental design, such as a randomized or quasi-experimental design; e) studies included a non-internet comparison group; and f) articles published in the English language. Studies that did not meet all inclusion criteria were deemed ineligible and were excluded. Additionally, studies that did not provide enough data to allow for the calculation of effect sizes were deemed ineligible.

Search method

A comprehensive search strategy was undertaken to identify all possible studies for inclusion. The following electronic databases were searched: Pubmed, Proquest, Scopus, PsychINFO, CINHAL, and Sport Discuss. The search process was limited to articles published or provided ahead of publication access between January 1990 and June 2011. To locate potential studies the following search string was used: (“physical activity” OR exercise OR “physical fitness” OR walking) AND (internet OR “website delivered” OR “web based” OR,” world wide web”) AND (education OR behavio* OR intervention). All references including duplicates were then imported into EndNote (bibliographic software). Reference lists of all relevant review articles [2, 5–13] were manually searched for potential studies not yet identified [14].

Screening of articles

After the removal of duplicates, articles underwent two phases of screening to identify the final sample. Phase one involved scanning article abstracts for inclusion criteria to rule out literature that clearly did not meet the inclusion criteria. In phase two, full text versions of the remaining articles were obtained and further screened to identify the final set of articles for inclusion.

The initial search strategy (excluding duplicates) identified 2651 potentially relevant articles. Following title/abstract screening and screening of relevant review articles, a reference list of 172 potentially relevant articles remained. After assessing the full text articles, the final set of articles for inclusion in this meta-analysis resulted in 34 primary articles (Figure 1) representing 34 unique interventions [15–48].

Figure 1

Selection for studies of internet delivered physical activity interventions.

Data extraction

Once finalised for inclusion, studies were collated and coded independently by two of the researchers (CD and CV), any discrepancies in coding were resolved through discussions [8, 49]. The coding framework was pilot tested (i.e., both researchers independently coded two test articles) and refined prior to the first article being coded. Characteristics were coded under four general categories including: study design (e.g., sample size, physical activity mode targeted [i.e. leisure time or total], duration of intervention), participant characteristics (e.g., age, gender, population health status, and baseline physical activity levels), intervention features (e.g., number of intervention contacts, type of tailoring, presence of a theoretical underpinning, interactive features [e.g. goal setting, quizzes]) and intervention results (sample sizes, physical activity measures and any additional information to allow for the calculation of effect sizes). The list was developed based on previous reviews [2, 5–8, 10, 50] and perusal of original research articles published on the topic. Articles were coded to provide descriptive information and to allow for subsequent moderator analysis. Tables 1 & 2 contain information of coded characteristics for the first three categories. The coding framework is not exhaustive of all intervention aspects and only characteristics reported in sufficient detail across studies are subsequently reported on.

Table 1 Study design and intervention characteristics

Table 2 Intervention features

The majority of characteristics included in the first three categories, study design, participant characteristics and intervention features were also included in the moderator analysis. Potential moderating variables (see Table 3 for the list of moderator variables) were included if able to effectively extract data and code the variable. For example, the presence of a theoretical underpinning as a moderating variable of intervention effectiveness could not be examined, as only two studies did not specify a theoretical framework. Additionally, a number of continuous moderating variables were recoded into categorical variables to enable a more meaningful analysis to be undertaken. For example, although attrition was initially coded as a continuous variable it was re-coded as (1) above, or (2) below the average cross-study attrition rate of 23% for the internet intervention groups. Coding of articles resulted in an agreement rate of 92% between the researchers with any discrepancies being resolved.

Table 3 Summary statistics and effect sizes by moderator variable for change in physical activity as a result of internet-delivered interventions

Study quality was assessed based on a previously developed methodological assessment tool, [51] which was modified to specifically address quality assessment for internet-delivered interventions [5]. This evaluation was based on five main criteria: a) study design; b) selection and specification of the study sample; c) specification of illness/conditions; d) reproducibility of the study; and e) outcomes specification and measurement. During the quality assessment process, studies could receive a score up to 18 points; the score obtained by each study was divided by 18 and multiplied by 100 to provide a “Study Quality Percentage”. Study Quality Percentages were then classified as good (66.7% or higher), fair (between 50 to 66.6%) and poor (less than 50%) [52].

Data analysis

Effect sizes (d) were computed to represent the impact of the internet-delivered interventions on physical activity [53]. The effect size (d) is defined as the standardised mean difference and allows meaningful comparisons across measurement instruments. A positive effect size indicates a more favourable change in physical activity for the intervention condition. If studies reported statistics other than means and standard deviations (e.g., F p), efforts were made to estimate d from the information provided [54]. For studies that used more than one follow up measure, effect sizes were calculated using data from the time point closest after intervention completion. This time-point was used in order to best determine the actual effects of the intervention. A separate effect size analysis was calculated for studies that reported six months or greater post-intervention follow up data; this was done to investigate the effectiveness of internet-delivered interventions in producing long-term behaviour change. To assess the possibility of publication bias the Egger test was used [55].

The fixed effects model was explored in relation to the summary effect size to estimate the mean distribution of effects. The heterogeneity statistic (Q) was calculated to determine whether studies shared a common effect size. Q represents the observed weighted sum of squares and df is the expected weighted sum of squares [53]. If heterogeneity was present among the effect sizes, further analysis was undertaken to examine study level moderating factors of physical activity outcomes [53]. The Bonferroni correction factor was applied to adjust the alpha value required for statistical significance within each of the three moderator categories (study design, participant characteristics, intervention features).

Results

Description of included studies

Articles were published from 2001 to 2011, with the modal year being 2007 (n = 8). Of the 34 studies, 21 were from the United States, 4 from Australia, 3 from Canada, 2 from England, and 1 each from Belgium, Taiwan, Scotland and South Korea. At baseline the 34 studies included a total sample size of 11,885 across all groups (including sample size from studies that included more than two groups) or 9,638 participants for the two comparison groups being examined in the current meta-analysis. Physical activity was the primary behaviour targeted in 25 (74%) of the studies.

The average duration of interventions was 12.64 weeks with a range of 2–52 weeks. Overall attrition (20%) was reported in 31 of the included studies with an attrition of 23% experienced in the intervention groups as reported in 28 studies. Although all studies used internet delivery, variation existed in the type of delivery employed. Specifically, 21 studies (62%) included a combination of internet and email, 9 (26%) used internet only and the remaining 4 (12%) used email only. The type of comparison group used also varied; with 26 studies (76%) using a true/standard care control group; while 4 studies used an alternate intervention (12%) and 4 studies used a minimal intervention comparison group (12%). In terms of study quality, none of the studies were rated as poor; 10 (29%) were rated fair, and 24 (71%) as good quality.

Nine (26%) studies used a sample population that were classed as inactive (not meeting the national physical activity guidelines according to their respective country recommendations). The other 25 articles (74%) did not screen for activity status. The majority of studies represented the general population (n = 17; 50%), with the remaining studies involving population with overweight (n = 5; 15%); Type 2 diabetes (n = 4; 11%); arthritis (n = 1; 3%); cardiac rehabilitation (n = 1; 3%); metabolic syndrome (n = 1; 3%); physical disabilities (n = 1; 3%); chronic disease (n = 1; 3%); multiple sclerosis (n = 1; 3%); a diagnosed mental illness (n = 1; 3%) and cardio obstructive pulmonary disorder (n = 1; 3%). The average age represented across studies was 43.06 years, 65% of the overall sample was female and, among the 18 articles that reported on ethnicity, 92% of the sample was Caucasian. The number of times participants logged in to the study website was reported for only 11 (32%) of the included studies. The average number of logins per person/per week was 3.08. Tables 2 & 3 outline study characteristics in more detail.

Indication of results

The estimated overall mean effect of internet-delivered interventions on physical activity was d = 0.14 (p < 0.001; Figure 2), suggesting that internet-delivered interventions had a small but significantly greater impact on physical activity change than the comparison conditions. The results of the Egger test revealed that publication bias was present (p < 0.001). Thus, as recommended by Sterne and colleagues [56], no statistical methods were used to correct for publication bias, as corrections would be based on assumptions and therefore could produce potentially flawed results. Homogeneity tests from the fixed-effect analysis revealed significant heterogeneity across studies (Q = 73.75; p < 0.001). The overall mean effect for sustained physical activity at least 6 months post- intervention (n = 11) resulted in a small but significant effect size d = 0.11 (p < 0.01).

Figure 2

Forrest plot of effect sizes representing effect on physical activity behaviour.

Study Design

Among the 10 study design variables (Table 3), post-intervention sample size (Q_b (1) = 13.1, p < 0.001) was the only significant moderator of internet-delivered interventions on physical activity. Specifically, studies that included less than 35 participants per group at the post-intervention follow up measure (d = 0.40) demonstrated larger effect sizes than the studies that had a post-intervention sample size of 35 or greater (d = 0.12).

Participant Characteristics

Of the four participant characteristic variables (Table 3), initial physical activity level (Q_b (1) = 8.83, p < 0.05) was found to be significant moderator of physical activity change. Studies that screened participants and included only participants classified as sedentary or insufficiently active produced a greater effect size (d = 0.37) than studies that did not screen participants for physical activity (d = 0.12).

Intervention Features

The inclusion of educational components was the only significant moderator (Q_b (1) = 8.02, p < 0. 005) of physical activity change among the 11 intervention features coded (Table 3). Specifically, interventions consisting of educational components producing a larger effect size (d = 0.20) than interventions that did not (d = 0.08).

Discussion

The magnitude of the overall effect size indicates that internet-delivered programs have a small but positive effect on physical activity (d = 0.14). This finding is consistent with previous narrative reviews suggesting the ability of internet-delivered interventions to produce modest effects on physical activity [7, 13]. The results also suggest these interventions produce variable effects on physical activity, as evident by the variance in effects across individual studies, which has also been highlighted by previous research [2, 7, 13]. Despite significant variability in intervention effects and the presence of possible moderator variables, the combined intervention outcomes resulted in improved short-term physical activity, with a smaller effect being found for longer-term behaviour change. This study presents the first meta-analytical review to extensively examine the impact of moderators on the effectiveness of internet-delivered interventions to increase physical activity.

The effect size is similar to recent findings from a meta-analysis investigating the effect of physical activity interventions for healthy adults across all modes of delivery [57]. The review did not specifically examine internet-delivered interventions but found the effect size across all delivery modes (d = 0.19) was similar to other mediated interventions (email and telephone) to increase physical activity levels (d = 0.15) [57]. However, the effect size was smaller than face-to-face interventions (d = 0.29) [57]. A previous review also attempted to conduct a meta-analysis on the effects of distance interventions to increase physical activity, but due to heterogeneity and poor study quality opted to do a systematic narrative review instead [50]. Since the cut-off date for the search strategy of that study [50], an additional 26 studies were published that were included in the current meta-analysis. Furthermore, due to the increase in publications and higher quality of these additional studies, it was possible to conduct a moderator analysis in the current study.

Internet-delivered interventions have the potential to produce small but significant increases in physical activity levels. Given the potential breadth of delivery, the public health impact of producing small changes in physical activity across a population has the potential for large positive changes at the population level [58]. This finding has even greater potential when considering the increase in effectiveness when specifically targeting a sedentary population (d = 0.37). Sedentary individuals are at higher risk of developing a number of chronic conditions and facing premature morbidity and mortality [1], hence it is encouraging to observe the effect sizes are larger for interventions targeting insufficiently active individuals. The potential implications for population level change remain tempered by questions as to whether these small effect sizes are clinically relevant. Additionally, the moderator analysis indicated that studies with a larger sample size have a smaller effect (d = 0.12) and it may be argued that these effect sizes are more representative of the actual effect of internet-delivered interventions on physical activity levels. Furthermore despite internet delivered interventions making claims about using the internet for reaching large populations at low cost, surprisingly few have evaluated their cost-effectiveness. We could not examine this factor in the current meta-analysis, and it is recommended that future studies evaluate the cost-effectiveness in terms of development, maintenance and breadth of delivery of internet-delivered interventions to allow comparisons to traditional modes of delivery.

Recently, meta-analyses examining the effectiveness of behavioural medicine interventions have been critiqued for including underpowered studies (less than 35 participants per group). It was argued that small studies are more likely to be published if they find positive results, increasing the likelihood of publication bias [55]. The outcomes of the present review confirm the presence of bias. Studies that do not find statistically significant results may not be being published either due to author’s not attempting to publish or journals not accepting the article for publication. Regardless, the findings should be interpreted within this limitation. There remains a need to include well-designed, randomised controlled trials that include adequate sample size at baseline to allow for attrition and to maintain sufficient power at the scheduled follow-up periods.

Increasing physical activity levels and maintaining the behaviour are important in terms of generating sustained health benefits [59]. Eleven studies followed participants for six months or more post-intervention. This resulted in a small effect size (d = 0.11) and further investigation is needed to determine the effectiveness of internet-delivered interventions to produce long-term change in physical activity levels. Future interventions should include long-term follow up measures for physical activity to identify overall effectiveness. Additionally, studies that targeted physical activity only or physical activity and additional behaviours produced similar effect sizes. This finding is supported by previous research [7] and provides justification for further investigation given the role of multiple behavioural risk factors in the development of non-communicable chronic disease.

Identifying factors that enhance intervention effectiveness can inform the development of future research to produce greater physical activity change. For instance, including structured educational materials that involved the exchange of information intended to influence physical activity was the only intervention feature found to moderate intervention effectiveness. Providing education has previously shown to be an effective behaviour change technique to increase physical activity among chronically ill adults [10]. Some of the intervention features examined, such as email reminders and updated content, are not necessarily included in an attempt to optimise intervention effectiveness but are incorporated as part of intervention design to enhance exposure to the program [60]. Previous research has demonstrated that intervention features such as the number of intervention contacts, tailored content, goal setting, self-monitoring and updated content enhance intervention effectiveness [7]. Although these intervention features were not found to be significant moderators on physical activity change in the current analysis, future research should attempt to isolate the impact of specific intervention features on physical activity change through implementing high quality study designs (such as randomised controlled trials, having adequate sample sizes, using validated instruments to measure study outcomes and appropriate reporting of results) that will allow such investigation.

Based on 11 of the included studies, the average number of logins was 3.08 per-person-per-week which exceeds the traditional one-contact–per-week that is common among face-to-face interventions. Due to lack of data, it was not possible to analyse the decline in website logins over time, however it is an issue often identified throughout the literature [7, 60]. Several of the studies that did track decay of logins over time have reported that the majority of intervention logins occurred within the first few weeks of the study duration with a very steep decline shortly thereafter [31, 61, 62], hence it is an important issue. Enhancing participant engagement is directly related to increased exposure to the intervention and research has identified a clear dose–response relationship between the intensity of the intervention and resulting behaviour change [61, 63]. It is therefore apparent that maintaining website engagement is an important factor in relation to the potential effectiveness [31, 33, 61, 64, 65]. Due to limited reporting of login and other website engagement data, the impact of intervention features on program engagement could not be evaluated. Results from a recent systematic review suggest that intervention features, such as provisions for peer or counsellor support, email and/or phone contact with visitors and regular website updates, were related to increased exposure in internet-delivered healthy lifestyle interventions [60]. However, the authors noted significant issues in the consistency of reporting engagement measures and recommended that future interventions apply consistent engagement measures across studies [60]. Consistent with the current study, Brouwer and colleagues [60] identified that the most common measure of website engagement was the average number of logins to the intervention website, therefore future web-based interventions should also consistently report on it. Nevertheless, additional measures of website engagement are required, such as decline in website logins over time and time exposed to specific intervention elements.

Limitations

The results presented should be interpreted within the limitations of the current meta-analysis. Due to the low number of articles we were unable to combine variables and conduct a meta-regression when examining the impact of moderators. This is important as features that were significant moderators also displayed significant heterogeneity, which indicated the presence of other factors still influencing the effect size. In this respect moderator analysis should be interpreted with caution. A number of features were also unable to be examined as part of the moderator analysis due to insufficient reporting among primary articles. Another limitation is that the majority of studies mostly incorporated the use of self-report measures for physical activity, included largely Caucasian and well education samples. Although the majority were based on valid and reliable measures, the better measure is still to use objective measures. In terms of internet delivery this can prove challenging if programs are being widely disseminated. Finally, the effect size cannot be translated to represent a more meaningful and clinically relevant change in physical activity level (for example minutes of moderate or vigorous physical activity) as studies vary widely in the form of measured used for assessing behaviour.

Conclusion

Overall, the findings demonstrate internet-delivered interventions are effective in producing small but significant increases in physical activity. Although the effects are small, producing such changes in behaviour across a large population can have powerful implications at a population level. To fully harness the potential of internet-delivered interventions to produce population-wide effects it is important that interventions target insufficiently active individuals as well as more diverse populations. Programs are continuing to evolve with advances in technology, but it is imperative that rigorous high quality research continues to explore the effectiveness of internet delivered interventions. Future research in the area should focus on the various aspects of internet-delivered interventions that increase the engagement and retention of the target audience so as to better understand the elements that will enhance effectiveness of this type of intervention.