Skip to main content

Table 1 Study internal and external validity coding criteria

From: Computer-tailored physical activity behavior change interventions targeting adults: a systematic review

  Criteria description Scoring for criteria
Internal validity criteria
1 Was the method of randomization appropriate? Y = 1; N = 0
2 Were baseline groups equivalent on important demographic measures? Y = 1; N = 0; UK = 0
  If No, was analysis conducted to estimate/adjust for effect of demographic measure on outcomes? Y = 1; N = 0; UK = 0
3 Did the design of the study include comparison to a no treatment control group or a group with either no technology or no tailoring? Y = 1; N = 0
4 Was retention rate ≥ 80% at post-test/post-intervention follow-up? Y = 1; N = 0
5 Were Outcome Measurement instruments valid? Was there a description of instrument reliability/validity (reference or coefficients) OR did they use a well-established known valid measure? Y = 1; N = 0
6 Was an objective measure of behavior change used or did they rely solely on self-report measures? Objective = 1; Self-report only = 0
7 Was power analysis reported to determine sample size? Y = 1; N = 0
8 Were analyses conducted with consideration for missing data that maintain fidelity of the randomization (e.g. intention to treat, imputation)? Note: if 100% retention then N/A Y = 1; N = 0
9 Was the intervention based on theory? Y = 1; N = 0
External validity criteria
1 Were recruitment methods and/or inclusion & exclusion criteria sufficiently described? Both = 1, Either = 0.5; None = 0
2 Were participation/recruitment rates provided OR Are analyses reported on the similarity and differences between participants versus either those who decline or the intended target audience (individuals or settings)? Y = 1; N = 0
3 Was a large heterogeneous sample used? Was the representativeness of participants described? Was a homogenous/heterogeneous sample sought for target population? Do the exclusion criteria used reduce the generalizability of findings? Generalizable population = 1
4 Was the representativeness of the setting described? Was the study conducted in an uncontrolled/controlled setting? Can their findings only be generalized to the limited conditions within which the research was carried out? Generalizable setting (real-life) = 1; Controlled = 0
5 Were all participants who entered trial accounted for at its conclusion i.e. Are data on attrition by condition reported OR was dropout rate described? Y = 0.5; N = 0
  Are drop-outs' compared to completers OR are the dropout's characteristics and reasons for drop-out described? Y = 0.5; N = 0
6 Was the use of comparison conditions relevant to real-world decisions? (the computer-tailored treatment group was compared with either non-technology or non-tailored or alternative programs rather than no treatment) Y = 1; N = 0
7 Are data on the costs presented? Y = 1; N = 0
8 Was there sufficient description of the intervention, including: method of tailoring, duration and intensity (amount of contact time required)? Y = 1; N = 0
9 Are data reported on maintenance or longer-term effects? Short-term = 0; Medium term = 0.5; Long-term = 1