Table 1 Description of terms for validity and reliability in PA and SB measurement

 Face validity

The extent to which a measure looks like it will, or appears to, provide the desired information. Assessed by expert consensus and theoretical consideration.

Likewise for the proposed data processing and generation of outcome variables. Assessed by expert consensus and theoretical consideration.

 Content validity

The extent to which a measure covers all aspects of the intended behavioural or physiological domains or dimensions (see Fig. 1). Assessed by examination of domain or dimension of interest.

Likewise for the proposed data processing and generation of outcome variables.

 Convergent validity

The extent of the agreement with another (non-criterion) measure that should assess the same PA or SB parameter based on face and content validity. Assessed quantitatively.

Useful when the criterion is very resource intensive.

This approach also allows assessment of whether the measures can be used interchangeably, or the data from the two measures pooled or otherwise compared.

 Criterion validity

The extent of the agreement between a measure and another already held as being a criterion or gold standard. Assessed quantitatively. Called absolute validity when compared to measure known to provide perfectly true values.

 Concurrent validity

Assessment of convergent or criterion validity when measures taken at same time.

 Predictive validity

Assessment of convergent or criterion validity when measures taken at different times.

 Internal validity

The extent to which conclusions drawn from the experimental data are free from confounding issues which cause bias such as reactivity and missing data; similar to methodological quality. Assessed by examination of relevant issues.

 External validity

The extent to which conclusions drawn from the data are generalizable to the wider populations. Assessed by examination of age, sex, ethnic origin, socio-economic status, etc., of study sample.

This could be assessed by a theoretical justification or empirical demonstration such as field testing and small scale “proof of concept” studies. These should assess participant feedback (e.g. satisfaction and burden) as well as data issues (e.g. can meaningful information be produced in reasonable time frames?)

 Test-retest reliability

The extent to which test scores are consistent from one test administration to the next; keeping as many conditions (e.g. researcher, timing, preparation, etc.) as possible unchanged. Assessed quantitatively.

This estimate incorporates any factors that cannot be controlled e.g. intra-rater reliability, behaviour change, etc.

 Inter/intra-rater reliability

The extent to which test scores are consistent when measurements are taken by different people using the same methods (inter-rater) or at different times by the same person (inter-rater). Assessed quantitatively.

 Inter/intra-instrument reliability

The extent to which test scores are consistent when measurements of the same thing are taken by different versions of the same instrument (inter-instrument) or repeatedly by the same version of an instrument (intra-instrument). Assessed quantitatively.

 Behavioural reliability

The extent to which stability in behaviour has been considered when assessing other aspects of reliability.