Table 5 Descriptive statistics of signalling questions that contributed most to the high risk of bias
Bias domain | Signalling question | Judgement | Frequency of eligible outcomes | Percentage of eligible outcomesa |
|---|---|---|---|---|
Bias due to confounding | Did the authors use an appropriate analysis method that adjusted for all the critically important confounding domains? | Yes | 0 | 0 % |
No | 15 | 100 % | ||
Critically important confounding domains not controlled for | Â | Â | Â | |
Differences in baseline outcome measurements | - | 10 | 66.6 % | |
Differences in baseline demographic characteristics | - | 9 | 60 % | |
Any unusual events | - | 4 | 27 % | |
Socioeconomic or political influences | - | 2 | 13 % | |
What variables were used to match intervention and control sites? | Â | Â | Â | |
Demographic variables (e.g., ethnicity, income, education) | - | 5 | 62.5 % | |
Features, facilities or amenities | - | 5 | 62.5 % | |
Size | - | 2 | 25 % 12.5 % | |
Land use | - | 1 | ||
Same neighbourhood | - | 1 | 12.5 % | |
Is the control site well matched to the intervention site? | Yes | 4 | 26.7 % | |
No No information | 9 2 | 60 % 13.3 % | ||
Were there multiple control sites? | Yes | 9 | 60 % | |
No | 6 | 40 % | ||
Bias in selection of participants into the study | Is there a fully justified sample size calculation? | Yes | 0 | 0 % |
No | 15 | 100 % | ||
Is there a clear and sufficient description of the sample? | Yes | 5 | 33 % | |
No | 7 | 47 % | ||
Not applicableb | 3 | 20 % | ||
Bias in measurement of interventions | Did the authors describe… |  |  |  |
… what was modified in the intervention? | Yes | 15 | 100 % | |
No | 0 | 0 % | ||
… where the intervention was implemented? | Yes | 9 | 60 % | |
No | 6 | 40 % | ||
… how long it took to construct the intervention? | Yes | 4 | 26.7 % | |
No | 8 | 53.3 % | ||
No (and potential overlap with intervention construction) | 3 | 20 % | ||
Bias due to departures from intended interventions | Was individual-level intervention exposure measured? | Yes | 4 | 67 % |
No | 2 | 33 % | ||
Was individual-level intervention exposure measured objectively? | Yes | 0 | 0 % | |
No | 4 | 100 % | ||
Bias in measurement of outcomes | Was the outcome measure valid and reliable? | Yes | 7 | 47 % |
No | 8 | 53 % | ||
Were the outcomes measured over a period of more than one week at each time point? | Yes | 3 | 37.5 % | |
No | 5 | 62.5 % | ||
Were there multiple follow-up time points? | Yes | 4 | 27 % | |
No | 11 | 73 % | ||
Bias in selection of the reported result | Was a study protocol published? | Yes | 0 | 0 % |
No | 15 | 100 % | ||
Did the authors provide a clear and compelling justification for not publishing a study protocol? | Yes | 0 | 0 % | |
No | 15 | 100 % |