Skip to main content

Table 2 Description of the methodological quality, design elements and additional analyses

From: A systematic review of the effect of infrastructural interventions to promote cycling: strengthening causal inference from observational data

Reference (country) Quality criteria [19]a Methodological items [12]b
Study designc Participation and representativeness Comparability at baseline Credibility of data collection methods Retention Attributability of effect to intervention Multiple comparison groups Complementing research methodologies
Cycling behavior
Aittasalo [23] (Finland) C Participation: 49%
Only limited data was available regarding the working-age population in the region. The study population was broadly representative with the general adult population in the region
No comparison group Survey: no info on validity 45% • Half of the workplaces went through economic problems and workforce adjustment during the study No other comparison groups • Published protocol
• Survey among employees
• Safety monitoring
• Count data
Aldred [24] (UK) A Participation: 2% There was an underrepresentation of 16 to 24 year olds, non-white individuals and unemployed individuals. Participants were more likely to have a car or van in the household, and to have cycled Comparisons groups were broadly comparable. Adjusted for a wide range of variables 7-day recall instrument with acceptable validity 50% • Dose response effects were reported
• The first interventions were targeting areas perceived as more receptive to cycling and walking interventions
No other comparison groups • Survey among residents
Brown [25] (US) A Participation: 29% Representativeness was not shown Adjusted for some of the characteristics in which the groups significantly differed at baseline GPS and accelerometer data, using validated algorithm 59% • Multiple improvements to other nearby infrastructure
• Spill-over effect occurred: control residents were exposed to the intervention
No other comparison groups • Published protocol
• Survey among residents
Health indicators:
• Energy expenditure
• BMI
Brown [26] (US) C Same as above No comparison group Same as above Same as above No comments made No other comparison groups Same as above
Burbidge and Goulias [27] (US) C Participation was not shown. Study population was older, had less cars in the household and were more often unemployed No comparison group 1-day activity diary, modified from a validated household activity diary 56% No comments made No other comparison groups • Survey among residents and new residents
• Intercept survey
Health indicators:
• Physical activity
Chowdhury [28] (New Zealand) C Participation was not shown. Study population was representative No comparison group Survey, methods not described Not applicable No comments made No other comparison groups • Survey among residents
Crane [29] (Australia) A Participation was not shown. Study population was higher educated and more physically active than the general population Adjusted for some of the characteristics in which the groups significantly differed Survey: no info on validity
Travel diary: no info on validity
48% • No dose response effects were observed
• Suburbs furthest away from the cycle way were quite diverse in infrastructure
• Spill-over effect occurred: users of the cycle way included participants living in control areas
No other comparison groups • Published protocol
• Survey among residents
• Count data
Health indicators:
• Physical activity
• Quality of life
Deegan [30] (UK) C Participation and representativeness were not shown No comparison group Census data, methods not described Not applicable • Congestion charge and bombings on public transport resulted in sharp increases in cycling levels • The increase in cycling in the intervention areas was larger than observed in other areas • Safety monitoring
Dill [31] (US) A Participation: 3% Representativeness was not shown Adjusted for variables that were tested to be significant GPS and accelerometer data, shown to successfully predict 79% of the cycling trips 72% • The city may have chosen to install bicycle boulevards in areas where residents were supportive of new cycling infrastructure
• Unknown changes in the physical and social environment in specific areas may have influenced the results
• Data collection by means of GPS and accelerometers may have changed behavior
No other comparison groups • Survey among residents
• Count data
Health indicators:
• Physical activity
Evenson [32] (US) C Participation: 47% Study population was more highly educated No comparison group Non validated method of interviewing 64% • Questions mentioning the trail were only asked at follow-up and after assessing cycling behavior
• Substitution of physical activity behavior may have occurred
• Comparing users and non-users of the intervention did not change the results • Survey among residents
Health indicators:
• Physical activity
Goodman [33] (UK) A Participation: 16% Study population was broadly representative, except that fewer young adults were included, and they were somewhat healthier, better educated, and less likely to have children Adjusted for a wide range of variables 7-day recall instrument with acceptable validity
Survey, validated
42% • Dose response effects were reported
• The increase in cycling was only seen for users of the intervention
• Comparing users and non-users of the intervention showed that the increase in cycling was only seen for users of the intervention • Published protocol
• Survey among residents
Health indicators:
• Physical activity
Song [34] (UK) C Same as above No comparison group 7-day recall instrument with acceptable validity
Survey, validated
45% • The increase in cycling may have resulted from the economic crisis, rising fuel costs, and the ageing of the sample No other comparison groups Same as above
Hirsch [35] (US) A Participation and representativeness were not shown. Adjusted for a wide range of variables Census data (before) and a community survey (after): no info on validity Not applicable • Dose response effects were reported
• Unknown if people moving into the neighborhood cycle more, or if existing residents change their behaviors
• Other infrastructure changes, including a new light rail service, may have influenced the results
• Historical trends showed that the increase in cycling in the intervention period was larger than in previous years No other methods used
Krizek [36] (US) C Participation and representativeness were not shown. No comparison group Census data, methods not described Not applicable Many potential factors were listed, but only those with an explanation were listed here:
• Minor other infrastructural improvements were made in the study areas
• Small demographic differences were not the sole explanation of the results
• Intervention areas had already a higher cycling level at baseline. The facilities might be the effect, rather than the cause, of high cycling levels
• The increase in cycling in the intervention area was larger than observed in the area as a whole No other methods used
Lanzendorf [37] (Germany) C Participation and representativeness were not shown. No comparison group National travel survey, valid for comparison over time according to the authors Not applicable • Hard to disentangle the effects of the infrastructure and marketing campaigns. A combination of both may have the largest impact • The increase in cycling in the intervention cities was comparable to the change in other big cities, but larger than in the country as a whole • Document analysis and expert interview to analyze the development of cycling policies
Merom [38] (US) A Participation: 48% Representativeness was not shown. Not adjusted for characteristics in which the groups statistically differed at baseline Telephone interviews, validated 79% No comments made No other comparison groups • Survey among residents
• Bike counts
• Campaign reach
Panter [39] (UK) A Participation was not shown.
The sample contained a higher percentage of woman, older adults and those with a degree, and a smaller proportion of those who rented their home
Adjusted for a wide range of variables 7-day recall instrument with acceptable validity
Survey, validated
41% • Dose response effects were reported No other comparison groups • Published protocol
• Survey among residents
Health indicators:
• Physical activity
Pedroso [40] (US) B Participation and representativeness were not shown No comparison group Census data, methods not described Not applicable • Several other programs and interventions were implemented during the study period No other comparison groups • Safety monitoring
Smith [41] (US) C Participation and representativeness were not shown No comparison group Methods not described Not applicable • The percentage of cyclists using bike lanes declined over time No other comparison groups • Safety monitoring
Wilmink and Hartman [42] (The Netherlands) A Participation and representativeness were not shown No information on comparability Home interview, no info on validity Not shown • There was no change observed in total mobility over time No other comparison groups • Survey among residents
• Bike counts
Usage of the infrastructure
Aittasalo [23] (Finland) C Not applicable No comparison group Automatic counters: 4 locations, continuous measurements for 2 years Not applicable • Half of the workplaces went through economic problems and workforce adjustment during the study No other comparison groups • Published protocol
• Survey among employees
• Safety monitoring
• Cycling behavior
Barnes [43] (US) C Not applicable No comparison group 54 h of video observations: before and after at 1 location, 6 days, 4.5 h per day Not applicable • No unusual weather or traffic patterns were observed
• It is unclear whether cyclist simply changed their routes
No other comparison groups • Safety monitoring
Crane [29] (Australia) B Not applicable No comparison group Automatic counters: 2 locations, measurements in October for 3 years on weekdays, 6 h per day Not applicable • Results may reflect population growth • The increase in cyclists was only seen in the intervention area, while it decreased in the city as a whole
• Historical trends in the number of cyclists were comparable between the intervention areas and the city as a whole
• Published protocol
• Survey among residents
• Cycling behavior
Health indicators:
• Physical activity
• Quality of life
Dill [31] (US) C Not applicable No comparison group Not described in the paper Not applicable • Unknown changes in the physical and social environment in specific areas No other comparison groups • Survey among residents
• Cycling behavior
Health indicators:
• Physical activity
Fitzhugh [44] (US) A Not applicable Broadly comparable 72 h of direct observations: before and after at 3 locations, 2 days, 6 h per day Not applicable • Study neighborhoods were not exposed to any marketing or awareness campaigns
• Spill-over effect may have occurred: people cycling may not live in the intervention neighborhood
No other comparison groups No other methods used
Goodno [45] (US) C Not applicable No comparison group Not described in the paper Not applicable • Weather conditions and seasonality may have influenced the results • The increase in cyclists in the intervention area was larger than in the city as a whole • Survey among residents
• Survey among business owners
• Safety monitoring
• Intercept survey
Hans [46] (Denmark) B Not applicable No comparison group Automatic counters calibrated by visual/manual counts: 2 locations, continuous measurements for 3 years Not applicable • Most of the increase in cyclists can be attributed to switching from alternative routes No other comparison groups • Intercept survey
Heesch [47] (Australia) - direct observations C Not applicable No comparison group 7.5 h of direct observations:
Before: 1 location, 1 day, 2.5 h
After: 2 locations, 1 day, 2.5 h
Not applicable • The findings suggest some shifting of cyclist No other comparison groups • Intercept survey
• Mobile phone application to capture movements of cyclists
Heesch [47] (Australia) - mobile phone application A Only 10% of the population uses the app, and those were not representative of the broader cycling community Comparison streets were all connecting the suburbs and the city center 1 year counts made by a mobile phone application: 4 locations, continuous measurement for 1 year Not applicable • The findings suggest some shifting of cyclist
• The increase in people using the app may have influenced the results
• Data on trips was analyzed, and it is unknown if the same cyclists were travelling more frequent, or if more cyclists were travelling
No other comparison groups • Intercept survey
• Direct observations
Law [48] (UK) C Not applicable No comparison group Before: direct observations: 21 locations, 1 day, 10 hAfter: automatic counters: 21 locations, 1 day,12 h Not applicable • The intervention effect is likely to be over-estimated due to seasonal differences
• Change in data collection methods may have influenced the results
No other comparison groups • Safety monitoring
Marques [49] (Spain) B Not applicable No comparison group Counts data, no description of the protocol Not applicable • Changes in population were not meaningful
• Changes in data procedures over time may have influenced the results
No other comparison groups • Safety monitoring
McCartney [50] (UK) B Not applicable No comparison group 560 h of digital video recordings manually checked: before and after at 5 locations, 4 days, 14 h Not applicable • Displacement effects were observed
• Weather and seasonality may have influenced the results
• Traffic conditions may have influenced the results
• The relative increase in cyclists in the intervention area was larger than in the city as a whole No other methods used
Merom [38] (US) B Not applicable No comparison group Automatic counters: 4 locations, continuous measurements for 5 months Not applicable No comments made No other comparison groups • Survey among residents
• Cycling behavior
• Campaign reach
Nguyen [51] (Singapore) A Not applicable No information on comparability Direct observations: few weekdays during peak periods, no precise description of the protocol Not applicable • Shifting of routes was observed
• No major change in land use
• Possibly reverse causation since segments that were improved had a high demand before the intervention
• The increase in cyclists was even larger on segments that were already improved before start of the current study • Survey among residents
• Intercept survey
Parker [52] (US) C Not applicable No comparison group 216 h of direct observations; Before: 1 location, 10 days, 9 h
After: 1 location, 14 days, 9 h
Not applicable • Displacement from other streets may have occurred
• It is possible that more people ride a bike because of the rising costs of car ownership
• The population increase may have influenced the results, but it is unlikely that this explains the total change in cycling
No other comparison groups No other methods used
Parker [53] (US) A Not applicable No information on comparability 660 h of direct observations: before and after at 3 location, 10 days, 11 h Not applicable • Some displacement of cyclists from nearby streets was observed
• Change in population size is unlikely to be the reason for the increase in cycling
No other comparison groups No other methods used
Wilmink and Hartman [42] (The Netherlands) A Not applicable No comparison group Count data, no description of the protocol: 250 locations Not applicable • Population growth may have influenced the findings No other comparison groups • Survey among residents
• Cycling behavior
  1. aNone of the studies was a randomized experiment, therefore randomization was not applicable for any of the studies and was not shown.
  2. bNone of the studies presented data for neutral outcomes that were hypothesized to be unaffected by the new infrastructure designed to promote cycling, therefore this parameter was not shown.
  3. cA = controlled before-after study; B = uncontrolled study with at least two before and two after data points; C = uncontrolled study with only 1 before and after data point