From: Challenges in conducting natural experiments in parks—lessons from the REVAMP study
Study specific experiences | Lessons learnt/implications for future studies |
---|---|
Partnerships: The inclusion of industry partners had multiple benefits including study ownership, greater insight into the intervention, and facilitation of research translation. | Consider the inclusion of partners prior to the commencement of the project. |
Flexibility: Extraneous factors affected the timing and delivery of the intervention in this natural experiment and changes were required to the timeline and study design. | These study designs need to be scientifically robust, yet flexible enough to cope with unpredictable events and a changing environment that is outside the control of the researcher. |
Identification of suitable controls: Identification of a control site with similar characteristics to the intervention site (e.g. size, features, area level SES), with no planned changes over the study period was challenging. | Although challenging, an adequate control site is essential to ensure experimental design. It may be necessary to relax the control site criteria rather than have no control site. |
Data collection: Factors such as weather, staffing and special events impacted data collection scheduling. | To ensure data collection is unaffected, establish clear study protocols on cancelling observation days, staff schedules, and have contingency plans when these events occur. |
Contingency budget: Additional costs were incurred due to changes outside of the researcher’s control (e.g. changes in timelines and rescheduling of data collection due to poor weather). | Incorporate contingency funding into research budgets for natural experiments or alternatively, funders should allow researchers to apply for additional funding to support unanticipated changes outside of their control. |
Timing of funding cycles: The timing of the natural experiment was carefully planned to align with university/government funding cycles. | Researchers need to plan in advance to allow for long lead times before the commencement of interventions; however, funders could have flexible funding rounds to accommodate natural experiment evaluations. |