Methodology | Open | Published:
Modified ground-truthing: an accurate and cost-effective food environment validation method for town and rural areas
International Journal of Behavioral Nutrition and Physical Activityvolume 13, Article number: 37 (2016)
A major concern in food environment research is the lack of accuracy in commercial business listings of food stores, which are convenient and commonly used. Accuracy concerns may be particularly pronounced in rural areas. Ground-truthing or on-site verification has been deemed the necessary standard to validate business listings, but researchers perceive this process to be costly and time-consuming. This study calculated the accuracy and cost of ground-truthing three town/rural areas in Minnesota, USA (an area of 564 miles, or 908 km), and simulated a modified validation process to increase efficiency without comprising accuracy. For traditional ground-truthing, all streets in the study area were driven, while the route and geographic coordinates of food stores were recorded.
The process required 1510 miles (2430 km) of driving and 114 staff hours. The ground-truthed list of stores was compared with commercial business listings, which had an average positive predictive value (PPV) of 0.57 and sensitivity of 0.62 across the three sites. Using observations from the field, a modified process was proposed in which only the streets located within central commercial clusters (the 1/8 mile or 200 m buffer around any cluster of 2 stores) would be validated. Modified ground-truthing would have yielded an estimated PPV of 1.00 and sensitivity of 0.95, and would have resulted in a reduction in approximately 88 % of the mileage costs.
We conclude that ground-truthing is necessary in town/rural settings. The modified ground-truthing process, with excellent accuracy at a fraction of the costs, suggests a new standard and warrants further evaluation.
The community food environment has been widely recognized as a key determinant of dietary behavior and weight outcomes among youth [1–6]. Several studies conducted in urban settings have linked poor quality food environments surrounding schools, such as convenience and other small food stores that sell sugar-sweetened beverages, to adverse diet and weight outcomes [4, 7–9]. Yet, findings have not always been consistent [10–13]. A number of substantial challenges with food environment assessment methodologies make it difficult to critically evaluate the existing body of research, as well as the credibility of their conclusions [3, 14].
In the U.S., the community food environment outside of urban areas (from here on referred to as town/rural areas) has remained understudied, even though these areas have demonstrated a dearth of healthy, high-quality foods [15–17]. Research on town/rural food environments in the U.S. has so far been concentrated in a few pockets [16–21], even while town/rural foodscapes are likely be heterogeneous and context-dependent. A better understanding of the role of town/rural food environments on health is warranted, particularly in light of a recent meta-analysis estimating that the odds of obesity are 26 % higher among youth in rural areas compared with their urban counterparts .
A central challenge in conducting food environment research is obtaining accurate food environment data. The most common sources of data listing food outlets are commercial business listings (e.g., Dun & Bradstreet, InfoUSA), which are business marketing tools not meant to be used in health research . These data are convenient and, by some counts, have fueled more than two-thirds of the growing body of research on the food environment and health [24, 25]. Yet, a number of studies call into question the accuracy of these data [20, 21, 23, 26, 27]. Store omissions on these business lists can lead to measurement error [14, 28], resulting in attenuation bias in regression estimates if omissions are random, and more complex bias if errors are more likely in certain store types. Even studies that report reasonable levels of list accuracy acknowledge that accuracy varies by the type of list used [29–31] and by store type [32, 33]. Discrepancies between secondary data listings and actual foodscapes may be particularly pronounced in town/rural area, where geocoding errors are more common , stores may close more frequently , and “non-traditional” sources of food like dollar stores may be more common [26, 35].
Researchers have increasingly called for ground-truthing as a necessary process to validate business lists [14, 21, 23, 35–37], which involves canvassing all of the streets within a geographic area to enumerate all existing food establishments. Few researchers actually carry this process out, perceiving it as costly and time consuming [38–40]. Of the studies included in a recent systematic review on the retail food environment around schools , only 5 out of 28 studies using food store lists reported conducting any kind of on-site validation. Few studies have quantified the time and resources needed to ground-truth , but Fleischhacker  reported that it took 20 data collection days to ground-truth 1502 miles (2417 km). More resource-efficient validation methods could encourage the adoption of higher validation standards among research teams, particularly those conducting community-based research, both in the U.S. and elsewhere.
Developing a method for characterizing the food environment in town/rural settings that is both cost-effective and valid is, therefore, a critical step in conducting rigorous studies that evaluate the link between the food environment and health in understudied and under-resourced settings. Such work is essential in order to address the current lack of evaluable research, quantify the impact of “obesogenic” environments on health outcomes, and identify opportunities for intervention. In order to address some of the barriers related to ground-truthing in the U.S., this study pilot-tested the cost and accuracy of a modified validation method, in order to accurately characterize the food environment in the town/rural setting in Minnesota.
During the summer of 2014, traditional ground-truthing was conducted in the areas surrounding three high-schools located 60–70 miles (97 – 113 km) outside of the Minnesota Twin Cities metropolitan area (Minneapolis and St. Paul), encompassing 564 miles (908 km) of road. The study area of interest was the food environment surrounding schools, as this study was conducted as part of a larger study examining school breakfast participation in rural Minnesota (Project BreakFAST) . The study area included a road network buffer of 3–5 miles (4.8 – 8 km) surrounding each school, which encompassed the area in which at least 80 % of the students enrolled in the study at the three schools lived. The three schools were selected from a total of 16 schools participating in the intervention study because they represented a range of school characteristics (e.g., school size ranged from 325 to 1605 students, 6 % to 17 % of whom were minority students). Two sites were classified as distant towns and the third site was classified as a remote town by the National Center for Education Statistics (NCES) locale codes.
Traditional ground –truthing
A list of stores in the study area was obtained from Esri’s Business Analyst (BA), a GIS business analytics system that relies on a list of more than 18 million U.S. businesses from Dun & Bradstreet. North American Industry Classification System (NAICS) retailer codes were used to extract a list of stores from BA that might reasonably sell food. Retailer types included supermarkets, grocery stores, supercenters, convenience stores, gas stations (with or without convenience stores), dollar stores, specialty stores, full-service/limited service restaurants, discount department stores, pharmacies/drug stores, and other miscellaneous retailers (e.g., food/health supplement stores and department stores). Because the focus on food environment features relevant to youth, liquor stores and bars were excluded prior to the analysis (though “bar and grille” restaurants were retained). Also excluded before the analysis were emergency food assistance providers (e.g., food banks) and impermanent retailers (e.g., farmer’s markets).
Road maps for the study area were created before data collection using ArcGIS 10.3. In teams of two, data collectors: 1) drove each street in the study area to identify food retail outlets; 2) logged food outlet geographic coordinates (longitude, latitude); and 3) conducted a “windshield survey” [42, 43] to correspond with each store, including the store outlet name, store type, address, hours open, and a storefront picture. Data collectors entered the store to determine whether the store sold food or beverages where it could not be determined from the store exterior (e.g., some gas-marts or gift shops). In this pilot study, several technology devices were tested while developing the ground-truthing protocol. The first data collection site used a portable Garmin navigational device to manually record waypoint positioning and a camera to capture a storefront image. The second two study areas used a Sky Pro GPS Receiver XGPS160 (a high-sensitivity, Wide Area Augmentation System–enabled GPS unit) paired with GPS Tracks HD (an iPad application connected to the GPS device via Bluetooth) to track and record route, positioning, and waypoints in real time.
At the conclusion of data collection at each site, the ground-truthed track history was compared against BA listings. Four classifications for stores emerged: (1) open/found, (2) new store, (3) not found, (4) ineligible. Stores were classified as open/found if they were found during ground-truthing and matched a BA store name and location. Consistent with a previous protocol , matches included exact matches, as well as close matches (e.g., Mizuki Fusion listed as Zhang Ke Mizuki Fusion), and lenient matches where both names suggested a similar vendor type and product line (e.g., Papa Murphy’s instead of Midwest Pizza Group, El Progresso Market instead of Texano Groceries). Addresses were compared to make sure matches were near the same intersection . Stores that were found during ground-truthing that had not appeared on the BA list were classified as new stores. Outlets on the BA list that were not found during ground-truthing (either because they were wrongly listed or because they were no longer present at that location) were classified as not found. Outlets that were found, but should not have been included as food stores, were deemed ineligible. This included exclusive establishments for specific populations, or establishments requiring special membership (e.g., institutionalized settings, cafeterias in hospitals, country clubs)  and stores that, upon visiting, were confirmed not to sell food.
Stores that were open/found were considered to be “true positives.” New stores were considered to be “false negatives.” Those not found, closed, or ineligible were all considered to be “false positives,” as these stores would likely have been erroneously assumed to be present, open, and relevant to the food environment if no validation had been done.
The positive predictive value (PPV) of the BA list was the probability that stores were located and open where they were listed on the BA list, calculated as true positives/(true positives + false positives). Sensitivity was the probability that open stores were listed on the BA list, calculated as true positives/(true positives + false negatives).
Cost metrics included two mileage measures: (1) ground-truthing mileage (2) total mileage (including traveling to and from the sites). Mileage costs were estimated at $0.565 per mile, the mileage reimbursement rate at the institution where the research took place. Cost metrics also included time (hours spent ground-truthing, miles per hour during ground-truthing, and total number of hours of field work for two data collectors). At the first site, mileage and time estimates were calculated from the car odometer and standard car clock; in the other sites, hours and miles per hour were automatically monitored by the HD Tracks app; tracking was paused during breaks.
While conducting ground-truthing, the researchers made a number of observations about the spatial patterning of store locations. They noted that nearly all stores were concentrated in a small number of commercial clusters; that stores listed outside these areas were likely to be false positives; and that most of the data collection time was spent driving through areas where there were no retailers (e.g., unpaved back roads, long stretches of remote country roads, pockets of housing developments with cul de sacs, and residential grids in town centers). This led to a hypothesis that a more efficient ground-truthing protocol could be developed for town/rural areas with minimal compromises in accuracy.
To test this hypothesis, a modified protocol was proposed. According to this protocol, data collectors would use the original BA lists to identify key, targeted areas (central commercial clusters) that were likely to yield the most information for validation. ArcGIS was used to simulate the accuracy (PPV and sensitivity) of this process by locating the ground-truthed stores that fell inside and outside central commercial clusters, defined in this study as the 1/8 mile (200 m) buffer around any cluster of at least two stores. Accuracy results (PPV and sensitivity) were recalculated, comparing the list of stores that would have been generated if only the central commercial areas had been ground-truthed. Two scenarios were considered: (1) a list comparison that assumed all stores outside the central commercial clusters were open; and (2) a list comparison that assumed all stores outside the central commercial clusters were closed.
Potential cost-savings were estimated for mileage and number of trips taken. It was not possible to estimate time savings since the rate of data collection (miles per hour) in central commercial clusters was likely substantially different from the rate in areas where there were no retailers.
Accuracy results are presented in Table 1. Findings showed that the PPV for the BA list ranged from 0.50 to 0.65 across the three sites (average 0.57). Sensitivity ranged from 0.50 to 0.70 (average 0.62). Out of 136 stores identified, 34 % were not on the BA list. Furthermore, 45 % of the stores on the BA list should not have been on the list because they were not found (n = 45) or ineligible (n = 16).
Under the modified ground-truthing scenario, when it was assumed that stores outside the targeted observation area were open, the PPV ranged from 0.82 to 0.93 (average 0.88) and the sensitivity was 0.96 to 1.00 (average 0.98). In this scenario, only 4 new stores (out of a total of 46) would have been missed. However, 21 false positives would still have been included on the list (out of a total of 61). When it was assumed that stores outside the observation area were closed, the PPV was perfect in all three sites (1.00) because there were no false positives. The sensitivity ranged from 0.92 to 1.00 (average 0.95).
Findings also showed that the ground-truthing process was time-intensive (Table 2). Six full days (averaging 9.5 h each) for two staff were needed to ground-truth 564 miles (908 km) of road. Ground-truthing also had additional “hidden” mileage costs. Back-tracking was required to reach all corners of the buffer zone, and these added 27 %, on average, to the mileage. Additionally, data collection required travel to and from the sites; when data could not be completed in one day, additional mileage to and from the site was accrued. Including back-tracking and to-from transit, 1510 total miles (2430 km) were driven to ground-truth 564 miles (908 km) of road.
Modified ground-truthing would have resulted in a road-network distance of 66 miles (106 km) to be ground-truthed (a savings of 88 %). The process would have required only one visit per site (a 50 % savings in data collection trips). Assuming a similar amount of back-tracking as in traditional gound-truthing (an extra 27 %), the modified process would have resulted in an estimated total of 460 miles (740 km) driven (a 70 % reduction in total miles) compared with traditional ground-truthing.
Conducting traditional ground-truthing in three town/rural sites revealed that on-site validation is, indeed, necessary for accurate analyses of food retail environments. The process also revealed that the current gold-standard of ground-truthing methods is not an efficient method of validation. The positive predictive value and sensitivity observed in the current study are similar or slightly lower than those previously reported in similar studies [20, 27, 29, 36]. For instance, one study reported that the PPV for all stores outside of urban areas using Dun & Bradstreet ranged from 0.67 to 0.78, while the sensitivity fell in the range on 0.54 to 0.65 . Yet, despite the necessity of validation, the equivalent of nearly three weeks of full-time work (114 total staff hours) were needed to ground-truth a relatively small study area and identify 46 new stores.
Additionally, systematic canvassing is tedious work, particularly in areas where stores are few and far between. Streamlining the process without compromising accuracy would allow researchers to place much-needed resources into other components of the research project. This type of streamlining would also make the validation process feasible for community-based organizations conducting assessments on community food environments, and could even make use of citizen science for collecting or verifying data in very remote areas.
A modified ground-truthing process in which only the central commercial clusters were validated would have resulted in substantial time and monetary savings. Results estimated an 88 % reduction in the total number of roads to validate and mileage costs, as well as a 50 % reduction in data collection trips. Further, this simulation suggested that the modified process would have resulted in only very modest compromises in accuracy. These savings would be possible because so few stores were actually found outside of central commercial zones in rural areas. Our results demonstrated that assuming that stores not directly observed within clusters were closed offered the best overall best accuracy. This process demonstrated perfect PPV and only a small compromise in sensitivity (0.95).
Other validation techniques, such as remote-sensing (e.g., Google Street View), have also been touted as cost-effective validation tools [38, 40]. In urban areas, the reliability of such tools has been variable [32, 37, 39, 44]. Outside urban areas, however, use of these methods may present particular challenges . Not all streets can be visualized via remote-sensing , and store closings are more common in rural areas , meaning that images may not be current. Date stamps are becoming more common on remote sensing images in the U.S., but images still may be out of date or misaligned with the health data to which it is being linked . Visualizing shopping centers (which are often a cornerstone of commerce outside of urban areas and often have a haphazard spatial arrangement) may also be problematic due to image disruption and lack of image continuity . Until issues with remote sensing can be resolved and evidence of their accuracy in rural areas can be demonstrated, modified ground-truthing offers promise as a cost-effective and valid method for creating accurate exposure metrics of the food environment. In conducting food environment research based on business lists, one cost that remains constant across validation methods (traditional ground-truthing, modified ground-truthing, remote sensing or no ground-truthing) is the cost of obtaining business listings. Commercial databases like Dun & Bradstreet are one choice for business lists, and may be licensed to some institutions at a relatively affordable price, but it should be noted that other economical options for data acquisition exist. For instance, administrative data on licensed food outlets may be obtained for free from government agencies (e.g., local health departments or state agriculture departments), and in some cases may be the most reliable [20, 29]. Reliable food environment metrics are needed to accurately estimate the relationship between the food environment and dietary behaviors and health outcomes. In the current body of literature, an abundance of mixed findings on the food environment-diet relationship [3, 11, 23, 24] have led to excessive replication of studies with flawed exposure measures. As a result, the current literature offers few clear conclusions that can be translated into evidence-based policy or interventions for improving nutrition environments in rural areas.
While promising, the modified ground truthing procedure tested in this study requires further exploration. Next steps for research might include testing the appropriateness of this protocol in both more rural and more urban areas. In the most remote rural areas, a larger buffer distance for ground-truthing might be required. In urban areas, the value of using modified ground-truthing might depend on the spatial arrangement of urban food retailers. For instance, modified ground-truthing might be less cost-effective in urban areas with a dense, regular patterning of stores (e.g., New York City ), but more cost-effective in areas where stores tend to cluster in certain areas (e.g.,cities with greater sprawl, suburban areas). Once ground-truthing protocols are established, an important next step might be to test the feasibility of adding a brief checklist that reflects store healthy food availability in the modified ground-truthing process. Currently, researchers often designate food retailers as “unhealthy” or “healthy” based solely on their store type, without regard to what they actually sell [14, 23, 24, 35, 49, 50]. As long as data collectors are visiting stores, adding a modified NEMS-S and NEMS-R with just 9 or 16 items  might be one way to gather a contained amount of information that could be used to create more nuanced geographic exposure measures, although the added time required to do this would need to be evaluated.
The results of this pilot study should be considered within the study limitations. First, the results of the modified ground-truthing process use simulations only; actually conducting modified ground-truthing could determine whether there were unanticipated costs or challenges associated with the method. For instance, if central commercial areas were geographically dispersed, costs would include substantial unforeseen mileage. Another limitation was that, given the small study area and sample size, we did not report the PPV and sensitivity by store type, even though previous studies have indicated differences by store type [23, 26–28]. Next, this was a small pilot study conducted in one region of Minnesota and represents a small geographic area. As such, store geography may not be representative of more remote rural areas, or of town/rural areas in other parts of the country or other countries. Despite limited generalizability, modified ground-truthing is a practical idea that could be adapted to other regions, both within the U.S. and outside, with a relatively simple assessment of local store geography – for instance, widening the buffer distance if stores are more dispersed. Additionally, it should be noted that ground-truthing is only useful for generating food environment variables that measure residents’ potential exposure to food outlets ; actual, realized exposure of the food environment, which might be measured by GPS tracking or wearable cameras , might be more directly relevant to behaviors and health outcomes. While acknowledging this as a limitation, we also wish to recognize that the broader study goal was to advance methods for determining spatial exposures, given that researchers do not always have the resources for detailed tracking of individuals, and reliance on spatial measures shows no signs of slowing.
Finally, one of the limitations of ground-truthing is that it can only capture the present environment. Some of the mismatch between business lists and ground-truthed lists is likely due to temporality – for instance, stores that were once open, but closed before the researchers visited. Temporality is, therefore, a component of validity that must be considered, especially when linking food environment measures to health measures. When linking older health measures to the food environment, ground-truthing may yield inaccurate exposure measures due to temporal mismatch. Ideally, food environments should be validated as close as possible to the time that health measures are collected.
Taken together with other literature, results from this study of three town/rural areas in Minnesota indicate that an on-site validation process is, indeed, a necessary step in avoiding list errors when conducting community food environment research. Excellent accuracy can be achieved through careful selection of key areas to focus validation efforts, indicating that a modified process could become a new standard for validation. It is unclear to what extent criteria for validating stores may vary in different types of town/rural settings. Given the current reliance on commercial business listings in public health research, such exploration would be a worthwhile investment, particularly for research conducted in low-resource community settings.
Gittelsohn J, Kumar MB. Preventing childhood obesity and diabetes: is it time to move out of the school? Pediatr Diabetes. 2007;8 Suppl 9:55–69.
Jennings A, Welch A, Jones AP, Harrison F, Bentham G, van Sluijs EMF, Griffin SJ, Cassidy A. Local food outlets, weight status, and dietary intake: associations in children aged 9–10 years. Am J Prev Med. 2011;40:405–10.
Larson N, Story M, Nelson M. Neighborhood environments disparities in access to healthy foods in the US. Am J Prev Med. 2009;36:74–81.
Laska MN, Hearst MO, Forsyth A, Pasch KE, Lytle L. Neighbourhood food environments: are they associated with adolescent dietary intake, food purchases and weight status? Public Health Nutr. 2010;13:1757–63.
Powell LM, Auld MC, Chaloupka FJ, O’Malley PM, Johnston LD. Associations between access to food stores and adolescent body mass index. Am J Prev Med. 2007;33:S301–7.
Swinburn B. Obesity Prevention in Children and Adolescents. Child Adolesc Psychiatr Clin N Am. 2009;18:209–23.
He M, Tucker P, Irwin JD, Gilliland J, Larsen K, Hess P. Obesogenic neighbourhoods: the impact of neighbourhood restaurants and convenience stores on adolescents’ food consumption behaviours. Public Health Nutr. 2012;15:2331–9.
Langellier BA. The food environment and student weight status, Los Angeles County, 2008–2009. Prev Chronic Dis. 2012;9, E61.
Tang X, Ohri-Vachaspati P, Abbott JK, Aggarwal R, Tulloch DL, Lloyd K, Yedidia MJ. Associations between Food Environment around Schools and Professionally Measured Weight Status for Middle and High School Students. Child Obes Print. 2014;10(6):511–7.
An R, Sturm R. School and residential neighborhood food environment and diet among California youth. Am J Prev Med. 2012;42:129–35.
Williams J, Scarborough P, Matthews A, Cowburn G, Foster C, Roberts N, Rayner M. A systematic review of the influence of the retail food environment around schools on obesity-related outcomes. Obes Rev Off J Int Assoc Study Obes. 2014;15:359–74.
Shier V, An R, Sturm R. Is there a robust relationship between neighbourhood food environment and childhood obesity in the USA? Public Health. 2012;126:723–30.
Timperio A, Ball K, Roberts R, Campbell K, Andrianopoulos N, Crawford D. Children’s fruit and vegetable intake: associations with the neighbourhood food environment. Prev Med. 2008;46:331–5.
Lucan SC. Concerning limitations of food-environment research: A narrative review and commentary framed around obesity and diet-related diseases in youth. J Acad Nutr Diet. 2015;115(2):205–212.
Liese AD, Weis KE, Pluto D, Smith E, Lawson A. Food store types, availability, and cost of foods in a rural environment. J Am Diet Assoc. 2007;107:1916–23.
Bustillos B, Sharkey JR, Anding J, McIntosh A. Availability of more healthful food alternatives in traditional, convenience, and nontraditional types of food stores in two rural Texas counties. J Am Diet Assoc. 2009;109:883–9.
Dean WR, Sharkey JR. Rural and urban differences in the associations between characteristics of the community food environment and fruit and vegetable intake. J Nutr Educ Behav. 2011;43:426–33.
Sharkey JR, Johnson CM, Dean WR, Horel SA. Association between proximity to and coverage of traditional fast-food restaurants and non-traditional fast-food outlets and fast-food consumption among rural adults. Int J Health Geogr. 2011;10:37.
Jilcott SB, McGuirt JT, Imai S, Evenson KR. Measuring the Retail Food Environment in Rural and Urban North Carolina Counties. J Public Health Manag Pract. 2010;16:432–40.
Fleischhacker SE, Rodriguez DA, Evenson KR, Henley A, Gizlice Z, Soto D, Ramachandran G. Evidence for validity of five secondary data sources for enumerating retail food outlets in seven American Indian Communities in North Carolina. Int J Behav Nutr Phys Act. 2012;9:137.
Gustafson AA, Lewis S, Wilson C, Jilcott-Pitts S. Validation of food store environment secondary data source and the role of neighborhood deprivation in Appalachia, Kentucky. BMC Public Health. 2012;12:688.
Johnson JA, Johnson AM. Urban–rural Differences in Childhood and Adolescent Obesity in the United States: A Systematic Review and Meta-Analysis. Child Obes. 2015;11(3):233–41.
Lucan SC, Maroko AR, Bumol J, Torrens L, Varona M, Berke EM. Business List vs Ground Observation for Measuring a Food Environment: Saving Time or Waste of Time (or Worse)? J Acad Nutr Diet. 2013;113:1332–9.
Caspi CE, Sorensen G, Subramanian SV, Kawachi I. The local food environment and diet: A systematic review. Health Place. 2012;18:1172–87.
McKinnon R, Reedy J, Morrissette M, Lytle L, Yaroch A. Measures of the Food Environment A Compilation of the Literature, 1990–2007. Am J Prev Med. 2009;36:S124–33.
Liese AD, Colabianchi N, Lamichhane AP, Barnes TL, Hibbert JD, Porter DE, Nichols MD, Lawson AB. Validation of 3 food outlet databases: completeness and geospatial accuracy in rural and urban food environments. Am J Epidemiol. 2010;172:1324–33.
Powell LM, Han E, Zenk SN, Khan T, Quinn CM, Gibbs KP, Pugach O, Barker DC, Resnick EA, Myllyluoma J, Chaloupka FJ. Field validation of secondary commercial data sources on the retail food outlet environment in the U.S. Health Place. 2011;17:1122–31.
Cummins S, Macintyre S. Are secondary data sources on the neighbourhood food environment accurate? Case-study in Glasgow, UK. Prev Med. 2009;49:527–8.
Fleischhacker SE, Evenson KR, Sharkey J, Pitts SBJ, Rodriguez DA. Validity of Secondary Retail Food Outlet Data A Systematic Review. Am J Prev Med. 2013;45:462–73.
Hosler AS, Dharssi A. Identifying retail food stores to evaluate the food environment. Am J Prev Med. 2010;39:41–4.
Lake AA, Burgoine T, Greenhalgh F, Stamp E, Tyrrell R. The foodscape: classification and field validation of secondary data sources. Health Place. 2010;16:666–73.
Bader MDM, Ailshire JA, Morenoff JD, House JS. Measurement of the local food environment: a comparison of existing data sources. Am J Epidemiol. 2010;171:609–17.
Han E, Powell LM, Zenk SN, Rimkus L, Ohri-Vachaspati P, Chaloupka FJ. Classification bias in commercial business lists for retail food stores in the U.S. Int J Behav Nutr Phys Act. 2012;9:46.
Kravets N, Hadden WC. The accuracy of address coding and the effects of coding errors. Health Place. 2007;13:293–8.
Sharkey JR. Measuring potential access to food stores and food-service places in rural areas in the U.S. Am J Prev Med. 2009;36(4 Suppl):S151–155.
Liese AD, Barnes TL, Lamichhane AP, Hibbert JD, Colabianchi N, Lawson AB. Characterizing the Food Retail Environment: Impact of Count, Type, and Geospatial Error in 2 Secondary Data Sources. J Nutr Educ Behav. 2013;45:435–42.
Rossen LM, Pollack KM, Curriero FC. Verification of retail food outlet location data from a local health department using ground-truthing and remote-sensing technology: Assessing differences by neighborhood characteristics. Health Place. 2012;18:956–62.
Rundle AG, Bader MDM, Richards CA, Neckerman KM, Teitler JO. Using Google Street View to audit neighborhood environments. Am J Prev Med. 2011;40:94–100.
Clarke P, Ailshire J, Melendez R, Bader M, Morenoff J. Using Google Earth to conduct a neighborhood audit: reliability of a virtual audit instrument. Health Place. 2010;16:1224–9.
Charreire H, Mackenbach JD, Ouasti M, Lakerveld J, Compernolle S, Ben-Rebah M, McKee M, Brug J, Rutter H, Oppert J-M. Using remote sensing to define environmental characteristics related to physical activity and dietary behaviours: a systematic review (the SPOTLIGHT project). Health Place. 2014;25:1–9.
Nanney MS, Shanafelt A, Wang Q, Leduc R, Dodds E, Hearst M, Kubik M, Grannon K, Harnack L. Project BreakFAST: Rationale, design, and recruitment and enrollement methods of a randomized controlled trial to evaluate an intervention to improve school breakfast participation in rural high schools. Contemporary Clinical Trials Communication. 2016;3:12–22.
Sharkey JR, Horel S, Han D, Huber Jr JC. Association between neighborhood need and spatial access to food stores and fast food restaurants in neighborhoods of colonias. Int J Health Geogr. 2009;8:9.
Sharkey JR, Horel S. Neighborhood Socioeconomic Deprivation and Minority Composition Are Associated with Better Potential Spatial Access to the Ground-Truthed Food Environment in a Large Rural Area. J Nutr. 2008;138:620–7.
Ben-Joseph E, Lee JS, Cromley EK, Laden F, Troped PJ. Virtual and actual: relative accuracy of on-site and web-based instruments in auditing the environment for physical activity. Health Place. 2013;19:138–50.
Ward MH, Nuckols JR, Giglierano J, Bonner MR, Wolter C, Airola M, Mix W, Colt JS, Hartge P. Positional accuracy of two methods of geocoding. Epidemiol Camb Mass. 2005;16:542–7.
Kelly CM, Wilson JS, Baker EA, Miller DK, Schootman M. Using Google Street View to audit the built environment: inter-rater reliability results. Ann Behav Med Publ Soc Behav Med. 2013;45 Suppl 1:S108–112.
Griew P, Hillsdon M, Foster C, Coombes E, Jones A, Wilkinson P. Developing and testing a street audit tool using Google Street View to measure environmental supportiveness for physical activity. Int J Behav Nutr Phys Act. 2013;10:103.
Curtis JW, Curtis A, Mapes J, Szell AB, Cinderich A. Using Google Street View for systematic observation of the built environment: analysis of spatio-temporal instability of imagery dates. Int J Health Geogr. 2013;12:53.
Creel JS, Sharkey JR, McIntosh A, Anding J, Huber JC. Availability of healthier options in traditional and nontraditional rural fast-food outlets. BMC Public Health. 2008;8:395.
Janevic T, Borrell LN, Savitz DA, Herring AH, Rundle A. Neighbourhood food environment and gestational diabetes in New York City. Paediatr Perinat Epidemiol. 2010;24:249–54.
Partington SN, Papakroni V, Menzies T. Optimizing data collection for public health decisions: a data mining approach. Bmc Public Health. 2014;14:593.
Cowburn G, Matthews A, Doherty A, Hamilton A, Kelly P, Williams J, Foster C, Nelson M. Exploring the opportunities for food and drink purchasing and consumption by teenagers during their journeys between home and school: a feasibility study using a novel method. Public Health Nutr. 2016;19:93–103.
This research was supported by the National Institutes of Health (grant number R25CA163184). The research presented in this paper is that of the authors and does not reflect the official policy of the NIH. Partial article contents were presented at a poster session at the American Public Health Association Annual Meeting & Exposition in November 2015. No financial disclosures were reported by the authors of this paper.
The authors declare that they have no competing interests.
CC conceptualized the study, oversaw data collection and analyses, and drafted the manuscript. RF led data collection, data cleaning, and analysis, documented all methods, and contributed to the final draft of the manuscript. All authors read and approved the final manuscript.