Reproducibility and comparative validity of a food frequency questionnaire for Australian children and adolescents
International Journal of Behavioral Nutrition and Physical Activity volume 6, Article number: 62 (2009)
Dietary intake during childhood and adolescence is of increasing interest due to its influence on adult health, particularly obesity, cardiovascular disease and diabetes. There is a need to develop and validate dietary assessment methods suitable for large epidemiologic studies of children and adolescents. Limited large scale dietary studies of youth have been undertaken in Australia, due partly to the lack of a suitable dietary intake tool. A self-administered, semi-quantitative food-frequency questionnaire (FFQ), the 'Australian Child and Adolescent Eating Survey' (ACAES), was developed for youth aged 9-16 years. This study evaluated reproducibility and comparative validity of the ACAES FFQ using assisted food records (FRs) as the reference method.
The ACAES FFQ was completed twice (FFQ1 and FFQ2) at an interval of 5 months, along with four one-day assisted FRs. Validity was evaluated by comparing the average of the FRs with FFQ2 (n = 113) as well as with the average of FFQ1 and FFQ2 (n = 101). Reproducibility was evaluated by comparing FFQ1 and FFQ2 (n = 101). The two methods were compared using correlations, Kappa statistics and Bland-Altman plots.
Correlation coefficients for comparative validity ranged from 0.03 for retinol to 0.56 for magnesium for transformed, energy-adjusted, deattenuated nutrient data, with correlation coefficients greater than 0.40 for total fat, saturated fat, monounsaturated fat, carbohydrate, sugars, riboflavin, vitamin C, folate, beta-carotene, magnesium, calcium and iron. Correlation coefficients for reproducibility ranged from 0.18 for vitamin A to 0.50 for calcium for transformed, energy-adjusted, deattenuated nutrient data. The ACAES FFQ ranked individuals reasonably accurately, with the comparative validity analysis showing that over 50% of participants were classified within one quintile for all nutrients, with only a small percentage grossly misclassified (0-7%).
The ACAES FFQ is the first child and adolescent specific FFQ available for ranking the dietary intakes of Australian children and adolescents for a range of nutrients in epidemiologic research and public health interventions.
Developing efficient, cost-effective and valid tools for assessing the dietary intakes of children and adolescents are key research priorities [1, 2]. The accurate assessment of dietary intake is critical to understanding the associations between food intake and obesity  as well as the influence of childhood dietary intake on chronic disease risk in adulthood [4–6]. The prevalence of obesity further necessitates the need for tools to assist in monitoring food consumption patterns. Ten per cent of the world's school-aged children are overweight or obese, with dramatically higher rates in developed regions, and prevalence continues to rise rapidly worldwide . In New South Wales, Australia, the prevalence of overweight and obesity in children and adolescents doubled between 1985 and 1997  with recent estimates at 25% .
Dietary assessment presents an ongoing challenge, particularly in large populations, with specific challenges for assessing the intakes of children and adolescents [10–12]. Although 24-hour recalls and food records (FRs) have been used successfully, the time and economic constraints of these methods make them unsuitable for most large scale studies . Food frequency questionnaires (FFQs) have a lower respondent burden, are less time-consuming, less intrusive, relatively inexpensive and do not require trained interviewers, rendering them more practical for large-scale epidemiologic studies .
There has been limited nutrition data collected in large, representative samples of Australian children and adolescents. Only three national surveys have been conducted in the past 50 years: the 1985 National survey of schoolchildren (aged 10-15 years) , the 1995 National Nutrition Survey  which included children aged 2-18 years and the 2007 Australian National Children's Nutrition and Physical Activity Survey (ages 2-16 years) . Yet the long-term nature of the development of diet-related chronic disease necessitates the collection of longitudinal dietary data. Frequency data can explain much of the variation in dietary intake and FFQs provide sufficient accuracy to relate individual diets in childhood and adolescence to subsequent health outcomes [18, 19]. FFQs have been used in adults to predict associations between dietary intake and disease specific mortality and morbidity including, colon cancer, heart disease and diabetes . In the US, the association between diet and health outcomes has been monitored in the Growing Up Today Study (GUTS), a longitudinal study of 16,882 9-14 year olds using the Youth-Adolescent Questionnaire (YAQ). Data collected in GUTS by the YAQ FFQ has been used to investigate the relationship between BMI and dietary intake , BMI and snack foods  and has demonstrated a relationship between BMI and intake of sugar-sweetened beverages  and BMI and family mealtimes . There is currently no widely accepted FFQ for school-aged children in Australia. The Australian Child and Adolescent Eating Survey (ACAES) was developed in response to these gaps.
The aim of this study was to assess the reproducibility and comparative validity of a semi-quantitative, self-completed FFQ designed for school-aged children aged 9 to 16 years in the Hunter region, New South Wales, Australia.
Development of the ACAES FFQ
The ACAES FFQ is based on the YAQ, validated in the US by Rocket et al. . Focus groups and pre-testing were undertaken with students aged 9-16 years to develop a comprehensive food list and refine the format of the FFQ. Five focus groups were held with a representative sample of 61 students: 31 primary students (9-12 years) and 30 secondary students (13-16 years), from two primary and two secondary schools. Students were asked what they usually ate for breakfast, snacks, lunch and dinner. A list was compiled and they were asked to add to this list until all the food and drinks usually consumed were included. Focus group results were used to modify the food list of the YAQ to reflect the Australian food supply, the local vernacular and the foods frequently consumed by the target population.
Once the food list was developed, a draft of the ACAES FFQ was pre-tested with a convenience sample of 41 primary and secondary students. Students were asked to complete the FFQ and circle anything that was not clear. On completion of the survey, the students discussed with a research assistant, any aspects of the FFQ they did not understand. As a result of the pre-testing, small modifications were made to the food list and ranges within response categories.
The final version of the ACAES FFQ is a 120-item FFQ with 15 supplementary questions regarding age, use of vitamin supplements, food behaviours and sedentary behaviours. The FFQ is designed as a self-administered tool, to collect information about the dietary intake of 9-16 year olds over the previous 6 months. An individual response for each food, or food type, is required, with frequency options ranging from 'Never' to '4 or more times per day', but varied depending on the food. The ACAES is semi-quantitative with a standard portion size provided for each food item and determined using 'natural' serving size (eg. slice of bread). In the absence of a natural serving size, portion sizes were derived from the 1995 National Nutrition Survey (NNS) (unpublished data purchased from the ABS). These data were for two age groups: 9-13 years and 13-16 years for boys and girls separately. Although boys had larger serve sizes than girls and the older age group had larger serve sizes than the younger, the differences were small. The average portion size for each item was calculated and used for all participants. This was the approach used by Rockett et al in the development of the Youth Adolescent Questionnaire (YAQ)  on which the ACAES FFQ was modelled. There were eight items without a 'natural' serving size or NNS data. For these foods, either FoodWorks 'Unspecified' serve sizes were used (5 items) or packet serve sizes (3 items). For composite items (those including more than one food) the NNS data was used and the serving size was weighted according to the NNS consumption data so that the foods consumed by the largest numbers of this age group were weighted more heavily. Due to the seasonal availability of some fruits, a separate section was included in the FFQ for seasonal fruit. The availability of seasonal fruit was determined by obtaining supermarket literature and contacting Sydney Markets, the largest fruit and vegetable market in New South Wales, which is 180 kilometres from Newcastle. Both sources indicated the number of months each year that seasonal fruit was available. The frequency categories for seasonal fruit were listed as for other food items and participants were asked to answer the question based on intake when the fruit is available. The average daily intake of seasonal fruit was calculated by adjusting for the number of months per year the fruit was available.
Participants for the comparative validity and reproducibility study
The protocol for this study was approved by the University of Newcastle Human Research Ethics Committee (Approval No. H-498-0203) and the NSW Department of Education and Training (SERAP No 03.48). A total of 224 students in years 4, 6, 8 and 10 (aged 9-16 years) from seven schools in the Hunter region, New South Wales, Australia, were invited to participate in the study. Two rural primary schools, two urban primary schools, two rural secondary schools and one urban secondary school were included. Socio-economic status was measured using the Socio-Economic Indexes for Areas (SEIFA) allocated on the basis of school postcode and compared to the NSW average for the Index of Relative Socio-Economic Advantage/Disadvantage .
Collection of dietary data
The ACAES FFQ was administered at the beginning and end of the five month study period (Figure 1). The reference method of four assisted food records (FRs) was undertaken during the same period. The protocol was designed to capture weekly and seasonal variation.
The ACAES FFQ was completed by the children and adolescents, at school. Blank forms for the FRs were distributed to participants at school two school days prior to each data collection visit. The students were asked to complete the FR for the following day (or the day after if it was to be recorded for a Sunday). When the research team visited the school two school days later, the completed FRs were reviewed individually with the participants to clarify all food types and quantities. Any participants that had not completed a FR, instead completed a 24-hour recall (37% of records) with a research assistant following a standardised proforma. The FRs and 24-hour recalls were completed for weekdays (75% of records) and weekends (25% of records) to capture differences in eating habits. To help participants give accurate portion sizes, incremental cup and spoon measures, glasses and a ruler were used. Data from FFQ1, FFQ2 and the FRs/24-hour recalls were entered and nutrient intakes computed in FoodWorks (Version 3.02.581) using the following databases: Australian AusNut 1999 database (All Foods) Revision 14 and AusFoods (Brands) Revision 5 .
A total of 63 FFQs and 10 FRs were excluded from the analysis due to implausible energy intakes. The cut-offs were less than 2,090 kilojoules or greater than 20,900 kilojoules for an individual record, as used in previous studies .
The distribution of intakes of all nutrients, except energy, were examined for extreme values using scatterplots generated for the average nutrient values of the FRs plotted against the average of FFQ1 and FFQ2. The original FRs and/or FFQs for these participants were checked and a decision made about whether or not to exclude the record from the analysis. The two main reasons for exclusion were more than 20 main meals a week reported on the FFQ (n = 5) and food records where intakes were reported to be 'less than usual' for 3 or 4 food records (n = 2).
FFQ records were excluded if more than 10% (12 items) of the questions did not have a response (n = 36), with the number of missing items ranging from 12 - 112. Food record data was not included in the analysis if participants had completed less than three FRs. A total of 61 food records were excluded from the analysis, for 37 participants that completed less than three food records.
All statistical analysis was performed using STATA version 8 (2003, Stata Corporation, College Station, TX, USA) .
The raw data for most nutrients were not normally distributed but were positively skewed, as commonly found with dietary data . Statistical and graphical assessment was used to determine the most appropriate transformation to achieve normal distribution. Logarithmic and square root transformations were attempted, with the square root transformation producing the most normalised distributions for all but a few nutrients. For simplicity, the square root transformation was used for all nutrients . The nutrients were energy-adjusted using the method of Willett and Stampfer .
As recommended [19, 32–34], comparative validity of the FFQ was evaluated by comparing the average of the FRs to FFQ2 as well as to the average of FFQ1 and FFQ2. Reproducibility was assessed by comparing the nutrient intake data from FFQ1 and FFQ2.
Means, standard deviations, medians and interquartile ranges were calculated for 22 nutrients. Spearman correlation coefficients were calculated using the unadjusted nutrient data. Pearson correlation coefficients were calculated on the data after statistical adjustment (transformation, energy-adjustment and de-attenuation). To adjust for within-person variation in the FRs, deattenuated correlation coefficients were calculated . Misclassification error was assessed using the Kappa statistic  to compare classification of nutrient data into quintiles. In most applications, the number of categories used for calculating kappa statistics varies from two to five [37, 38]. In validation studies of dietary intake, tertiles , quartiles  and quintiles  have all been used in the calculation of kappa statistics. The proportion grossly misclassified was also estimated.
The Bland Altman method was used to evaluate the agreement between the FFQ and FRs and between FFQ1 and FFQ2 . The Bland Altman method plots the individual differences between the two measurements against the mean of the measurements. For comparative validity, the difference between the average of FFQ1 and FFQ2 and the average of the food records [FFQavg-FRavg]s was plotted on the y-axis and the mean of the FFQ1/FFQ2 average and the FR average [(FFQavg-FRavg)/2] was plotted on the x-axis. For reproducibility, the difference between FFQ1 and FFQ2 [FFQ1-FFQ2] was plotted on the y-axis and the average of FFQ1 and FFQ2 [(FFQ1+FFQ2)/2] was plotted on the x-axis. These plots indicate the direction of bias and whether it is constant across levels of intake. The 'limits of agreement' or LOA (the mean difference ± 2SDs for the difference between the two measurements) determine whether the agreement between the two methods is acceptable . The differences were checked for normality as calculation of the 95% LOA assumes differences are normally distributed . Where Bland Altman plots showed a tendency for the differences to increase as the magnitude of the measurement increased, the data was log-transformed and replotted. If the relationship remained, the differences were regressed onto the means. Interpretation of the Bland-Altman results was based on the categories defined by Tang . 'Good agreement' is shown when the difference between the two measurements is approximately equal to one standard deviation of the average nutrient intake from the reference method. 'Fairly good agreement' is when the difference between the two measurements is approximately equal to two standard deviations of the average nutrient intake from the reference method and 'Bad (or poor) agreement' occurs if the difference between the two measurements is approximately equal to three standard deviations of the average nutrient intake from the reference method.
A total of 224 students were recruited to the study. Following data cleaning, records for 113 participants remained for the comparative validity analysis (50% of those recruited) using the assisted FRs and the FFQ2 data. Complete records for 101 participants remained for the validity analysis using the average of FFQ1 and FFQ2, with the same number for the reproducibility analysis (45% of those recruited). Table 1 reports the baseline characteristics for age, height, weight, BMI and BMI z-score for the study participants. There were no significant differences between the baseline characteristics of the participants and those excluded from the analysis.
The SEIFA codes for all schools were below the NSW average for the Index of Relative Socio-Economic Advantage/Disadvantage .
The median and interquartile ranges for energy and nutrients are presented in Table 2. The median values for FFQ2 and the FRs were exactly the same for monounsaturated fat, niacin equivalent, folate and iron. For all other nutrients, the values for FFQ2 were higher than those of the FRs, with the exception of polyunsaturated fat, carbohydrate, thiamin and vitamin C. The interquartile range of most nutrients tended to be wider for the FFQ data than the FRs. The dietary intakes estimated using FFQ1 were slightly higher than for FFQ2 (data not shown). This resulted in higher median values for all nutrients for the average of FFQ1 and FFQ2 when compared to the FRs.
Table 3 shows the correlation coefficients for comparative validity.
When the unadjusted data for FFQ2 and the average of the FRs was compared, Spearman's correlation coefficients ranged from 0.09 (retinol) to 0.35 (calcium). The median correlation for the unadjusted nutrient intakes of the two methods was 0.25. After transformation, the mean Pearson correlation was 0.24, ranging from 0.13 (niacin equivalent) to 0.37 (calcium). Deattenuation of the correlations for the transformed data increased the mean correlation to 0.27, ranging from 0.03 (retinol) to 0.41 (calcium). When the transformed data was energy-adjusted, the mean correlation coefficient increased to 0.34, with substantially larger coefficients for most nutrients except retinol, niacin equivalent, fibre and polyunsaturated fat. Deattenuation of the correlations for the transformed, energy-adjusted data increased the mean correlation to 0.39, ranging from 0.17 (niacin equivalent) to 0.56 (magnesium). The mean correlation for FFQ1 and the average of the FRs was higher than for FFQ2 for the crude data, the transformed data and the de-attenuated, transformed data (0.32, 0.34 and 0.38, respectively). However, the correlations for FFQ1 and the average of the FRs dropped following energy adjustment, resulting in lower correlations (data not shown) than for the energy-adjusted data for FFQ2.
Reproducibility was evaluated by calculating correlation coefficients for FFQ1 and FFQ2, as shown in Table 4. Spearman correlations for the unadjusted data ranged from 0.34 (sugars) to 0.53 (niacin equivalent). The median correlation for the unadjusted nutrient intakes of the two FFQs was 0.46. After transformation, the mean correlation using Pearson correlation coefficients was lower (0.44), ranging from 0.34 (sugars) to 0.51 (niacin equivalent). When the transformed data was energy-adjusted, the mean correlation coefficient decreased from 0.44 to 0.32, with substantially smaller coefficients for all nutrients, except magnesium and calcium which increased.
Table 5 shows the results of the kappa analysis undertaken as part of the comparative validity.
For the comparative validity analysis, the weighted kappa values for the nutrient intake data from FFQ2 and the average of the FRs ranged from 0.09 (retinol) showing 'slight' agreement, to 0.36 (iron), showing 'fair' agreement. All other nutrients showed 'slight' to 'fair' agreement. The proportion of individuals correctly classified into the same quintile was highest for sugars (31%) and lowest for zinc and folate (19%). The percentage classified within one quintile was highest for beta-carotene and calcium (65%) and lowest for zinc (51%). The percentage grossly misclassified (those ranked in the lowest quintile for the FFQ data but the highest quintile for assisted FRs) was small (1-7%).
For the reproducibility analysis of the nutrient intake data from FFQ1 and FFQ2, niacin (0.54) showed the strongest weighted kappa value and was classified as having 'moderate' agreement, while sugars had the lowest weighted kappa value (0.36) indicating 'fair' agreement. As shown in Table 6, with the exception of carbohydrate, sugars and vitamin A, all nutrients showed 'moderate' agreement with values between 0.41 and 0.60. The proportion of individuals correctly classified into the same quintile was highest for niacin equivalent (39%) and lowest for sugars (23%). The percentage classified within one quintile was highest for protein (79%) and lowest for vitamin A (63%). A very small percentage of individuals were grossly misclassified, ranging from none for Vitamin C and 5% for fibre.
When applying the Bland Altman method to the comparative validity data, the mean difference between the methods using the raw nutrient intake data was positive for most nutrients, indicating that when compared to FRs, the ACAES FFQ provides higher estimates for the intake of all nutrients, except polyunsaturated fat and thiamin.
Figure 2 shows the Bland-Altman plot for comparative validity analysis using the raw data for beta-carotene. The plot is scattered, indicating the bias is constant across levels of intake. The limits of agreement are approximately equal to two standard deviations of the FR data, showing fairly good agreement  between the FRs and the average of the FFQs.
The Bland Altman plots showed a tendency for the differences to increase as the magnitude of measurement increased for all nutrients, except beta-carotene, polyunsaturated fat, thiamin and vitamin C. The data for nutrients showing this trend were log transformed in an attempt to remove this relationship [42, 44]. For calcium, folate, sugars, riboflavin, thiamin and vitamin A this created a plot with a more consistent bias across levels of intake as shown in Figure 3 for calcium. However, the wide 95% LOA (antilog values of -53% to 336%) indicated discrepancies between the two methods for some individuals.
Log-transformation did not remove this relationship for energy, protein, total fat, saturated fat, monounsaturated fat, carbohydrate, fibre, niacin, retinol, iron and zinc. For these nutrients, the absolute residuals from a fitted regression line method was used  and this regression analysis showed significant relationships existed for these nutrients.
Bland Altman analysis of the reproducibility data showed a small, positive mean difference for all nutrients, indicating that FFQ1 tended to give slightly higher estimates of nutrient intake than FFQ2. The plots for all nutrients were scattered, showing consistent bias across all levels of intake. The LOA were wide (greater than ± 2SDs of the assisted FRs) indicating poor agreement between the two FFQs across the range of intakes.
Comprehensive FFQs have been developed to assess the total diets of children and adolescents in the United States , Norway , the United Kingdom  and Italy  with a limited number of validation studies undertaken. These studies demonstrated that for these populations, dietary intakes could be measured reasonably well using an FFQ. Differences in study methods, populations and between-person variation make it difficult to compare validity across studies internationally. However, it is worth noting that the results from the present study are similar to those of previous FFQ validation analyses for children and adolescents [25, 47, 49–53].
The validity of the ACAES FFQ was assessed by comparing the nutrient estimates from the average of the FRs with ACAES FFQ2 as well as comparing the average of the FRs with the average of ACAES FFQ1 and ACAES FFQ2. Although ACAES FFQ2 represents the conceptually appropriate time sequence, the process of keeping FRs might alter awareness of food intake and artificially improve accuracy in completing it . However, comparing ACAES FFQ1 with the FRs, would tend to underestimate validity because ACAES FFQ1 asked about diet prior to the study period. Therefore, the use of the average of ACAES FFQ1 and ACAES FFQ2 provides a combination of minimal and maximal estimates and is likely to be a more accurate indication of true validity .
Food frequency questionnaires tend to estimate higher nutrient intakes when compared to 24-hour recalls and FRs . This was shown in the present study, with the ACAES FFQ providing higher estimates of intakes for all nutrients, except polyunsaturated fat, carbohydrate, thiamin, niacin and vitamin C, when compared to assisted FRs.
The correlation results from this study are comparable to those for the YAQ . This was anticipated given that ACAES was modified from the YAQ and both studies had a similar design. When the correlation results for relative validity of the ACAES FFQ are compared to the YAQ , the correlation for carbohydrate is equivalent (r = 0.47), while the correlations for fat, saturated fat, folate, iron and riboflavin (Vitamin B2) are marginally lower. The correlations for beta-carotene, magnesium and zinc were higher for the ACAES FFQ compared to the YAQ. Correlations for all other nutrients were either lower (energy, protein, monounsaturated fat, fibre, thiamin, vitamin C, vitamin A, calcium), not significant (polyunsaturated fat, niacin and retinol) or not common to both studies (sugars, niacin equivalent).
The mean correlation coefficients for the common nutrients between both studies (without vitamin supplementation) were calculated, excluding those nutrients in our study with correlation coefficients that were not significant (>0.05) (polyunsaturated fat, niacin and retinol). Rockett observed a mean correlation of 0.42 between the 2nd YAQ and the mean of three 24-hour recalls , compared to 0.39 for this study, indicating similar correlations between the two methods in both studies.
When compared to the correlations for other comparative validity studies, the ACAES FFQ showed similar results. The ACAES FFQ had higher correlations than the Block Kids Questionnaire (BKQ) for fibre and calcium, but not for energy . Our study had almost identical correlations for calcium as two FFQs designed specifically to assess calcium intake in children and adolescents in the US and Italy [50, 48, 51]. As found in the present study, others have also shown poor agreement for vitamin A , polyunsaturated fat [52, 55] and protein [47, 53]. This is probably due to the large day-to-day variation in the intakes of these nutrients by children and adolescents, particularly girls. To reflect the usual intakes of vitamin A, polyunsaturated fat and protein, a greater number of recording days is necessary, with as many as 46 days recording required to estimate the usual vitamin A intake for females aged 5-17 years .
In order to detect associations between diet and disease, it is suggested that correlations need to be at least 0.3 or 0.4 . The present study had significant correlations greater than 0.3 for the deattenuated data of all nutrients except protein and vitamin A.
The reproducibility results of the YAQ  were similar to the present study. The mean correlation for the YAQ was 0.41 for the log-transformed, energy adjusted data for the 7 nutrients presented. For the 21 nutrients examined in the ACAES FFQ reproducibility analysis, the mean correlations were 0.46 for raw data, 0.44 for transformed, 0.32 for transformed, energy-adjusted data. In the present study, the correlations for many nutrients dropped following transformation and again following energy adjustment. The square-root transformation was applied to all nutrients due to its appropriateness for the majority of nutrients for the ACAES FFQ1 and FFQ2 data. However, it was not the most statistically appropriate transformation for some nutrients in ACAES FFQ2 and is likely to be the reason the correlation coefficients for these nutrients dropped following transformation. The reduction in the reproducibility correlations following energy adjustment has occurred in other studies of reproducibility [57–60] and is likely to be due to systematic errors of over- and under-estimation between ACAES FFQ1 and FFQ2. Although we have no way to assess this directly, adjustment for energy increases correlation coefficients when the variability in nutrient intake is related to energy intake, but results in lower correlations when the nutrient variability depends on systematic errors of overestimation and underestimation . The correlation coefficient for reproducibility for total fat intake in our study was higher for crude data (0.49) when compared to the reproducibility correlation (0.41) for an FFQ for children aged 2-5 years developed specifically to assess fat intake  demonstrating good validity for total fat intake.
Bland Altman analyses have not often been reported in studies comparing FFQs with diet records in children and adolescents . In the only other study reporting Bland Altman results for children (11-13 years) , the findings were similar. The mean differences were positive and of very similar magnitude to the current study, with the limits of agreement wide for all nutrients presented (energy, fat, sugar, calcium and protein). Similarly, the Bland Altman plots from the raw data showed strong trends of increasing difference with increasing intakes. The Bland Altman results for the present study show that the ACAES FFQ is not suitable for estimating absolute intakes for children and adolescents, but is appropriate for ranking intakes. The similarity of results between the present study and that of Lietz et al , suggests that large variation in the agreement between methods may be characteristic of child/adolescent populations and that these results may be due to the usual variability of dietary intake of children and adolescents. Agreement between FFQs and 24-hour recalls or food records may be lower in children and adolescents than in adults due to the greater day to day variation in their dietary intake . Adolescents have highly variable food patterns, with possibly half of the foods they eat varying greatly .
The reference method of choice for FFQ validation studies is weighed food records or diet records . Although 24-hour recalls have less respondent burden, their sources of error tend to be more correlated with the error in an FFQ due to reliance upon memory, conceptualisation of portion sizes and distortion of reported diet . There was a high proportion of children aged 9-12 years (71.7%) in the sample and it was expected that some of these participants would be less likely to complete a food record. The 24-hour recall method was undertaken with participants that had not completed their food records due to it's suitability to participants with limited cooperation or literacy . Interestingly, approximately the same proportion of younger children (9-12 years) and older children (13-16 years) completed 24-hour recalls (36.8% and 38.1% respectively) because they had not completed a food record.
Standard portion sizes were applied to the ACAES FFQ because frequency has been found to be more discriminatory than portion size [63, 64]. Adding open-ended questions regarding portion size can actually reduce validity of an FFQ due to sources of error in conceptualising serve sizes and large, within-person variations in serving sizes when the same food is consumed on different occasions .
A general limitation of validation studies is that the results are not necessarily transferable to another population . A sample size of at least 50 is desirable for each demographic group , and ideally between 100 and 200 participants . The sample size in the present study was inadequate to compare the validity and reproducibility of subsets for age, gender or BMI category. Performance of the ACAES FFQ would need to be tested in populations of different SES and ethnicity.
A total of 43 participants were excluded due to implausible energy intakes reported on the ACAES FFQ, representing 19% of the number recruited. A total of 36 of these were for high intakes, while seven were for implausibly low intakes. This is a high proportion and we feel that the main reason was the completion of the ACAES FFQ in a class environment. It seemed that by completing the FFQ as a group, there may have been more distractions compared to completing it on their own or with a parent present. Some of the excluded students marked the highest intake category for almost all food items. In a subsequent study by the same research team using the ACAES FFQ with primary school children, only 2 out of 60 participants (3.3% of the sample) were excluded due to implausible energy intakes (data not shown). These children completed the FFQ independently at the school, rather than in a class environment. In future studies, it is recommended that the ACAES FFQ is completed by the child independently, rather than in a class environment.
Despite these limitations, there are many aspects of the study design and analysis that are likely to contribute to an underestimation of the true validity and reproducibility of the ACAES FFQ. Food records were used predominantly (63% of records) and combined with 24-hour recalls (37% of records). Due to differences in the biases of food records and FFQs, the predominant use of food records is likely to underestimate validation coefficients when compared to 24-hour recalls as the reference method . The day of the week is also likely to influence the results of validation studies. In the present study, the proportion of records collected on weekdays (75%) and weekends (25%) is close to the actual proportion of weekdays and weekend days (71% and 29%, respectively). When comparing 14 days versus 2 days of food records in Australian children, Jenner et al found that correlation coefficients were slightly higher when both days of the two day records were taken on weekdays . It is possible that this is due to a perceived 'usual' intake during the week and greater variation on weekends. Participants reported a 'usual' intake for 64% of the records collected. Of the remaining 36% of records, half were reported as 'less than usual' with the other half reported as 'more than usual'. This large proportion of 'more' or 'less' than usual intakes may be typical of the intakes of this age group, but is likely to contribute to reduced agreement with the FFQ, where they record their perceived 'usual' intake. The conservative approach to excluding records from the data analysis also contributes to underestimating the validity of the FFQ. For example, respondents with food records that were inconsistent with their FFQs were not excluded if their responses were plausible. The most extreme examples of these were intakes of liver in the FFQ, but none reported on the FR days (resulting in particularly poor agreement for vitamin A and retinol) and reporting items in their food records that were not on the FFQ (eg. oysters, popcorn, slurpee, added sugar, chocolate topping, sherbet).
Although 24-hour recalls capture rich information on food consumption, they measure episodically consumed foods poorly. Recent statistical modelling has suggested that combining FFQ data with a limited number of 24-hour recalls may provide the most accurate method of estimating usual dietary intake at the individual level, supporting the use of both methods in national surveillance .
This study evaluates the strengths and limitations of a self-administered FFQ developed for Australian children and adolescents aged 9 to 16 years. When compared to assisted FRs, the ACAES FFQ overestimates nutrient intakes and is therefore not suitable for estimating absolute intakes for individuals in this age group. However, with the exception of polyunsaturated fat and retinol, it demonstrates an acceptable ability to correctly classify participants into quintiles of intake.
This FFQ has comparable results across a wide range of nutrients to other FFQs designed for children and adolescents evaluated for validity and reliability. It provides an important contribution to the tools available for assessing usual intakes in children and is a useful tool for estimating the dietary intakes of total fat, saturated fat, carbohydrate, sugars, fibre, vitamin C, folate, beta-carotene, calcium and iron in clinical practice, epidemiologic research and public health interventions among Australian youth and adolescents.
Potischman N, Cohen BE, Picciano MF: Dietary recommendations and identified research needs for The National Children's Study. J Nutr. 2006, 136 (3): 686-9.
Taren D, et al: Dietary assessment methods: where do we go from here?. Public Health Nutr. 2002, 5 (6A): 1001-3.
Davison KK, Birch LL: Childhood overweight: a contextual model and recommendations for future research. Obes Rev. 2001, 2 (3): 159-71. 10.1046/j.1467-789x.2001.00036.x.
Andersen LB, et al: The relationship between physical fitness and clustered risk, and tracking of clustered risk from adolescence to young adulthood: eight years follow-up in the Danish Youth and Sport Study. Int J Behav Nutr Phys Act. 2004, 1 (1): 6-10.1186/1479-5868-1-6.
WHO: Expert consultation on diet, nutrition and the prevention of chronic diseases. WHO Technical Report Series No. 916. 2003, WHO: Geneva
Dietz WH: The obesity epidemic in young children. Reduce television viewing and promote playing. Bmj. 2001, 322 (7282): 313-4. 10.1136/bmj.322.7282.313.
Lobstein T, Baur L, Uauy R: Obesity in children and young people: a crisis in public health. Obes Rev. 2004, 5 (Suppl 1): 4-104. 10.1111/j.1467-789X.2004.00133.x.
Booth ML, et al: The epidemiology of overweight and obesity among Australian children and adolescents, 1995-97. Aust N Z J Public Health. 2001, 25 (2): 162-9. 10.1111/j.1753-6405.2001.tb01840.x.
Booth M, et al: NSW Schools Physical Activity and Nutrition Survey (SPANS) 2004: Full Report. 2006, NSW Department of Health: Sydney
McPherson RS, et al: Dietary assessment methods among school-aged children: Validity and reliability. Preventive Medicine. 2000, 31: S11-S33. 10.1006/pmed.2000.0631.
Livingstone MB, Robson PJ, Wallace JM: Issues in dietary intake assessment of children and adolescents. Br J Nutr. 2004, 92 (Suppl 2): S213-22. 10.1079/BJN20041169.
Persson LA, Carlgren G: Measuring children's diets: evaluation of dietary assessment techniques in infancy and childhood. Int J Epidemiol. 1984, 13 (4): 506-17. 10.1093/ije/13.4.506.
Buzzard M: 24-hour dietary recall and food record methods. Nutritional Epidemiology. Edited by: Willett W. 1998, Oxford University Press: Oxford
Willett W: Food-frequency methods. Nutritional Epidemiology. Edited by: Willett W. 1998, Oxford University Press: Oxford
Department of Community Services and Health: National dietary survey of schoolchildren (aged 10-15 years): 1985. No. 2 Nutrient intakes. 1988, Canberra: AGPS
ABS: National Nutrition Survey: Nutrient Intakes and Physical Measurements. 1998, Australian Bureau of Statistics: Canberra
CSIRO: 2007 Australian National Children's Nutrition and Physical Activity Survey: Main Findings. Canberra. 2008
Rockett HRH, Colditz GA: Assessing diets of children and adolescents. American Journal of Clinical Nutrition. 1997, 65 (suppl): 116S-22S.
Willett W, Lenart E: Reproducibility and validity of food-frequency questionnaires. Nutritional Epidemiology. Edited by: Willett W. 1998, Oxford University Press: Oxford
Willett WC: Invited commentary: comparison of food frequency questionnaires. Am J Epidemiol. 1998, 148 (12): 1157-1159. 10.1006/pmed.2001.0850.
Rockett HR, et al: Cross-sectional measurement of nutrient intake among adolescents in 1996. Prev Med. 2001, 33 (1): 27-37. 10.1038/sj.ijo.0802762.
Field AE, et al: Snack food intake does not predict weight change among children and adolescents. Int J Obes Relat Metab Disord. 2004, 28 (10): 1210-6. 10.1038/oby.2004.94.
Berkey CS, et al: Sugar-added beverages and adolescent weight change. Obes Res. 2004, 12 (5): 778-88. 10.1038/oby.2005.104.
Taveras EM, et al: Family dinner and adolescent overweight. Obes Res. 2005, 13 (5): 900-6. 10.1006/pmed.1997.0200.
Rockett HR, et al: Validation of a youth/adolescent food frequency questionnaire. Prev Med. 1997, 26 (6): 808-16. 10.1016/S0002-8223(95)00086-0.
Rockett HR, Wolf AM, Colditz GA: Development and reproducibility of a food frequency questionnaire to assess diets of older children and adolescents. J Am Diet Assoc. 1995, 95 (3): 336-40. 10.1016/S0002-8223(95)00086-0.
Trewin D: Socio-Economic Indexes for Areas 2039.0. 2001, Canberra: Australian Bureau of Statistics
Xyris Software (Australia) Pty Ltd: FoodWorks Professional Version 3.02.581. Brisbane. 2004
StataCorp: Stata Statistical Software: Release 8.2. 2003, College Station, TX: Stata Corporation LP
Kirkwood BR, Sterne JAC: Essential Medical Statistics. 2003, Oxford: Blackwell Science, Second
Willett WC, Stampfer MJ: Total energy intake: Implications for epidemiologic analyses. American Journal of Epidemiology. 1986, 124 (1): 17-27. 10.1079/NRR200370.
Cade J, et al: Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr. 2002, 5 (4): 567-87. 10.2307/2529310.
Cade JE, et al: Food-frequency questionnaires: a review of their design, validation and utilisation. Nutrition Research Reviews. 2004, 17: 5-22. 10.1079/NRR200370.
Ministry of Agriculture Fisheries and Food Project: Consensus document on the development, validation and utilisation of food frequency questionnaires. 2000, Nuffield Institute for Health, University of Leeds Institute of Human Nutrition, University of Southampton
Willett W: Correction for the effects of measurement error. Nutritional Epidemiology. Edited by: Willett W. 1998, Oxford University Press: Oxford
Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-74. 10.1097/00001648-199603000-00016.
Sim J, Wright CC: The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005, 85 (3): 257-68. 10.1038/sj.ejcn.1602124.
Brenner H, Kliebsch U: Dependence of weighted kappa coefficients on the number of categories. Epidemiology. 1996, 7 (2): 199-202. 10.1079/BJN20051671.
Vereecken CA, et al: Young adolescents' nutrition assessment on computer (YANA-C). Eur J Clin Nutr. 2005, 59 (5): 658-67. 10.1079/PHN2004604.
Huybrechts I, et al: Validity and reproducibility of a semi-quantitative food-frequency questionnaire for estimating calcium intake in Belgian preschool children. Br J Nutr. 2006, 95 (4): 802-16. 10.1079/BJN20051671.
Flood VM, et al: Issues in assessing the validity of nutrient data obtained from a food-frequency questionnaire: folate and vitamin B12 examples. Public Health Nutr. 2004, 7 (6): 751-6. 10.1191/096228099673819272.
Bland JM, Altman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986, 1 (8476): 307-10.
Gibson RS: Principles of Nutritional Assessment. 2005, Oxford: Oxford University Press, Second
Bland JM, Altman DG: Measuring agreement in method comparison studies. Statistical Methods in Medical Research. 1999, 8: 135-160. 10.1017/S1368980099000762.
Tang HK: Diet, physical activity, environments and their relationship to the emergence of adolescent overweight and obesity in Ho Chi Minh City, Vietnam. PhD thesis. 2005, University of Newcastle: Faculty of Health
Andersen LF, et al: Evaluation of a quantitative food frequency questionnaire used in a group of Norwegian adolescents. Eur J Clin Nutr. 1995, 49 (8): 543-54. 10.1097/01.MPG.0000153004.53610.0E.
Robinson S, et al: Assessing the diet of adolescent girls in the UK. Public Health Nutr. 1999, 2 (4): 571-7. 10.1016/j.jada.2008.02.015.
Bertoli S, et al: Validation of food frequency questionnaire for assessing dietary macronutrients and calcium intake in Italian children and adolescents. J Pediatr Gastroenterol Nutr. 2005, 40 (5): 555-60. 10.1016/j.jada.2004.02.031.
Cullen KW, Watson K, Zakeri I: Relative reliability and validity of the Block Kids Questionnaire among youth aged 10 to 17 years. J Am Diet Assoc. 2008, 108 (5): 862-6. 10.1016/j.jada.2007.12.006.
Jensen JK, et al: Development of a food frequency questionnaire to estimate calcium intake of Asian, Hispanic, and white youth. J Am Diet Assoc. 2004, 104 (5): 762-9. 10.1016/j.jada.2004.02.031.
Wong SS, et al: Evaluation of a computerized food frequency questionnaire to estimate calcium intake of Asian, Hispanic, and non-Hispanic white youth. J Am Diet Assoc. 2008, 108 (3): 539-43. 10.1079/PHN2002344.
Stein AD, et al: Consistency of the Willett semiquantitative food frequency questionnaire and 24-hour dietary recalls in estimating nutrient intakes of preschool children. Am J Epidemiol. 1992, 135 (6): 667-77.
Lietz G, et al: Can the EPIC food-frequency questionnaire be used in adolescent populations?. Public Health Nutr. 2002, 5 (6): 783-9. 10.1186/1471-2458-5-135.
Block G, et al: Comparison of two dietary questionnaires validated against multiple dietary records collected during a 1-year period. J Am Diet Assoc. 1992, 92 (6): 686-93.
Baer HJ, et al: Use of a food frequency questionnaire in American Indian and Caucasian pregnant women: a validation study. BMC Public Health. 2005, 5: 135-10.1186/1471-2458-5-135.
Nelson M, et al: Between- and within-subject variation in nutrient intake from infancy to old age: estimating the number of days required to rank dietary intakes with desired precision. Am J Clin Nutr. 1989, 50 (1): 155-67. 10.1038/sj.ejcn.1601588.
Willett WC, et al: Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol. 1985, 122 (1): 51-65. 10.1590/S0102-311X2007000900026.
Slater B, et al: Validation of a semi-quantitative adolescent food frequency questionnaire applied at a public school in Sao Paulo, Brazil. Eur J Clin Nutr. 2003, 57 (5): 629-35. 10.1038/sj.ejcn.1601588.
Marchioni DM, et al: Reproducibility of a food frequency questionnaire for adolescents. Cad Saude Publica. 2007, 23 (9): 2187-96. 10.1006/pmed.2000.0701.
Rimm EB, et al: Reproducibility and validity of an expanded self-administered semiquantitative food frequency questionnaire among male health professionals. Am J Epidemiol. 1992, 135 (10): 1114-26.
Dennison BA, Jenkins PL, Rockwell HL: Development and validation of an instrument to assess child dietary fat intake. Prev Med. 2000, 31 (3): 214-24. 10.1006/pmed.2000.0701.
Frank GC, et al: A food frequency questionnaire for adolescents: defining eating patterns. J Am Diet Assoc. 1992, 92 (3): 313-8.
Hunter DJ, et al: Variability in portion sizes of commonly consumed foods among a population of women in the United States. Am J Epidemiol. 1988, 127 (6): 1240-9. 10.1093/ije/dyg310.
Flegal KM: Evaluating epidemiologic evidence of the effects of food and nutrient exposures. Am J Clin Nutr. 1999, 69 (6): 1339S-1344S.
Plummer M, Kaaks R: Commentary: An OPEN assessment of dietary measurement errors. Int J Epidemiol. 2003, 32 (6): 1062-3. 10.1016/j.jada.2006.07.002.
Jenner DA, et al: A comparison of methods of dietary assessment in Australian children aged 11-12 years. Eur J Clin Nutr. 1989, 43 (10): 663-73. 10.1002/sim.4780111005.
Subar AF, et al: The food propensity questionnaire: concept, development, and validation for use as a covariate in a model to estimate usual food intake. J Am Diet Assoc. 2006, 106 (10): 1556-63. 10.1016/j.jada.2006.07.002.
Cole TJ, Green PJ: "Smoothing reference centile curves: the LMS method and penalized likelihood.". Stat Med. 1992, 11 (10): 1305-19. 10.1002/sim.4780111005.
Cole TJ, Pan H: LMS Growth (computer program), Medical Research Council. 2002
This research project was funded by the Hunter Medical Research Institute; however, the views expressed in this manuscript are those of the authors. The authors acknowledge the nutrition and dietetics students who assisted with data collection; G Berry and S Noble for assistance with data entry; the support of the parents and staff at participating schools; and the students who participated in the study.
A scannable version of the ACAES FFQ is currently under development and will be available for research application.
JW collected the data, performed the statistical analysis and drafted the manuscript. JW, CC, DS, MD and MG developed the study, and participated in its design and coordination. CC and DS provided assistance for the statistical analysis and helped to draft the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Watson, J.F., Collins, C.E., Sibbritt, D.W. et al. Reproducibility and comparative validity of a food frequency questionnaire for Australian children and adolescents. Int J Behav Nutr Phys Act 6, 62 (2009). https://doi.org/10.1186/1479-5868-6-62