Measuring physical activity among pregnant women using a structured one-week recall questionnaire: evidence for validity and reliability

PIN3 Physical Activity Questionnaire

A one-week recall questionnaire was developed to evaluate physical activity among pregnant women enrolled in the PIN3 Study. The questionnaire was interviewer administered and designed to capture moderate and vigorous physical activity. It is available as additional file 1. The questionnaire can be contrasted with an interviewer-administered 7-day recall [14], but instead of recalling day-by-day the participant recalls mode-by-mode. The PIN3 physical activity questionnaire assessed frequency and duration of all moderate and vigorous physical activities the woman participated in, including activity done for recreation, at work, for transportation, childcare, adult care, and both indoor and outdoor household activities. Using recreational activity as an example, the question asked: "In the past week, did you participate in any non-work, recreational activity or exercise, such as walking for exercise, swimming, or dancing, that caused at least some increase in breathing and heart rate?" If the participant responded 'Yes', then the interviewer asked her to list all types of activities, one by one, with the following question: "What type of recreational activities did you do during the past week?" For each activity, the participant reported the number of sessions per week, duration of each session, and the perceived intensity level using the following options: 'fairly light,' 'somewhat hard,' and 'hard or very hard'. These intensity categories corresponded to the Borg scale of perceived exertion [15]. These questions were repeated for work, transportation, child and adult care giving, indoor household, and outdoor household activity.

The scoring of the questionnaire included assessment of intensity of activity in two ways. First, perceived (or relative) intensity using the modified Borg scale [15] to capture the participant's perception of intensity and to derive number of hours in the past week spent in moderate to vigorous physical activity, We classified activities reported as "somewhat hard" as moderate and activities reported as "hard or very hard" as vigorous. Second, absolute intensity classifying activities using published metabolic (MET) tables [11, 12] to determine the number of MET-hours per week spent in physical activity. A number of activities reported by women were not listed in the compendium. Thus, two raters determined the MET intensity based on other activities in the compendium; when the two raters disagreed they met and resolved by consensus. The final compendium of activities used for scoring is available elsewhere http://www.cpc.unc.edu/projects/pin/design_pin3/docs_3/PIN-MET-Table-080207.pdf. To determine moderate activity using absolute intensity, we defined moderate intensity as both 3 to 6 METS, a definition often used for adults [16], and 4.8 to 7.1 METS, the estimated values for moderate activity specific to adults 20 to 39 years of age [17].

The questionnaire took on average 10-20 minutes to administer. Intra- and inter-interviewer quality control measures, such as expert review of taped interviews, were established to ensure that interviewers were asking questions reliably and systematically.

Concurrent-Related Validity Assessment

For this portion of the study, pregnant women from central North Carolina were recruited by placing flyers in local clinics and screened over the telephone. Women were not enrolled if they were non-English speaking, under the age of 18 years, carrying multiple gestations, did not have a telephone from which they could complete the phone interviews, or more than 28 weeks' gestation during the screening call. The women wore an accelerometer for one week, kept a daily structured diary, and following this completed a questionnaire that included the one-week recall of the PIN3 physical activity questionnaire. Participants were paid $50 at the completion of the study; later in the study an electromagnetic field monitor was added to the protocol [18] and participants were paid $75.

Test-Retest Reliability Assessment

For this portion of the study, we relied on a sample of PIN3 participants. The PIN3 Study recruited pregnant women less than 20 weeks' gestation seeking prenatal care at clinics associated with the University of North Carolina Hospitals. Trained staff identified women through review of all medical charts of new prenatal patients. Women were not enrolled if they were non-English speaking, under the age of 16 years, carrying multiple gestations, not planning to continue care or deliver at the study hospital, or did not have a telephone from which they could complete the phone interviews. Selected women were asked to participate in two telephone interviews, at 17-22 and 27-30 weeks' gestation. The study website http://www.cpc.unc.edu/pin provides greater detail on the protocols and measures.

We evaluated the physical activity questionnaire test-retest reliability among pregnant women within 48 hours of their telephone interview completion as part of the PIN3 Study. We enrolled women purposefully, with approximately equal numbers in six strata: (i) completing the first (17-22 weeks) or second (27-30 weeks) telephone interview and (ii) self-report of either being currently employed in a strenuous job, active at leisure, or inactive at either work or leisure. During the second interview, women were asked to recall the same week that the first interview was conducted, so that recall periods matched. Women were paid $5 for participation in each interview.

Other Measures

During the first interview for both the validity and reliability samples, women were asked to report their education, current work status, race, general health, and parity (live plus still births). For the reliability sample only, age was reported at the interview and the medical record was abstracted and included self-reported weight and measured height for the determination of pre-pregnancy body mass index (BMI). BMI values were grouped using the Institute of Medicine recommendations for pregnant women, in effect during that time period, into low (<19.8 kg/m²) or normal weight (19.8-<26.0 kg/m²), overweight (26.0-<30.0 kg/m²), and obese (> = 30.0 kg/m²) [32]. For the validity samples, age was obtained during recruitment and calculated based on her date of birth as her age on the interview date.

Statistical Analysis

All analyses were conducted using SAS version 9.1 (Cary, NC). Evidence for validity was assessed using Spearman correlation coefficients (SCC) with 95% confidence intervals (CI) comparing either the diary or accelerometer results to the questionnaire. We conducted analyses comparing the accelerometer to the questionnaire using the full sample, the imputed sample, and the smaller sample defined by complete accelerometry data. Bland-Altman plots [33] were used to assess agreement between physical activity measurements from the questionnaire with either the accelerometer or the diary among the full sample. On the plots, the x-axis is the average of two measurements, while the y-axis is the difference between the two measurements. Horizontal lines represent the upper and lower limits of agreement and the mean difference of the two measurements. In addition, Pitman's test of differences in the variances was calculated for each comparison [34].

Test-retest reliability was assessed using intraclass correlations coefficients (ICC) with corresponding 95% CI. ICC were calculated using a two-way analysis of variance model and conceptualized as the proportion of the total variance explained by between-participant variance. Bland-Altman plots [33] were used to compare questionnaire results at the two time periods. Stratified analyses were performed to determine if differences in reliability differed by gestational age at interview date, race, education, age, parity, or pre-pregnancy BMI. As a rough guide, we followed the ratings suggested by Landis and Koch [35] for agreement level: 0-0.20 slight, 0.21-0.40 fair, 0.41-<0.60 moderate, 0.61-<0.80 substantial, and 0.81-<1.00 almost perfect. Each physical activity entry was screened and outliers were pairwise deleted from the analysis.

Description of Participants

In total, 177 pregnant women participated in the validity study and 109 pregnant women participated in the reliability study. For the validity study, a standard measurement day in which 70% of the women were wearing the accelerometer was defined from 9:00 to 21:00 on weekdays and 10:30 to 21:00 on weekends (see Figure in additional file 3). Thus, the minimum number of hours of nonmissing data allowed to be considered as having complete accelerometry data was 8.4 hours on weekdays (12 hours * 70%) and 7.4 hours on weekends (10.5 hours * 70%). Among our sample, 120 women (68%) met this criteria and were defined as the "complete Actigraph sample".

Concurrent-Related Validity Assessment Using a Diary

Concurrent-Related Validity Assessment Using an Accelerometer

Test-Retest Reliability Assessment

Test-retest reliability remained moderate to almost perfect, using either perceived or absolute intensity, when exploring by modality for work, recreation, outdoor household, child and adult care, and transportation activity (Table 5). The exception to this was moderate to vigorous indoor household activity, where the ICC was 0.39 for perceived intensity and 0.36 for absolute intensity using a 3 MET lower bound.

Evidence for Validity Comparing to the Diary

With the lack of a single comparison standard measure for physical activity, we used both a self-reported (diary) and objective (accelerometer) measure to compare to the physical activity questionnaire. The PIN3 structured diary provided estimates of moderate to vigorous physical activity over the same week that the questionnaire recall occurred. SCC were moderate to substantial regardless of whether perceived or absolute intensity from the questionnaire was used. While there is some concern that keeping the structured diary may heighten participant's awareness and report of their recalled physical activities on the questionnaire for the validation portion of the study, thus biasing results to appear more favorable than they are, others have shown that this does not affect overall estimates of validity in other populations [37].

Evidence for Validity Comparing to the Accelerometer

A concern when comparing two self-reported measures against one another is that the errors inherent with each method may be correlated, resulting in inflated estimates of validity [9]. To account for this, we also compared the PIN3 physical activity questionnaire to an objective measure of physical activity. We found that agreement was higher when comparing the questionnaire to the diary (i.e., both self-reported measures) in comparison to the accelerometer.

For accelerometry, the challenge is that the cutpoints used to convert counts to moderate and vigorous intensity activity vary across studies, due in part to different calibration methods, different activities included in the calibration study, and differing populations. For adults, there seems to be more variability with moderate activity cutpoints as compared to vigorous activity cutpoints and no calibration studies have been conducted on pregnant women. We explored the sensitivity of the objective validity results by exploring three different accelerometry cutpoints for moderate and vigorous physical activity. The cutpoints from Freedson et al [20] and Swartz et al [28] were used in other pregnancy studies collecting accelerometry data [7, 9]. In addition, we used summary cutpoints from NHANES data by Troiano et al [29]. Similar to Chasan-Taber et al [7], we found that the correlation between the accelerometer and the questionnaire were lower for the Freedson et al [20] equation in which the count threshold was much higher, than for equations using lower thresholds from Swartz et al [28]. Others have also found that the higher count cutpoints for moderate activity may be set too high to capture a broad range of moderate activities [38]. Therefore, in addition to reporting comparisons using these cutpoints, we reported correlations with Actigraph counts, an indicator of overall physical activity that does not rely on thresholds.

We also explored our validation results using cutpoints for moderate to vigorous physical activity from a calibration study of Hendelman et al [21], which assigned moderate activity at 191 counts/minute and vigorous activity at 7525 counts/minute. This cutpoint for moderate activity was explored in other studies of pregnancy [7, 9, 39] and is much lower than the other values that we explored, while the vigorous cutpoint is higher than any we used. While the correlational results using Hendelman cutpoints generally produced similar results to our findings, the Bland-Altman plots indicated a strong bias with almost all women having higher values from the accelerometer as compared to the questionnaire. Thus, we chose not to report these results.

While the accelerometer is an acceptable measure to compare a physical activity measure against due to its objectivity and removal of recall and response bias, it is not without limitations. The accelerometer we used (Actigraph) is uniaxial; thus activities that involved upper body movement, lifting, or cycling are underestimated. Moreover, swimming activities were not counted, since we asked women to remove the accelerometer when in the water.

A recent comprehensive review summarized the results of studies comparing physical activity questionnaire data to objective measures [40]. When considering comparison to accelerometry, the average percent mean difference (calculated as (self-reported mean minus accelerometer mean)/accelerometer mean) across 60 studies of women was 138%. Most often, women's report of physical activity was higher than the accelerometer readings. In our study, this depended on the validity measure used. On the full sample, the percent mean difference ((mean from self-report - mean from accelerometer)/mean from accelerometer) for moderate to vigorous physical activity was 85% using the Freedson et al cutpoint and 96% using the Troiano et al cutpoint, indicating higher reporting from the questionnaire. In contrast, the percent mean difference for moderate to vigorous physical activity using the Swartz et al cutpoint was -69%, indicating higher reporting from the accelerometer. Using total counts from the accelerometer compared to total physical activity from the questionnaire, the percent mean difference was -88%.

Using the Troiano et al cutpoint, the US national data from 2003 to 2004 indicate that women 20 to 29 years of age average 24 minutes/day of moderate to vigorous physical activity, whereas women 30 to 39 years of age averaged 21 minutes/day [29]. Interestingly, when using the same accelerometer cutpoint we obtained a slightly lower average of 19 minutes/day of moderate to vigorous physical activity among our sample of 177 pregnant women. In contrast, the PIN3 questionnaire averaged 37 minutes/day for these same 177 pregnant women, a two-fold higher report.

Evidence for Reliability

Test-retest reliability agreement for the PIN3 questionnaire was moderate to almost perfect for moderate to vigorous physical activity. This is similar to reliability assessed from two other physical activity questionnaires, indicating agreement in measuring total physical activity with an ICC of 0.78 [7] and 0.84 [9]. Because we over sampled women who were active, our results may be an underestimate, since test-retest reliability is generally higher among sedentary adults. The period of recall was 24 to 48 hours, in order for women to recall the same time period and not introduce bias in reporting of different days. This short time period between interviews may have influenced their recall of activities, thus leading to higher estimates of reliability.

Limitations

Several limitations of this study should be noted. First, the distribution of characteristics of women in the validity and reliability samples showed were somewhat different due to the differing methods of recruitment. This would become important if the findings differed by important confounding variables. Second, it is not known how representative our sample is to the source population or how generalizable our results will be to other populations. Comparing to the US national data from NHANES described earlier, among women 20 to 39 years of age, the minutes of moderate to vigorous physical activity was slightly lower among our sample, which seemed reasonable since the women were pregnant. Third, we assigned MET values to all reported activities based on the compendium of physical activities for adults [11, 12]. It is not known how well these MET values correspond to similar activities performed while pregnant, but others indicate that they can differ from 0.2 to 2.0 METS [7]. The correlation between physical activity from self-report compared to accelerometry may decline over the course of pregnancy [39]. Studies assessing the metabolic intensity of activities among pregnant women at various gestational ages are needed to help quantify this error. Fourth, we have tested many associations but did not adjust for multiple testing. Therefore, significance should be interpreted with caution, and replication of results would be useful. Lastly, women were asked to recall moderate and vigorous activities that caused at least some increase in their breathing and heart rate. The activities that we classified as less than moderate, with a perceived intensity of "fairly light", may be under ascertained by some women since they were instructed to recall moderate to vigorous activities.