- Short paper
- Open Access
Does social desirability compromise self-reports of physical activity in web-based research?
International Journal of Behavioral Nutrition and Physical Activity volume 8, Article number: 31 (2011)
This study investigated the relation between social desirability and self-reported physical activity in web-based research.
A longitudinal study (N = 5,495, 54% women) was conducted on a representative sample of the Dutch population using the Marlowe-Crowne Scale as social desirability measure and the short form of the International Physical Activity Questionnaire. Social desirability was not associated with self-reported physical activity (in MET-minutes/week), nor with its sub-behaviors (i.e., walking, moderate-intensity activity, vigorous-intensity activity, and sedentary behavior). Socio-demographics (i.e., age, sex, income, and education) did not moderate the effect of social desirability on self-reported physical activity and its sub-behaviors.
This study does not throw doubt on the usefulness of the Internet as a medium to collect self-reports on physical activity.
This study illuminates the relation between social desirability and self-reported physical activity (PA) in web-based research. Self-report measures are a common way of gathering data in research on PA, because self-reports require fewer financial and logistical resources compared to more traditional methods such as accelerometers. Furthermore, interventions are nowadays increasingly delivered through the Internet [1, 2], because of accessibility, convenience, and anonymity. These interventions are routinely accompanied by measurements of PA. In view of the above, the Internet might be a useful environment to collect self-reports on PA. Some researchers, however, state that the social distance  and the impersonal nature of the Internet might inhibit trust development .
Social desirability - the tendency of respondents to distort self-reports in a favorable direction  - might compromise self-reports on PA. Research using latent state-trait models, which account for systematic effects of the situation in which measurement takes place, indicates that the largest proportion of variance in self-reports is attributable to differences in the social desirability trait as opposed to situation-specific conditions eliciting socially desirable self-reporting . If self-reports are indeed influenced by social desirability, taking into account variance due to social desirability may remove some of the error due to the use of self-reports and therewith improve self-reports' validity . Previous research found minimal evidence of an influence of social desirability on scores from two self-report measures of PA in young adults . This research, however, was not web-based, thereby ignoring the social distance and impersonal nature of the Internet. Mode comparison studies (i.e., in which web-based assessment is compared with, for example, paper-and-pencil assessment [9, 10] or with telephone interviewing ) have been conducted, but the focus of these studies was not on the influence of social desirability itself: These studies did not investigate whether differences in social desirability resulted in differences in self-reported PA. If social desirability is found out to cause distortion in web-based research, this would raise concerns about the validity of web-based research on PA. Therefore, we investigated the association between social desirability and web-based self-reported PA. We put forward the research question:
To what extent is social desirability associated with self-reported PA in web-based research?
Because of the social distance  and the impersonal nature of the Internet  and in line with previous findings on web-based research on substance use , we did not expect social desirability to bias web-based self-reports of PA. In addition, we investigated potential moderating effects of socio-demographics (i.e., age, sex, personal net monthly income in Euros , and level of education according to the definitions of Statistics Netherlands) on the effects of social desirability on self-reported PA. In line with a meta-analysis about social desirability distortion , we did not expect any moderating effects of socio-demographics.
A longitudinal study was conducted to investigate the relation between social desirability and self-reported PA in web-based research. Data were collected through the LISS panel (Longitudinal Internet Studies for the Social sciences; http://www.lissdata.nl). The reference population for the LISS panel is the Dutch speaking population permanently residing in the Netherlands. In co-operation with Statistics Netherlands addresses were drawn randomly from the nationwide address frame and potential panel members were contacted via post. The sample from the population registers includes individuals who do not have Internet access. These individuals were provided equipment to access the Internet via a broadband connection. Sample members with small-band Internet access were provided with broadband .
Social desirability measurements were obtained between May 2008 and August 2008 (T1). In total, 8,722 panel members were invited. Of those, 6,766 completed the social desirability measure (77.6%) - the Marlowe-Crowne Scale , which has been validated previously . A high score indicates a large tendency to provide socially desirable responses.
This initial sample of 6,766 panel members was re-invited between November 2008 and December 2008 (T2) to complete the follow-up measures on PA as measured by the short form of the International Physical Activity Questionnaire (IPAQ) . Of those invited, 5,495 completed the IPAQ (81.2%) and were included in the analyses (Table 1). Guidelines for data processing and analysis of the IPAQ (http://www.ipaq.ki.se/scoring.pdf) were adhered to.
Retainees across the two time points did not differ in sex (χ2(1, N = 6,603) = .23, p = .64), income (t(6,285) = .72, p = .47), and education (χ2(5, N = 6,603) = 10.24, p = .07) from panel members who dropped-out. Those who dropped-out, however, were younger than those who completed both the Marlowe-Crowne Scale and the IPAQ (42.1 versus 47.1 years, t(6,601) = 9.16, p < .01).
Multiple regression analyses revealed that social desirability was not associated with total PA (in MET-minutes/week), nor with its sub-behaviors (i.e., walking, moderate-intensity activity, vigorous-intensity activity, and sedentary behavior); both in terms of significance and effect sizes [19, 20] (Table 2). Variance due to age, sex, personal net monthly income, and education was taken into account by including these four variables as predictors in the models. Interaction terms between socio-demographics (i.e., age, sex, income, and education) and social desirability were left out of the final models, because none of them were statistically significant. Thus, socio-demographics did not moderate the effect of social desirability on total PA or its sub-behaviors.
In line with our hypothesis, this longitudinal study revealed no significant associations between social desirability and self-reported PA in web-based research. Moreover, in agreement with a meta-analysis on social desirability distortion , socio-demographics did not moderate the relationship between social desirability and self-reported PA. A possible explanation for the lack of a noteworthy association between social desirability and self-reported PA is that the online setting increases respondents' perceived privacy. Therefore, social desirability might have exerted less influence in this web-based study in comparison with, for example, disclosure in front of an interviewer . Another fact might have contributed to not finding an association between social desirability and self-reported PA: We examined a population-representative sample from an existing panel that was not focused on PA or health as opposed to measuring PA in the context of an intervention aimed at increasing PA. In the context of an intervention aimed at increasing PA the felt pressure to overstate one's PA is likely to be higher than in a non-intervention study, implying that social desirability might distort self-reports of PA in an intervention study after all. This is a subject for future research.
Although social desirability was not found to be related to self-reported PA in this web-based research, this does not imply that self-report measures reveal the same results as direct measures (e.g., accelerometers). A recent review found correlations between self-report and direct measures of PA to be low to moderate . This being only a general caution as this work was not about the comparative validity of self-reports versus direct measures.
Three final points need to be made: (1) Social desirability bias is not the only source of measurement error. Recall error, for example, may also lead to measurement error as may question format . (2) People who dropped-out of the study were younger than retainees. First, a certain level of drop-out is ubiquitous in longitudinal research, also on the web . Second, the dropout in these studies seems to be innocuous, because socio-demographics did not moderate the impact of social desirability on self-reported health risk behaviors. (3) These studies failed to find meaningful associations between social desirability and self-reported PA. Because an absence of evidence of an association does not equal evidence of absence of an association, future research is not precluded from revealing such an association after all. However, the longitudinal nature of this study (e.g., measurements are unlikely to be distorted by participants' unintentional and intentional attempts at portraying themselves as consistent) and the reliance on a large and representative sample gives us confidence in the robustness of our results. This study, therefore, does not throw doubt on the usefulness of the Internet as a medium of obtaining self-reports of PA.
Webb TL, Joseph J, Yardley L, Michie S: Using the Internet to promote health behavior change: a meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010, 12 (1): e4-10.2196/jmir.1376.
Bock BC, Graham AL, Whiteley JA, Stoddard JL: A review of web-assisted tobacco interventions (WATIs). J Med Internet Res. 2008, 10 (5): e39-10.2196/jmir.989.
Newman JC, Des Jarlais DC, Turner CF, Gribble J, Cooley P, Paone D: The differential effects of face-to-face and computer interview modes. Am J Public Health. 2002, 92: 294-297. 10.2105/AJPH.92.2.294.
Joinson AN: Knowing me, knowing you: reciprocal self-disclosure in Internet-based surveys. Cyberpsychol Behav. 2001, 4: 587-591. 10.1089/109493101753235179.
Paulhus DL: Measurement and control of response bias. Measures of personality and social psychological attitudes. Edited by: Robinson JP, Shaver PR, Wrightsman LS. 1991, San Diego: Academic press, 17-59.
Schmitt MJ, Steyer R: A latent state-trait model (not only) for social desirability. Pers Indiv Differ. 1993, 14: 519-529. 10.1016/0191-8869(93)90144-R.
Jago R, Baranowski T, Baranowski JC, Cullen KW, Thompson DI: Social desirability is associated with some physical activity, psychosocial variables and sedentary behavior but not self-reported physical activity among adolescent males. Health Educ Res. 2007, 22: 3.
Motl RW, McAuley E, DiStefano C: Is social desirability associated with self-reported physical activity?. Prev Med. 2005, 40: 735-739. 10.1016/j.ypmed.2004.09.016.
Ritter P, Lorig K, Laurent D, Matthews K: Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004, 6 (3): e29-10.2196/jmir.6.3.e29.
Wu RC, Thorpe K, Ross H, Micevski V, Marquez C, Straus SE: Comparing administration of questionnaires via the Internet to pen-and-paper in patients with heart failure: randomized controlled trial. J Med Internet Res. 2009, 11 (1): e3-10.2196/jmir.1106.
Nagelhout GE, Willemsen MC, Thompson ME, Fong GT, Van den Putte B, De Vries H: Is web interviewing a good alternative to telephone interviewing? Findings from the International Tobacco Control (ITC) Netherlands Survey. BMC Public Health. 2010, 10: 351-10.1186/1471-2458-10-351.
Crutzen R, Göritz AS: Social desirability and self-reported health risk behaviors in web-based research: three longitudinal studies. BMC Public Health. 2010, 10: 720-10.1186/1471-2458-10-720.
Imputation of income in household questionnaire LISS panel. [http://www.lissdata.nl//dataarchive/hosted_files/download/24]
Richman WL, Kiesler S, Weisband S, Drasgow F: A meta-analytic study of social desirability distortion in computer-administered questionnaires, tradition questionnaires, and interviews. J Appl Psychol. 1999, 84 (5): 754-775. 10.1037/0021-9010.84.5.754.
Start of the LISS panel: sample and recruitment of a probability-based Internet panel. [http://www.lissdata.nl/assets/uploaded/Sample%20and%20Recruitment_1.pdf]
Crowne DP, Marlowe D: A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology. 1960, 24: 349-354. 10.1037/h0047358.
Fischer DG, Fick C: Further validation of three short forms of the Marlowe-Crowne Scale of Social Desirability. Psychol Rep. 1989, 65: 595-600.
Craig CL, Marshall AL, Sjöström M, Bauman AE, Booth ML, Ainsworth BE, Pratt M, Ekelund U, Yngve A, Sallis JF, et al: International physical activity questionnaire: 12-country reliability and validity. Med Sci Sport Exer. 2003, 35 (8): 1381-1395. 10.1249/01.MSS.0000078924.61453.FB.
Crutzen R: A systematic review on computer-based education for patients with hypertension: what about effect sizes?. Health Educ J. 2010, 69: 365-366. 10.1177/0017896910376870.
Crutzen R: Adding effect sizes to a systematic review on interventions for promoting physical activity among European teenagers. Int J Behav Nutr Phy. 2010, 7: 29-10.1186/1479-5868-7-29.
Tourangeau R, Smith TW: Asking sensitive questions: the impact of data collection mode, question format, and question context. Public Opin Quart. 1996, 60: 275-304. 10.1086/297751.
Prince SA, Adamo KB, Hamel ME, Hardt J, Connor Gorber S, Tremblay M: A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phy. 2008, 5: 56-10.1186/1479-5868-5-56.
Gmel G, Lokosha O: Self-reported frequency of drinking assessed with a closed- or open-ended question format: a split-sample study in Switzerland. J Stud Alcohol Drugs. 2000, 61: 450-454.
Göritz AS: The long-term effect of material incentives on participation in online panels. Field Method. 2008, 20: 211-225.
This paper draws on data of the LISS panel of CentERdata.
The authors declare that they have no competing interests.
RC designed the study and AG substantially contributed to the interpretation of the data. RC drafted the manuscript and AG substantially contributed to revising it. Both authors approved the final version of the manuscript.
About this article
Cite this article
Crutzen, R., Göritz, A.S. Does social desirability compromise self-reports of physical activity in web-based research?. Int J Behav Nutr Phys Act 8, 31 (2011) doi:10.1186/1479-5868-8-31
- Physical Activity
- Sedentary Behavior
- Social Desirability
- International Physical Activity Questionnaire
- Total Physical Activity