Does social desirability compromise self-reports of physical activity in web-based research?
© Crutzen and Göritz; licensee BioMed Central Ltd. 2011
Received: 23 December 2010
Accepted: 14 April 2011
Published: 14 April 2011
This study investigated the relation between social desirability and self-reported physical activity in web-based research.
A longitudinal study (N = 5,495, 54% women) was conducted on a representative sample of the Dutch population using the Marlowe-Crowne Scale as social desirability measure and the short form of the International Physical Activity Questionnaire. Social desirability was not associated with self-reported physical activity (in MET-minutes/week), nor with its sub-behaviors (i.e., walking, moderate-intensity activity, vigorous-intensity activity, and sedentary behavior). Socio-demographics (i.e., age, sex, income, and education) did not moderate the effect of social desirability on self-reported physical activity and its sub-behaviors.
This study does not throw doubt on the usefulness of the Internet as a medium to collect self-reports on physical activity.
This study illuminates the relation between social desirability and self-reported physical activity (PA) in web-based research. Self-report measures are a common way of gathering data in research on PA, because self-reports require fewer financial and logistical resources compared to more traditional methods such as accelerometers. Furthermore, interventions are nowadays increasingly delivered through the Internet [1, 2], because of accessibility, convenience, and anonymity. These interventions are routinely accompanied by measurements of PA. In view of the above, the Internet might be a useful environment to collect self-reports on PA. Some researchers, however, state that the social distance  and the impersonal nature of the Internet might inhibit trust development .
Social desirability - the tendency of respondents to distort self-reports in a favorable direction  - might compromise self-reports on PA. Research using latent state-trait models, which account for systematic effects of the situation in which measurement takes place, indicates that the largest proportion of variance in self-reports is attributable to differences in the social desirability trait as opposed to situation-specific conditions eliciting socially desirable self-reporting . If self-reports are indeed influenced by social desirability, taking into account variance due to social desirability may remove some of the error due to the use of self-reports and therewith improve self-reports' validity . Previous research found minimal evidence of an influence of social desirability on scores from two self-report measures of PA in young adults . This research, however, was not web-based, thereby ignoring the social distance and impersonal nature of the Internet. Mode comparison studies (i.e., in which web-based assessment is compared with, for example, paper-and-pencil assessment [9, 10] or with telephone interviewing ) have been conducted, but the focus of these studies was not on the influence of social desirability itself: These studies did not investigate whether differences in social desirability resulted in differences in self-reported PA. If social desirability is found out to cause distortion in web-based research, this would raise concerns about the validity of web-based research on PA. Therefore, we investigated the association between social desirability and web-based self-reported PA. We put forward the research question:
To what extent is social desirability associated with self-reported PA in web-based research?
Because of the social distance  and the impersonal nature of the Internet  and in line with previous findings on web-based research on substance use , we did not expect social desirability to bias web-based self-reports of PA. In addition, we investigated potential moderating effects of socio-demographics (i.e., age, sex, personal net monthly income in Euros , and level of education according to the definitions of Statistics Netherlands) on the effects of social desirability on self-reported PA. In line with a meta-analysis about social desirability distortion , we did not expect any moderating effects of socio-demographics.
A longitudinal study was conducted to investigate the relation between social desirability and self-reported PA in web-based research. Data were collected through the LISS panel (Longitudinal Internet Studies for the Social sciences; http://www.lissdata.nl). The reference population for the LISS panel is the Dutch speaking population permanently residing in the Netherlands. In co-operation with Statistics Netherlands addresses were drawn randomly from the nationwide address frame and potential panel members were contacted via post. The sample from the population registers includes individuals who do not have Internet access. These individuals were provided equipment to access the Internet via a broadband connection. Sample members with small-band Internet access were provided with broadband .
Social desirability measurements were obtained between May 2008 and August 2008 (T1). In total, 8,722 panel members were invited. Of those, 6,766 completed the social desirability measure (77.6%) - the Marlowe-Crowne Scale , which has been validated previously . A high score indicates a large tendency to provide socially desirable responses.
Sample characteristics (N = 5,495)
M3 = 47.1 (SD3 = 16.0)
Personal net monthly income
Median = 1,300
Level of education
Intermediate secondary education (US: junior high school)
Higher secondary education/preparatory university education (US: senior high school)
Intermediate vocational education (US: junior college)
Higher vocational education (US: college)
Social desirability (Marlowe-Crowne)
M = 5.9 (SD = 1.5) (Scale: 0-10)
M = 342 (SD = 367)
M = 299 (SD = 355)
M = 157 (SD = 267)
M = 2,471 (SD = 1,365)
Total physical activity2
M = 3,579 (SD = 3,525)
Retainees across the two time points did not differ in sex (χ2(1, N = 6,603) = .23, p = .64), income (t(6,285) = .72, p = .47), and education (χ2(5, N = 6,603) = 10.24, p = .07) from panel members who dropped-out. Those who dropped-out, however, were younger than those who completed both the Marlowe-Crowne Scale and the IPAQ (42.1 versus 47.1 years, t(6,601) = 9.16, p < .01).
Effect of social desirability on self-reported physical activity
Total physical activity2
In line with our hypothesis, this longitudinal study revealed no significant associations between social desirability and self-reported PA in web-based research. Moreover, in agreement with a meta-analysis on social desirability distortion , socio-demographics did not moderate the relationship between social desirability and self-reported PA. A possible explanation for the lack of a noteworthy association between social desirability and self-reported PA is that the online setting increases respondents' perceived privacy. Therefore, social desirability might have exerted less influence in this web-based study in comparison with, for example, disclosure in front of an interviewer . Another fact might have contributed to not finding an association between social desirability and self-reported PA: We examined a population-representative sample from an existing panel that was not focused on PA or health as opposed to measuring PA in the context of an intervention aimed at increasing PA. In the context of an intervention aimed at increasing PA the felt pressure to overstate one's PA is likely to be higher than in a non-intervention study, implying that social desirability might distort self-reports of PA in an intervention study after all. This is a subject for future research.
Although social desirability was not found to be related to self-reported PA in this web-based research, this does not imply that self-report measures reveal the same results as direct measures (e.g., accelerometers). A recent review found correlations between self-report and direct measures of PA to be low to moderate . This being only a general caution as this work was not about the comparative validity of self-reports versus direct measures.
Three final points need to be made: (1) Social desirability bias is not the only source of measurement error. Recall error, for example, may also lead to measurement error as may question format . (2) People who dropped-out of the study were younger than retainees. First, a certain level of drop-out is ubiquitous in longitudinal research, also on the web . Second, the dropout in these studies seems to be innocuous, because socio-demographics did not moderate the impact of social desirability on self-reported health risk behaviors. (3) These studies failed to find meaningful associations between social desirability and self-reported PA. Because an absence of evidence of an association does not equal evidence of absence of an association, future research is not precluded from revealing such an association after all. However, the longitudinal nature of this study (e.g., measurements are unlikely to be distorted by participants' unintentional and intentional attempts at portraying themselves as consistent) and the reliance on a large and representative sample gives us confidence in the robustness of our results. This study, therefore, does not throw doubt on the usefulness of the Internet as a medium of obtaining self-reports of PA.
This paper draws on data of the LISS panel of CentERdata.
- Webb TL, Joseph J, Yardley L, Michie S: Using the Internet to promote health behavior change: a meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010, 12 (1): e4-10.2196/jmir.1376.View Article
- Bock BC, Graham AL, Whiteley JA, Stoddard JL: A review of web-assisted tobacco interventions (WATIs). J Med Internet Res. 2008, 10 (5): e39-10.2196/jmir.989.View Article
- Newman JC, Des Jarlais DC, Turner CF, Gribble J, Cooley P, Paone D: The differential effects of face-to-face and computer interview modes. Am J Public Health. 2002, 92: 294-297. 10.2105/AJPH.92.2.294.View Article
- Joinson AN: Knowing me, knowing you: reciprocal self-disclosure in Internet-based surveys. Cyberpsychol Behav. 2001, 4: 587-591. 10.1089/109493101753235179.View Article
- Paulhus DL: Measurement and control of response bias. Measures of personality and social psychological attitudes. Edited by: Robinson JP, Shaver PR, Wrightsman LS. 1991, San Diego: Academic press, 17-59.View Article
- Schmitt MJ, Steyer R: A latent state-trait model (not only) for social desirability. Pers Indiv Differ. 1993, 14: 519-529. 10.1016/0191-8869(93)90144-R.View Article
- Jago R, Baranowski T, Baranowski JC, Cullen KW, Thompson DI: Social desirability is associated with some physical activity, psychosocial variables and sedentary behavior but not self-reported physical activity among adolescent males. Health Educ Res. 2007, 22: 3.
- Motl RW, McAuley E, DiStefano C: Is social desirability associated with self-reported physical activity?. Prev Med. 2005, 40: 735-739. 10.1016/j.ypmed.2004.09.016.View Article
- Ritter P, Lorig K, Laurent D, Matthews K: Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004, 6 (3): e29-10.2196/jmir.6.3.e29.View Article
- Wu RC, Thorpe K, Ross H, Micevski V, Marquez C, Straus SE: Comparing administration of questionnaires via the Internet to pen-and-paper in patients with heart failure: randomized controlled trial. J Med Internet Res. 2009, 11 (1): e3-10.2196/jmir.1106.View Article
- Nagelhout GE, Willemsen MC, Thompson ME, Fong GT, Van den Putte B, De Vries H: Is web interviewing a good alternative to telephone interviewing? Findings from the International Tobacco Control (ITC) Netherlands Survey. BMC Public Health. 2010, 10: 351-10.1186/1471-2458-10-351.View Article
- Crutzen R, Göritz AS: Social desirability and self-reported health risk behaviors in web-based research: three longitudinal studies. BMC Public Health. 2010, 10: 720-10.1186/1471-2458-10-720.View Article
- Imputation of income in household questionnaire LISS panel. [http://www.lissdata.nl//dataarchive/hosted_files/download/24]
- Richman WL, Kiesler S, Weisband S, Drasgow F: A meta-analytic study of social desirability distortion in computer-administered questionnaires, tradition questionnaires, and interviews. J Appl Psychol. 1999, 84 (5): 754-775. 10.1037/0021-9010.84.5.754.View Article
- Start of the LISS panel: sample and recruitment of a probability-based Internet panel. [http://www.lissdata.nl/assets/uploaded/Sample%20and%20Recruitment_1.pdf]
- Crowne DP, Marlowe D: A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology. 1960, 24: 349-354. 10.1037/h0047358.View Article
- Fischer DG, Fick C: Further validation of three short forms of the Marlowe-Crowne Scale of Social Desirability. Psychol Rep. 1989, 65: 595-600.View Article
- Craig CL, Marshall AL, Sjöström M, Bauman AE, Booth ML, Ainsworth BE, Pratt M, Ekelund U, Yngve A, Sallis JF, et al: International physical activity questionnaire: 12-country reliability and validity. Med Sci Sport Exer. 2003, 35 (8): 1381-1395. 10.1249/01.MSS.0000078924.61453.FB.View Article
- Crutzen R: A systematic review on computer-based education for patients with hypertension: what about effect sizes?. Health Educ J. 2010, 69: 365-366. 10.1177/0017896910376870.View Article
- Crutzen R: Adding effect sizes to a systematic review on interventions for promoting physical activity among European teenagers. Int J Behav Nutr Phy. 2010, 7: 29-10.1186/1479-5868-7-29.View Article
- Tourangeau R, Smith TW: Asking sensitive questions: the impact of data collection mode, question format, and question context. Public Opin Quart. 1996, 60: 275-304. 10.1086/297751.View Article
- Prince SA, Adamo KB, Hamel ME, Hardt J, Connor Gorber S, Tremblay M: A comparison of direct versus self-report measures for assessing physical activity in adults: a systematic review. Int J Behav Nutr Phy. 2008, 5: 56-10.1186/1479-5868-5-56.View Article
- Gmel G, Lokosha O: Self-reported frequency of drinking assessed with a closed- or open-ended question format: a split-sample study in Switzerland. J Stud Alcohol Drugs. 2000, 61: 450-454.View Article
- Göritz AS: The long-term effect of material incentives on participation in online panels. Field Method. 2008, 20: 211-225.View Article
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.