Skip to main content

Advertisement

Table 4 Fitbit and Jawbone validity studies (listed by author's last name and publication year)

From: Systematic review of the validity and reliability of consumer-wearable activity trackers

  Sample characteristics Tracker wearing protocol Measurements Validity results
Author (year) n % female Activity Lab/field Validity criterion (measure assessed) Type Placement Measures
Adam Noah et. al (2013) [29] 16 38 6 min each of treadmill walking (3.5 mph), walking with incline (3.5 mph at 5 %), jogging (5.5 mph), and stair stepping (30.5 centimeter step at 96 beats/min) Lab Two Actical accelerometers (steps), indirect calorimetry using K4b2 Cosmed (EE) Ultra (Fitbit) Waist (one on each side) Steps/min, kilocalories/min Fitbit Ultra vs. Actical ICC: average 0.94, range 0.80–0.99 (steps); Fitbit Ultra vs. Cosmed ICC: average 0.77, range 0.58-0.87 (kilocalories)
23 43 Classic (Fitbit) Waist (one on each side) Steps/min, kilocalories/min Fitbit vs. Actical ICC: average 0.93, range 0.82–0.98 (steps); Fitbit vs. Cosmed ICC: average 0.74, range 0.18-0.72 (kilocalories)
Bai et. al (2015) [45] 52 46 20 min sedentary, 25 min treadmill at self-selected speed, 25 min resistance exercise Lab Indirect calorimetry using Oxycon Mobile (EE) Flex (Fitbit) Left wrist Kilocalories/80- min trial Overestimated overall EE by 20.4 kilocalories; Pearson CC 0.78; overall mean absolute error 16.8 %
UP24 (Jawbone) Right wrist Underestimated overall EE by 23.1 kilocalories; Pearson CC 0.77; overall mean absolute error 18.2 %
Case et. al (2015) [30] 14 71 Treadmill walking at 3.0 mph for 500 and 1500 steps, each done twice Lab Tally counter (steps) One (Fitbit) Waist Steps/trial 500 step trial (n = 27 observations) mean 498.6 (SD 3.7); 1500 step trial (n = 26 observations) mean 1497.0 (SD 10.7)
Zip (Fitbit) Waist Steps/trial 500 step trial (n = 27 observations) mean 498.6 (SD 10.8); 1500 step trial (n = 27 observations) mean 1498.4 (SD 10.4)
Flex (Fitbit) Wrist Steps/trial 500 step trial (n = 28 observations) mean 465.4 (SD 92.1); 1500 step trial (n = 28 observations) mean 1378.0 (SD 142.7)
UP24 (Jawbone) Wrist Steps/trial 500 step trial (n = 28 observations) mean 477.5 (SD 102.1); 1500 step trial (n = 28 observations) mean 1477.0 (SD 174.4)
Dannecker et. al (2013) [31] 19 (16 with Fitbit data) 47 (from n = 19) Resting, supine, sitting, standing, free living activity, and 6 random activities out of 8 (walking (2.5 mph, 3.5 mph, or 2.5 mph with 2.5 % grade), stepping, sweeping, cycling (75 watts), standing, sitting Lab 4 h stay in whole room calorimeter (EE) Classic (Fitbit) Belt at anterior superior iliac spine Total EE during the 3.5-h period while in the room calorimeter (omitted first 30 minutes) Root-mean-square error of tracker 28.7 % or 143 kilocalories; root-mean-square error of tracker after labeling activities 12.9 % or 64 kilocalories
de Zambotti et. al (2015a) [47] 28 100 One nights sleep (n = 10), 2 nights sleep (n = 18) Lab Polysomnography (sleep) UP (Jawbone) Non dominant wrist TST, sleep onset latency, WASO Overestimated TST by 26.6 ± 35.3 min (p < 0.001) and sleep onset latency by 5.2 ± 9.6 min (p = 0.005); underestimated WASO by 31.2 ± 32.3 min (p < 0.001)
de Zambotti et. al (2015b) [48] 65 43 One nights sleep Lab Polysomnography (sleep) UP (Jawbone) Non dominant wrist TST, sleep efficiency, sleep onset latency, WASO Overestimated TST by 10.0 min (p < 0.001), sleep efficiency by 1.9 % (p < 0.001), and sleep onset latency by 1.3 min (p = 0.33); underestimated WASO by 10.6 min (p < .001)
Diaz et. al (2015) [15] 23 57 6 min each of treadmill walking (1.9 mph, 3.0 mph, 4.0 mph) and jogging (5.2 mph) Lab Counting from a video recording (steps), indirect calorimetry using Ultima CPX (EE) One (Fitbit) 2 on right hip, 1 on left hip Steps/min, kilocalories/min Pearson CC 0.97–0.99 and mean difference −3.1 to −0.3 (steps); Pearson CC 0.86-0.87 (kilocalories) and mean difference −0.8 to 0.4 kilocalories
Flex (Fitbit) 1 on each wrist Steps/min, kilocalories/min Pearson CC 0.77-0.85 and mean difference −26.3 to −2.9 (steps); Pearson CC 0.88 and mean difference −0.2 to 2.6 (kilocalories)
One (Fitbit) Right hip Steps/day, MVPA min/day, kilocalories/day, sleep min/day Pearson CC 0.99 (steps), 0.91 (MVPA), 0.76 (kilocalories), 0.92 (sleep); ICC 0.95 (steps), 0.46 (MVPA), 0.55 (kilocalories), 0.90 (sleep); mean absolute difference 779 (steps), 58.6 (MVPA), 349 (kilocalories), 23.0 (sleep); range of differences = −890 to 1849 (steps), 1.0 to 137.2 (MVPA), −1724 to −83 (kilocalories), 45 to 76 (sleep)
Zip (Fitbit) Right hip Steps/day, MVPA min/day, kilocalories/day Pearson CC 0.99 (steps), 0.88 (MVPA), 0.81 (kilocalories); ICC 0.98 (steps), 0.36 (MVPA), 0.57 (kilocalories); mean absolute difference 447 (steps), 89.8 (MVPA), 484 (kilocalories); range of differences −970 to 1596 (steps), 10.0 to 157.2 (MVPA), −1145 to 218 (kilocalories)
Ferguson et. al (2015) [33] 21 52 48 h (including sleep, excluding showering) of free-living conditions, no activity restrictions/guidelines Field BodyMedia SenseWear model MF (steps, physical activity, EE, sleep); ActiGraph GT3X+ (steps, physical activity) UP (Jawbone) Left wrist Steps/day, MVPA min/day, kilocalories/day, sleep min/day Pearson CC 0.97 (steps), 0.81 (MVPA), 0.74 (kilocalories), 0.89 (sleep); ICC 0.97 (steps), 0.70 (MVPA), 0.27 (kilocalories), 0.85 (sleep); mean absolute difference 806 (steps), 18.0 (MVPA), 866 (kilocalories), 22.0 (sleep); range of differences −1978 to 2252 (steps), −4.7 to 96.5 (MVPA), −1937 to −94 (kilocalories), − 31 to 132 (sleep)
Gusmer et. al (2014) [34] 32 78 30-min phases of treadmill walking at slow and brisk speeds (±10 % of selfselected comfortable walking speed) Lab ActiGraph G1TM (steps), CPX Ultima metabolic cart (EE) Ultra (Fitbit) Right hip Steps/min, kilocalories/trial Pearson CC: slow walk: 0.97 (steps: mean 105.3 ActiGraph vs. 105.9 Ultra), 0.69 (kilocalories: mean 100.9 cart vs. 88.0 Ultra); brisk walking: 0.996 (steps: mean 114.2 ActiGraph vs. 113.9 Ultra), 0.94 (kilocalories: mean 121.9 cart vs. 100.9 Ultra)
Lauritzen et. al (2013) [23] 6 0 20-meter walk at participant's normal pace Lab Counting from a video recording (steps) Ultra (Fitbit) 1 on belt/pants pocket on dominant leg, 1 on wrist of dominant hand Steps/20-min trial Hip error 2.9 % (SD 2.3 %); wrist error 31.3 % (SD 30.7 %)
One (Fitbit) Waist Kilocalories/trial Mean absolute error 10.4 %; Pearson CC 0.81; root-mean-square error 40.1; did not fall in 90 % equivalence interval; systematic bias with slope −0.22 comparing One (x) to Oxycon (y); Pearson CC to ActiGraph 0.80
Lee et. al (2014) [35] 60 50 13 activities that were all 5 min in length except for treadmill (3 min each) totalling 69 minutes Lab Oxycon Mobile (EE); ActiGraph GT3X+ worn on hip, applied Sasaki et al. 2011 [39] algorithm (EE) Zip (Fitbit) Waist Kilocalories/trial Mean absolute error 10.1 %; Pearson CC 0.81; root-mean-square error 40.8; fell within 90 % equivalence interval from measured EE; systematic bias with slope - 0.29 comparing Zip (x) to Oxycon (y); Pearson CC to ActiGraph 0.77
UP (Jawbone) Left wrist Kilocalories/trial Mean absolute error 12.2 %; Pearson CC 0.74; root-mean-square error 45.8; did not fall in 90 % equivalence interval; no systematic direction of bias with slope - 0.03 comparing UP (x) to Oxycon (y); Pearson CC to ActiGraph 0.65
Mammen et. al (2012) [36] 10 50 One min on the treadmill at each of 8 speeds (4 walking and 4 running) Lab Manually count (steps) Ultra (Fitbit) Waist, inside the pants pocket, shirt collar (men) or bra (women) Steps/trial Waist-worn Ultra under counted at 2 km/hour (31 steps/min; p < 0.05) but had similar counts at > =3 km/hour. Pocket- worn Ultra under counted during running (10, 19, 34, 38 steps/min at 8, 9, 10, and 11 km/hour, respectively; p < 0.05), but recorded similar counts when walking (2, 3, 4.5, and 6 km/hour). Similar counts across walk/run trials for collar-(males) or bra-(females) worn Ultras.
Meltzer et. al (2015) [37] 63 51 One night's sleep Lab Polysomnography (sleep) Ultra (Fitbit) Non dominant wrist TST, sleep efficiency, WASO Normal mode overestimated TST by 41 minutes and sleep efficiency by 8 %, underestimated WASO by 32 minutes; 87 % sensitivity, 52 % specificity, 84 % accuracy. Sensitive mode underestimated TST by 105 minutes and sleep efficiency by 21 % and overestimated WASO by 106 minutes; 70 % sensitivity, 79 % specificity, 71 % accuracy.
Montgomery- Downs et. al (2012) [38] 24 40 One night's sleep Lab Polysomnography (sleep) Classic (Fitbit) Non dominant wrist TST, sleep efficiency Polysomnography recorded 465.0 min (SD 48.4) with 79.5 % sleep efficiency and 370.9 min (SD 70.3) TST; Fitbit measured 94.0 % sleep efficiency and 438.0 min TST; Fitbit overestimated sleep efficiency compared to polysomnography by 14.5 % (SD 10.7 %) and overestimated TST by mean 67.1 min (SD 51.3).
Sasaki et. al (2015) [39] 20 50 Visit 1: 6 min each of treadmill walking (3.0 at 5 % and 4.0 at 5 %) and jogging (5.5 mph), three trials; visit 2: 6 min each of household activities (choice from 3 activity routines) Lab Oxycon Mobile (EE) Classic (Fitbit) Belt around waist in line with the anterior axillary line Total EE (rest plus activity) Pearson CC 0.86; systematic underestimation of EE by the Fitbit with a mean bias of −4.5 ± 1.0 kcals/6 min; for 6 of 15 activities the Fitbit significantly underestimated EE (stairs, cycling, laundry, raking, treadmill 3.0 mph with 5 % grade, treadmill 4.0 mph with 5 % grade) and 1 of 15 activities the Fitbit significantly overestimated EE (carrying groceries)
Simpson et. al (2015) [46] 42 74 8 trials of walking 15 meters (self selected speed and 0.3-0.9 m/s at 0.1 increments) Lab Counting from a video recording (steps) One (Fitbit) Right waist, right ankle Steps/trial % error: 0.3 m/s: ankle 14.5, waist 98.4; 0.4 m/s: ankle 5.9, waist 82.0; 0.5 m/s: ankle 4.1, waist 40.4; 0.6 m/s: ankle 3.2, waist 21.6; 0.7 m/s: ankle 2.5, waist 10.5; 0.8 m/s: ankle 2.8, waist 7.0; 0.9 m/s: ankle 2.8, waist 5.6; Bland Altman mean difference −0.4 to 5.7 steps for ankle and 1.4 to 48.0 for waist
Stackpool et. al (2014) [40] 20 50 20 min each of: treadmill walking, treadmill running, elliptical cross-training, agility-related exercises Lab Manually counting (steps); Oxycon Mobile (EE) Ultra (Fitbit) Hip Steps and kilocalories for each 20-min bout Pearson CC: treadmill walking (0.99 steps, 0.24 kilocalories), treadmill running (0.44 steps, 0.63 kilocalories), elliptical (0.99 steps, 0.47 kilocalories), agility (0.47 steps, 0.67 kilocalories)
UP (Jawbone) Wrist Steps and kilocalories for each 20-min bout Pearson CC: treadmill walking (0.98 steps, 0.87 kilocalories), treadmill running (0.98 steps, 0.69 kilocalories), elliptical (0.99 steps, 0.40 kilocalories), agility (0.34 steps, 0.65 kilocalories)
Stahl and Insana (2014) [41] 10 30 During waking hours for 10 consecutive days Field Self-reported estimation of expended kilocalories/week from CHAMPS questionnaire (EE). Note: kilocalories/week divided by 7 to obtain kilocalories/day; then basal metabolic rate was added to the kilocalories/day. Classic (Fitbit) Waist Kilocalories/day Pearson CC 0.61; Fitbit underestimated by a mean of 195.0 kilocalories/day; 70 % of participant's data were within 1 SD and 100 % were within 2 SD
Storm et. al (2015) [42] 16 38 11-min walking protocol (included indoor and outdoor walking and steps) repeated at self-selected natural, slow, and fast speeds Lab OPAL sensors placed on each ankle (steps) One (Fitbit) Left waist Steps/11-min trial 1.1 % self-selected walk, 1.0 % fast walk; limits of agreement 15 ± 35 steps; under estimated for slow walk (−25 mean steps), self-selected walk (−12 mean steps), fast walk (−9 mean steps)
UP (Jawbone) Right wrist Steps/11-min trial Mean absolute error 10.1 % slow walk, 2.5 % self-selected walk, 2.1 % fast walk; limits of agreement 16 ± 135; under estimated for slow walk (−35 mean steps), self-selected walk (−4 mean steps), fast walk (−9 mean steps)
Takacs et. al (2014) [43] 30 50 5 min each of treadmill walking (0.90, 1.12, 1.33, 1.54, 1.78 meters/second) Lab Motion capture system and manually counting (steps); treadmill output (distance) One (Fitbit) 1 right hip, 1 left hip, 1 in front pocket of the dominant leg Steps/trial, distance/trial Steps: no significant difference (p > 0.05) between observed and One step counts at any of the 3 locations, ICC 0.97-1.00, relative error <1.3 %. Distance: significant differences between observed and One distance, ICC 0.0-0.05, relative error 5.0-39.6 %.
Tully et. al (2014) [44] 42 60 7 days of free-living wear excluding water activities and sleep Field ActiGraph GT3X and Yamax CW700 pedometer (steps, physical activity) Zip (Fitbit) Right waist Steps/day, MVPA min/day Spearman CC: 0.91 (ActiGraph steps), 0.86 (ActiGraph MVPA), 0.91 (Yamax steps)
  1. Abbreviations: CC correlation coefficient, CHAMPS Community Healthy Activities Model Program for Seniors, EE energy expenditure, ICC intraclass correlation coefficient, km kilometers, m meters, m/s meters/second, min minute, mph miles per hour, MVPA moderate to vigorous physical activity, SD standard deviation, TST total sleep time, WASO wake after sleep onset