Validation of sleep measurement in a multisensor consumer grade wearable device in healthy young adults.
Ontology highlight
ABSTRACT: Study objectives
Our objective was to examine the ability of a consumer-grade wearable device (Basis B1) with accelerometer and heart rate technology to assess sleep patterns compared with polysomnography (PSG) and research-grade actigraphy in healthy adults.Methods
Eighteen adults underwent consecutive nights of sleep monitoring using Basis B1, actigraphy, and PSG; 40 nights were used in analyses. Discrepancies in gross sleep parameters and epoch-by-epoch agreements in sleep/wake classification were assessed.Results
Basis B1 accuracy was 54.20 ± 8.20%, sensitivity was 98.90 ± 2.70%, and specificity was 8.10 ± 15.00%. Accuracy, sensitivity, and specificity for distinguishing between the different sleep stages were 60-72%, 48-62%, and 57-86%, respectively. Pearson correlations demonstrated strong associations between Basis B1 and PSG estimates of sleep onset latency and total sleep time; moderate associations for sleep efficiency, duration of light sleep, and duration of rapid eye movement sleep; and a weak association for duration of deep sleep. Basis B1 significantly overestimates total sleep time, sleep efficiency, and duration of light sleep and significantly underestimates wake after sleep onset and duration of deep sleep.Conclusions
Basis B1 demonstrated utility for estimates of gross sleep parameters and performed similarly to actigraphy for estimates of total sleep time. Basis B1 specificity was poor, and Basis B1 is not useful for the assessment of wake. Basis B1 accuracy for sleep stages was better than chance but is not a suitable replacement for PSG assessment. Despite low cost, ease of use, and attractiveness for patients, consumer devices are not yet accurate or reliable enough to guide treatment decision making in clinical settings.
SUBMITTER: Kanady JC
PROVIDER: S-EPMC7849656 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Study objectivesTo compare the quality and consistency in sleep measurement of a consumer wearable device and a research-grade actigraph with polysomnography (PSG) in adolescents.MethodsFifty-eight healthy adolescents (aged 15-19 years; 30 males) underwent overnight PSG while wearing both a Fitbit Alta HR and a Philips Respironics Actiwatch 2 (AW2) for 5 nights, with either 5 hours or 6.5 hours time in bed (TIB) and for 4 nights with 9 hours TIB. AW2 data were evaluated using two different wake and immobility thresholds. Discrepancies in estimated total sleep time (TST) and wake after sleep onset (WASO) between devices and PSG, as well as epoch-by-epoch agreements in sleep/wake classification, were assessed. Fitbit-generated sleep staging was compared to PSG.ResultsFitbit and AW2 under default settings similarly underestimated TST and overestimated WASO (TST: medium setting (M10) ≤ 38 minutes, Fitbit ≤ 47 minutes; WASO: M10 ≤ 38 minutes; Fitbit ≤ 42 minutes). AW2 at the high motion threshold setting provided readings closest to PSG (TST: ≤ 12 minutes; WASO: ≤ 18 minutes). Sensitivity for detecting sleep was ≥ 90% for both wearable devices and further improved to 95% by using the high threshold (H5) setting for the AW2 (0.95). Wake detection specificity was highest in Fitbit (≥ 0.88), followed by the AW2 at M10 (≥ 0.80) and H5 thresholds (≤ 0.73). In addition, Fitbit inconsistently estimated stage N1 + N2 sleep depending on TIB, underestimated stage N3 sleep (21-46 min), but was comparable to PSG for rapid eye movement sleep. Fitbit sensitivity values for the detection of N1 + N2, N3 and rapid eye movement sleep were ≥ 0.68, ≥ 0.50, and ≥ 0.72, respectively.ConclusionsA consumer-grade wearable device can measure sleep duration as well as a research actigraph. However, sleep staging would benefit from further refinement before these methods can be reliably used for adolescents.Clinical trial registrationRegistry: ClinicalTrials.gov; Title: The Cognitive and Metabolic Effects of Sleep Restriction in Adolescents; Identifier: NCT03333512; URL: https://clinicaltrials.gov/ct2/show/NCT03333512.CitationLee XK, Chee NIYN, Ong JL, Teo TB, van Rijn E, Lo JC, Chee MWL. Validation of a consumer sleep wearable device with actigraphy and polysomnography in adolescents across sleep opportunity manipulations. J Clin Sleep Med. 2019;15(9):1337-1346.
| S-EPMC6760396 | biostudies-literature
Project description:BackgroundActigraphs are wrist-worn devices that record tri-axial accelerometry data used clinically and in research studies. The expense of research-grade actigraphs, however, limit their widespread adoption, especially in clinical settings. Tri-axial accelerometer-based consumer wearable devices have gained worldwide popularity and hold potential for a cost-effective alternative. The lack of independent validation of minute-to-minute accelerometer data with polysomnographic data or even research-grade actigraphs, as well as access to raw data has hindered the utility and acceptance of consumer-grade actigraphs.MethodsSleep clinic patients wore a consumer-grade wearable (Huami Arc) on their non-dominant wrist while undergoing an overnight polysomnography (PSG) study. The sample was split into two, 20 in a training group and 21 in a testing group. In addition to the Arc, the testing group also wore a research-grade actigraph (Philips Actiwatch Spectrum). Sleep was scored for each 60-s epoch on both devices using the Cole-Kripke algorithm.ResultsBased on analysis of our training group, Arc and PSG data were aligned best when a threshold of 10 units was used to examine the Arc data. Using this threshold value in our testing group, the Arc has an accuracy of 90.3%±4.3%, sleep sensitivity (or wake specificity) of 95.5%±3.5%, and sleep specificity (wake sensitivity) of 55.6%±22.7%. Compared to PSG, Actiwatch has an accuracy of 88.7%±4.5%, sleep sensitivity of 92.6%±5.2%, and sleep specificity of 60.5%±20.2%, comparable to that observed in the Arc.ConclusionsAn optimized sleep/wake threshold value was identified for a consumer-grade wearable Arc trained by PSG data. By applying this sleep/wake threshold value for Arc generated accelerometer data, when compared to PSG, sleep and wake estimates were adequate and comparable to those generated by a clinical-grade actigraph. As with other actigraphs, sleep specificity plateaus due to limitations in distinguishing wake without movement from sleep. Further studies are needed to evaluate the Arc's ability to differentiate between sleep and wake using other sources of data available from the Arc, such as high resolution accelerometry and photoplethysmography.
| S-EPMC7498244 | biostudies-literature
Project description:BackgroundConsumer sleep trackers (CSTs) have gained significant popularity because they enable individuals to conveniently monitor and analyze their sleep. However, limited studies have comprehensively validated the performance of widely used CSTs. Our study therefore investigated popular CSTs based on various biosignals and algorithms by assessing the agreement with polysomnography.ObjectiveThis study aimed to validate the accuracy of various types of CSTs through a comparison with in-lab polysomnography. Additionally, by including widely used CSTs and conducting a multicenter study with a large sample size, this study seeks to provide comprehensive insights into the performance and applicability of these CSTs for sleep monitoring in a hospital environment.MethodsThe study analyzed 11 commercially available CSTs, including 5 wearables (Google Pixel Watch, Galaxy Watch 5, Fitbit Sense 2, Apple Watch 8, and Oura Ring 3), 3 nearables (Withings Sleep Tracking Mat, Google Nest Hub 2, and Amazon Halo Rise), and 3 airables (SleepRoutine, SleepScore, and Pillow). The 11 CSTs were divided into 2 groups, ensuring maximum inclusion while avoiding interference between the CSTs within each group. Each group (comprising 8 CSTs) was also compared via polysomnography.ResultsThe study enrolled 75 participants from a tertiary hospital and a primary sleep-specialized clinic in Korea. Across the 2 centers, we collected a total of 3890 hours of sleep sessions based on 11 CSTs, along with 543 hours of polysomnography recordings. Each CST sleep recording covered an average of 353 hours. We analyzed a total of 349,114 epochs from the 11 CSTs compared with polysomnography, where epoch-by-epoch agreement in sleep stage classification showed substantial performance variation. More specifically, the highest macro F1 score was 0.69, while the lowest macro F1 score was 0.26. Various sleep trackers exhibited diverse performances across sleep stages, with SleepRoutine excelling in the wake and rapid eye movement stages, and wearables like Google Pixel Watch and Fitbit Sense 2 showing superiority in the deep stage. There was a distinct trend in sleep measure estimation according to the type of device. Wearables showed high proportional bias in sleep efficiency, while nearables exhibited high proportional bias in sleep latency. Subgroup analyses of sleep trackers revealed variations in macro F1 scores based on factors, such as BMI, sleep efficiency, and apnea-hypopnea index, while the differences between male and female subgroups were minimal.ConclusionsOur study showed that among the 11 CSTs examined, specific CSTs showed substantial agreement with polysomnography, indicating their potential application in sleep monitoring, while other CSTs were partially consistent with polysomnography. This study offers insights into the strengths of CSTs within the 3 different classes for individuals interested in wellness who wish to understand and proactively manage their own sleep.
| S-EPMC10654909 | biostudies-literature
Project description:BackgroundVentilation monitoring during sleep is performed by sleep test instrumentation that is uncomfortable for the patients due to the presence of the flowmeter. The objective of this study was to evaluate if an innovative type 3 wearable system, the X10X and X10Y, is able to correctly detect events of apnea and hypopnea and to classify the severity of sleep apnea without the use of a flowmeter.Methods40 patients with sleep disordered breathing were analyzed by continuous and simultaneous recording of X10X and X10Y and another certified type 3 system, SOMNOtouch, used for comparison. Evaluation was performed in terms of quality of respiratory signals (scores from 1, lowest, to 5, highest), duration and classification of apneas, as well as identification and duration of hypopneas.Results580 periods were evaluated. Mean quality assigned score was 3.37±1.42 and 3.25±1.35 for X10X and X10Y and SOMNOtouch, respectively. The agreement between the two systems was evaluated with grades 4 and 5 in 383 out of 580 cases. A high correlation (r2 = 0.921; p<0.001) was found between the AHI indexes obtained from the two systems. X10X and X10Y devices were able to correctly classify 72.3% of the obstructive apneas, 81% of the central apneas, 61.3% of the hypopneas, and 64.6% of the mixed apneas when compared to SOMNOtouch device.ConclusionThe X10X and X10Y devices are able to provide a correct grading of sleep respiratory disorders without the need of a nasal cannula for respiratory flow measurement and can be considered as a type 3 sleep test device for screening tests.
| S-EPMC8051765 | biostudies-literature
Project description:Wearable, multisensor, consumer devices that estimate sleep are now commonplace, but the algorithms used by these devices to score sleep are not open source, and the raw sensor data is rarely accessible for external use. As a result, these devices are limited in their usefulness for clinical and research applications, despite holding much promise. We used a mobile application of our own creation to collect raw acceleration data and heart rate from the Apple Watch worn by participants undergoing polysomnography, as well as during the ambulatory period preceding in lab testing. Using this data, we compared the contributions of multiple features (motion, local standard deviation in heart rate, and "clock proxy") to performance across several classifiers. Best performance was achieved using neural nets, though the differences across classifiers were generally small. For sleep-wake classification, our method scored 90% of epochs correctly, with 59.6% of true wake epochs (specificity) and 93% of true sleep epochs (sensitivity) scored correctly. Accuracy for differentiating wake, NREM sleep, and REM sleep was approximately 72% when all features were used. We generalized our results by testing the models trained on Apple Watch data using data from the Multi-ethnic Study of Atherosclerosis (MESA), and found that we were able to predict sleep with performance comparable to testing on our own dataset. This study demonstrates, for the first time, the ability to analyze raw acceleration and heart rate data from a ubiquitous wearable device with accepted, disclosed mathematical methods to improve accuracy of sleep and sleep stage prediction.
| S-EPMC6930135 | biostudies-literature
Project description:BackgroundFrailty has detrimental health impacts on older home care clients and is associated with increased hospitalization and long-term care admission. The prevalence of frailty among home care clients is poorly understood and ranges from 4.0% to 59.1%. Although frailty screening tools exist, their inconsistent use in practice calls for more innovative and easier-to-use tools. Owing to increases in the capacity of wearable devices, as well as in technology literacy and adoption in Canadian older adults, wearable devices are emerging as a viable tool to assess frailty in this population.ObjectiveThe objective of this study was to prove that using a wearable device for assessing frailty in older home care clients could be possible.MethodsFrom June 2018 to September 2019, we recruited home care clients aged 55 years and older to be monitored over a minimum of 8 days using a wearable device. Detailed sociodemographic information and patient assessments including degree of comorbidity and activities of daily living were collected. Frailty was measured using the Fried Frailty Index. Data collected from the wearable device were used to derive variables including daily step count, total sleep time, deep sleep time, light sleep time, awake time, sleep quality, heart rate, and heart rate standard deviation. Using both wearable and conventional assessment data, multiple logistic regression models were fitted via a sequential stepwise feature selection to predict frailty.ResultsA total of 37 older home care clients completed the study. The mean age was 82.27 (SD 10.84) years, and 76% (28/37) were female; 13 participants were frail, significantly older (P<.01), utilized more home care service (P=.01), walked less (P=.04), slept longer (P=.01), and had longer deep sleep time (P<.01). Total sleep time (r=0.41, P=.01) and deep sleep time (r=0.53, P<.01) were moderately correlated with frailty. The logistic regression model fitted with deep sleep time, step count, age, and education level yielded the best predictive performance with an area under the receiver operating characteristics curve value of 0.90 (Hosmer-Lemeshow P=.88).ConclusionsWe proved that a wearable device could be used to assess frailty for older home care clients. Wearable data complemented the existing assessments and enhanced predictive power. Wearable technology can be used to identify vulnerable older adults who may benefit from additional home care services.
| S-EPMC7499164 | biostudies-literature
Project description:The functional validity of the signal obtained with low-cost electroencephalography (EEG) devices is still under debate. Here, we have conducted an in-depth comparison of the EEG-recordings obtained with a medical-grade golden-cup electrodes ambulatory device, the SOMNOwatch + EEG-6, vs those obtained with a consumer-grade, single dry electrode low-cost device, the NeuroSky MindWave, one of the most affordable devices currently available. We recorded EEG signals at Fp1 using the two different devices simultaneously on 21 participants who underwent two experimental phases: a 12-minute resting state task (alternating two cycles of closed/open eyes periods), followed by 60-minute virtual-driving task. We evaluated the EEG recording quality by comparing the similarity between the temporal data series, their spectra, their signal-to-noise ratio, the reliability of EEG measurements (comparing the closed eyes periods), as well as their blink detection rate. We found substantial agreement between signals: whereas, qualitatively, the NeuroSky MindWave presented higher levels of noise and a biphasic shape of blinks, the similarity metric indicated that signals from both recording devices were significantly correlated. While the NeuroSky MindWave was less reliable, both devices had a similar blink detection rate. Overall, the NeuroSky MindWave is noise-limited, but provides stable recordings even through long periods of time. Furthermore, its data would be of adequate quality compared to that of conventional wet electrode EEG devices, except for a potential calibration error and spectral differences at low frequencies.
| S-EPMC6630628 | biostudies-literature
Project description:ObjectivesConsumer-grade smart devices are now commonly used by the public to measure waking activity and sleep. However, the ability of these devices to accurately measure sleep in clinical populations warrants more examination. The aim of the present study was to assess the accuracy of three consumer-grade sleep monitors compared with gold standard polysomnography (PSG).DesignA prospective cohort study was performed.SettingAdults undergoing PSG for investigation of a suspected sleep disorder.Participants54 sleep-clinic patients were assessed using three consumer-grade sleep monitors (Jawbone UP3, ResMed S+ and Beddit) in addition to PSG.OutcomesJawbone UP3, ResMed S+ and Beddit were compared with gold standard in-laboratory PSG on four major sleep parameters-total sleep time (TST), sleep onset latency (SOL), wake after sleep onset (WASO) and sleep efficiency (SE).ResultsThe accelerometer Jawbone UP3 was found to overestimate TST by 28 min (limits of agreement, LOA=-100.23 to 157.37), with reasonable agreement compared with gold standard for TST, WASO and SE. The doppler radar ResMed S+ device underestimated TST by 34 min (LOA=-257.06 to 188.34) and had poor absolute agreement compared with PSG for TST, SOL and SE. The mattress device, Beddit underestimated TST by 53 min (LOA=-238.79 to 132) on average and poor reliability compared with PSG for all measures except TST. High device synchronisation failure occurred, with 20% of recordings incomplete due to Bluetooth drop out and recording loss.ConclusionPoor to moderate agreement was found between PSG and each of the tested devices, however, Jawbone UP3 had relatively better absolute agreement than other devices in sleep measurements compared with PSG. Consumer grade devices assessed do not have strong enough agreement with gold standard measurement to replace clinical evaluation and PSG sleep testing. The models tested here have been superseded and newer models may have increase accuracy and thus potentially powerful patient engagement tools for long-term sleep measurement.
| S-EPMC8578969 | biostudies-literature
Project description:BACKGROUND:The proliferation of wearable devices that collect activity and heart rate data has facilitated new ways to measure sleeping and waking durations unobtrusively and longitudinally. Most existing sleep/wake identification algorithms are based on activity only and are trained on expensive and laboriously annotated polysomnography (PSG). Heart rate can also be reflective of sleep/wake transitions, which has motivated its investigation herein in an unsupervised algorithm. Moreover, it is necessary to develop a personalized approach to deal with interindividual variance in sleep/wake patterns. OBJECTIVE:We aimed to develop an unsupervised personalized sleep/wake identification algorithm using multifaceted data to explore the benefits of incorporating both heart rate and activity level in these types of algorithms and to compare this approach's output with that of an existing commercial wearable device's algorithms. METHODS:In this study, a total of 14 community-dwelling older adults wore wearable devices (Fitbit Alta; Fitbit Inc) 24 hours a day and 7 days a week over period of 3 months during which their heart rate and activity data were collected. After preprocessing the data, a model was developed to distinguish sleep/wake states based on each individual's data. We proposed the use of hidden Markov models and compared different modeling schemes. With the best model selected, sleep/wake patterns were characterized by estimated parameters in hidden Markov models, and sleep/wake states were identified. RESULTS:When applying our proposed algorithm on a daily basis, we found there were significant differences in estimated parameters between weekday models and weekend models for some participants. CONCLUSIONS:Our unsupervised approach can be effectively implemented based on an individual's multifaceted sleep-related data from a commercial wearable device. A personalized model is shown to be necessary given the interindividual variability in estimated parameters.
| S-EPMC7439146 | biostudies-literature
Project description:There is a growing need for free-living monitoring of the full 24 h spectrum of behaviors with a single or integrated set of sensors. The validity of field standard wearable monitors in sleep and physical activity have yet to be assessed for the complementary behavior in the context of 24 h continuous monitoring. We conducted a free-living comparison study of the Actigraph GT3X+ (GT3X+) to assess sleep parameters as compared with the Actiwatch-64 (AW-64) and concurrently, the AW-64 to assess sedentary and physical activity behaviors as compared with the GT3X+. Thirty young adults (15 female, 19.2±0.86 years) wore both monitors for 3 consecutive days and 2 consecutive nights. Agreement of sleep, sedentary, and physical activity metrics were evaluated using analyses of variance, intraclass correlation coefficients, Bland-Altman plots with associated confidence limits, mean absolute percentage of errors and equivalence tests. For sleep, the GT3X+ showed high agreement for total sleep time and sleep efficiency, but underestimated wakefulness after sleep onset and sleep onset latency relative to the AW-64. For sedentary behavior and physical activity, the AW-64 showed a moderate agreement for activity energy expenditure, but not for sedentary, light or moderate-vigorous physical activities relative to the GT3X+. Overall our results showed good agreement of the GT3X+ with AW-64 for assessing sleep but a lack of agreement between AW-64 and GT3X+ for physical activity and sedentary behaviors. These results are likely due to the monitor placement (wrist vs hip), as well as the algorithm employed to score the data. Future validation work of existing and emerging technologies that may hold promise for 24 h continuous monitoring is needed.
| S-EPMC4783247 | biostudies-literature