Abstract
-
Objective
Our objective was to study the clinical monitoring capabilities of a low-cost fitness wristband while measuring patient satisfaction with a mobility permitting device in the emergency department.
-
Methods
Patients enrolled were on continuous three-lead telemetry monitoring in a high acuity zone of the emergency department. Patients were given a fitness band to wear while simultaneously monitored with standard three-lead monitor. A brief survey was conducted upon study end, and data was compared between wristband and three-lead telemetry. Median heart rate (HR) values were calculated, a Bland-Altman plot was generated, and sensitivity and specificity were calculated for comparison of the formal telemetry and the inexpensive wristband.
-
Results
Thirty-four patients with an average age of 61.5 years were enrolled. From June to October 2019, over 100 hours of data were collected. In comparison for comfort, participants scored 9.5 of 10, preferring wristband over telemetry. Using a correlation coefficient graph, we found a significant disparity of HR readings within a telemetry range of 40 to 140 beats/min. An R-value of 0.36 was detected. Using a Bland-Altman plot, we observed a significant difference in HR between the telemetry monitor and the wristband. The sensitivity and specificity of the wristband to detect bradycardia (HR <60 beats/min) were 76% and 86%, respectively, while the sensitivity and specificity of the wristband to detect tachycardia (HR >100 beats/min) were 92% and 51%, respectively.
-
Conclusion
Inexpensive fitness bands cannot be a suitable tool for monitoring patient’s HR because of inaccuracy in detecting bradycardia or tachycardia.
-
Keywords: Telemedicine; Monitoring, physiologic; Fitness trackers; Internet of things; Global health
Capsule Summary
What is already known
Exercise monitors have become less expensive and are universally used by patients in nonclinical settings.
What is new in the current study
While expensive exercise monitoring has been tested for clinical validity, a low cost fitness monitor has not been tested. Our goal was to find applicability of a low-cost fitness device for clinical use especially in low-resource emergency department settings. The combination of a low-cost heart monitor and a commercially available open source application were not able to reliably detect heart rates at a suitable safety level for clinical use.
INTRODUCTION
The practice of medicine in low-resource settings is fraught with challenges. In addition to scarce medical supplies, lack of infrastructure, and lack of financial support, there are often insufficient tools to adequately monitor and treat patients [
1]. While nursing ratios in the United States and Canada are on average 1:4.4 on general medicine wards, and as high as 1:1 or 1:2 for critically ill patients [
2], in low-resource settings these ratios can be as high as 1:31 [
3]. Under these conditions, it is no surprise that tasks as fundamental as routinely documenting vital signs can be untenable [
3-
5]. While telemetry monitoring is a useful tool to extend monitoring capability, it is often out of the financial reach of lower resource emergency departments (EDs) and hospitals.
As the authors have previously proposed [
6], there may be an opportunity to bridge this gap through the use of commercially available low-cost fitness monitoring devices. Increasingly ubiquitous fitness monitoring devices and smart watches monitor the heart rate (HR) through an inexpensive optical technique known as photoplethysmography [
7]. While there have been multiple studies validating moderate to high-cost exercise monitoring devices in the 150 to 400 US dollars range [
8-
10], there are now ultra-low-cost monitors available in the <50 US dollars range that offer the possibility of use in the lowest resource settings. These ultra-low-cost monitors have yet to be tested in clinical settings. In this study, we examine the clinical monitoring capabilities of a low-cost fitness band, which is available commercially in the 20 to 40 US dollars price range. Our objective was to study the clinical monitoring capabilities of a low-cost fitness wristband while measuring patient satisfaction with a mobility permitting device in the ED.
METHODS
Study design
Study participants were enrolled in the ED at Beth Israel Deaconess Medical Center located in Boston, Massachusetts, approved by the local institutional review board (2018P000380). Each participant presented with various medical complaints, not always involving cardiovascular issues. Due to the observational nature of the study, written consent was waived and verbal informed consent was utilized. Patients undergoing continuous telemetry as part of routine care were screened and selected by research staff to approach and obtain verbal consent. The decision to enroll a patient was based on if the patient was on continuous three-lead telemetry monitoring as well as placed in the high-acuity area of the ED in a tertiary, urban, academic medical center. These patients were of greater interest due to the potential for a more dynamic range of HR in critically ill patients. Patients were excluded if they were <18 years of age, unable to comfortably fit the device on their wrist, were on infectious disease contact precautions, had an allergy to silicone or aluminum, or if the application of the device would interfere with clinical care. Particular attention was given to those patients who were bradycardic and tachycardic so that the device could be tested on a broad range of HR. Patients who met no exclusion criteria were consented for inclusion in the study and a Xiao-mi Pulse fitness band Xiamoi Company, Beijing, China was applied to their wrist. This wristband communicated via Bluetooth to a nearby Android tablet, the Samsung Galaxy 8-inch edition manufactured by Samsung Electronics Co, Seoul, South Korea. While the included Xiao-mi Android application is only able to obtain “spot check” HR measurements, by using a publicly available application called “Notify & Fitness for Mi Band”, designed by OneZeroBit, Padova, Italy, we were able to obtain HR data as frequently as once per second. Patients were simultaneously monitored with a standard three-lead monitor as part of their clinical care, which sent HR information to our central hospital database every minute. When the subject was ready to leave the ED (either admitted or discharged), researchers removed the wristband and administered a brief verbal survey regarding the patient experience as well as collected basic demographic information.
Data collection
The android application session for each subject, including HR data and survey answers, were exported into a REDCap database with a subject number ID assigned for each participant. Contemporaneous HR telemetry data were exported to the REDCap database for comparison. Preliminary data analysis was performed when 100 hours of HR data had been collected. During this period, only 34 patients were enrolled from June to October 2019, collecting over 100 hours of data. Median HR values were calculated for every minute of wristband data for comparison with the gold standard telemetry data. Data analysis was performed in R version 4.0.0 (R Foundation for Statistical Computing, Vienna, Austria). Correlation between the two data sets was calculated utilizing the Pearson method. A Bland-Altman plot was generated, and sensitivity and specificity were calculated for identification of tachycardic and bradycardic episodes. The primary outcome of the study was the correlation between the gold standard telemetry and the wristband monitor. The secondary outcome was patient acceptability of this device. A convenient sample of patients was enrolled until 100 hours of data were collected.
RESULTS
Total 34 participants were enrolled. Females (n=22) accounted for 64.7% of the total study participants, averaging 61.5 years of age with a range of 22 to 88 (
Table 1). Satisfaction survey response descriptive statistics are reported in
Tables 1 and
2. In comparison to standard three-lead telemetry, participants notably scored in favor of the wristband with a satisfaction score of 9.5 of 10, an overall comfort score of 9.6 and preferred wristband (score of 10) over telemetry (score of 1) with a mean of 9.8 (
Table 2).
Fig. 1 shows the correlation coefficient between the median HR values of the wristband compared to the telemetry monitor. There was significant disparity of HR readings within a telemetry range of 40 to 140 beats/min. Significant bidirectional spikes were seen for the median watch HR to as high as 224 beats/min and as low as 26 beats/min. As a result, a low R-value of 0.36 was detected, thus indicating a poor correlation between the two HR monitoring devices.
A Bland-Altman plot was used to determine how the performance of the wristband compared to the telemetry across a range of HRs (
Fig. 2). We observed a significant difference in HRs between the telemetry monitor and the wristband. As the HR increased above tachycardia (HR >100 beats/min), a the magnitude of error grew significantly larger. At times, the wristband data showed a HR up towards the 200’s while the telemetry monitor showed a HR in the 60s.
The wristband’s sensitivity and specificity to detecting bradycardia (HR <60 beats/min) and tachycardia were also investigated (
Figs. 3,
4). The sensitivity and specificity of the wristband to detect bradycardia were 76% and 86%, respectively, while the sensitivity and specificity of the wristband to detect tachycardia were 92% and 51%, respectively.
Additionally, 39% of the data points captured by telemetry had no corresponding data collected by the wristband and did not contribute to our analysis.
DISCUSSION
While the overall satisfaction of using the wristband highly preferred over using the three-lead telemetry monitor, we found an unacceptable level of error in this device. Additionally, it is important to note that of the telemetry monitor data collected, 39% of those minutes were missing data from the wristband. This is likely due to the wristband moving out of range of the device or the application intermittently failing to connect to Bluetooth.
Figs. 1 and
2 highlight the significant discrepancies seen between the devices. Some of the data collected by the wristband demonstrated HRs as high as the 200 beats/min while the gold standard telemetry monitor read a HR of 60 beats/min. Many of these anomalies appeared to cluster, with the observation of HRs reading reliably for some time, and then suddenly having several minutes of incorrect elevated HRs in the 200 to 300 beats/min range. This phenomenon likely explains the significant discrepancies visualized in tachycardic range of our Bland-Altman analysis (
Fig. 2). We suspect that this intermittent error state likely contributed to much of error between the wristband and the gold standard telemetry
We also specifically examined the ability of the device to detect bradycardic and tachycardic events as they can be significant clinical indicators of adverse events.
Figs. 3 and
4 illustrate these detection rates of both bradycardia and tachycardia in the wristband device. We found unacceptable sensitivity and specificity for detecting both bradycardia (sensitivity 76%, specificity 86%) and tachycardia (sensitivity 92%, specificity 51%). Our analysis indicates that the use of a low-cost wristband with a publicly available monitoring application is not accurate enough for clinical use as a monitoring device.
In conclusion, utilizing a low-cost fitness band tested in our study is not a suitable tool for clinical monitoring. We found reliability, correlation, sensitivity, and specificity all to be unacceptable for clinical use. We hypothesize that the observed sudden periods of false tachycardia were most likely the cause of the significant deviation in test characteristics from prior studies on more expensive devices. This error state led to significant skew in test characteristics and tachycardia detection. It is likely that these errors were secondary to software errors rather than limitations of the photoplethysmography method itself. We observed no clear trigger to entering the error state and noted that it would often take several minutes to correct itself. Additionally, we observed a failure of connectivity 39% of the time. This did not appear to correct when distance between device and android tablet was decreased. As such, this inexpensive fitness band could not be a suitable tool for monitoring patient’s HR because of inaccuracy in detecting bradycardia or tachycardia.
Going forward, it is clear the patient acceptability is high for a wristband telemetry monitor. Future studies with a wristband device designed specifically for continuous clinical monitoring may have better success. We suspect that by using a publicly available application to operate the wristband in a manner in which it was not originally designed, significant issues with performance stability were introduced, notably Bluetooth connection issues. High loss rate was due to software instability inherent to utilization of a non-native application. This can be easily improved through purpose-built software and hardware but would require the initial investment of funds for development and would not provide the “off-the-shelf” availability that we had hoped for in this project. Unfortunately, at this time, there is no commercially available device at this ultra-low price point that offers out-of-the-box continuous monitoring. We are hopeful that as the trend towards less expensive and more reliable monitoring equipment continues, we may have the opportunity to extend monitoring capabilities to low resource clinical settings. Unfortunately, we are not there yet.
In conclusion, utilizing a low-cost fitness band in our study with a publicly available monitoring application is not a suitable tool for clinical monitoring.
As such, this inexpensive fitness band could not be a suitable tool for monitoring patient’s HR because of inaccuracy in detecting bradycardia or tachycardia.
Going forward, it is clear the patient acceptability is high for a wristband telemetry monitor. Future studies with a wristband device designed specifically for continuous clinical monitoring may have better success. We suspect that by using a publicly available application to operate the wristband in a manner in which it was not originally designed, significant issues with performance stability were introduced, notably Bluetooth connection issues. High loss rate was due to software instability inherent to utilization of a non-native application. This can be easily improved through purpose-built software and hardware but would require the initial investment of funds for development and would not provide the “off-the-shelf” availability that we had hoped for in this project. Unfortunately, at this time, there is no commercially available device at this ultra-low price point that offers out-of-the-box continuous monitoring. We are hopeful that as the trend towards less expensive and more reliable monitoring equipment continues, we may have the opportunity to extend monitoring capabilities to low resource clinical settings. Unfortunately, we are not there yet.
NOTES
-
No potential conflict of interest relevant to this article was reported.
Fig. 1.Correlation coefficient of R=0.36 when median heart rates of the wristband were compared to median heart rates of the telemetry.
Fig. 2.A Bland-Altman plot comparing the wristband to the telemetry heart rates readings. The larger the discrepancy between the two, the greater the distance from 0 a figure will appear. “Measurements” on X-axis are defined as heart rate.
Fig. 3.Area under the Receiver Operating Characteristic curve showing sensitivity and specificity of the wristband to detect a heart rate less than 60 beats/min. The sensitivity of detecting a heart rate <60 beats/min was 76% while the specificity of detecting heart rate <60 beats/min was 86%. AUC, area under curve.
Fig. 4.Area under the Receiver Operating Characteristic curve showing sensitivity and specificity of the wristband detecting a heart rate greater than 100 beats/min. The sensitivity of detecting a heart rate >100 beats/min was 92% while the specificity of detecting hear rate >100 beats/min was 51%. AUC, area under curve.
Table 1.Demographics and survey answers collected following the study (n=34)
Table 1.
|
Value |
|
Age (yr) |
61.5 (20–88) |
|
Sex, female |
22 (64.7) |
|
On a scale of 1 (extremely uncomfortable) to 10 (extremely comfortable) how would you rate the comfort of the wristband monitor you wore today? |
9.6/10 |
|
How does the comfort of the wristband compare to the comfort of the telemetry system (chest sticker and wires) that you were wearing today? (1 = much less comfortable, 10 = much more comfortable) |
9.5/10 |
|
If it was determined to be effective, would you prefer to wear a wristband monitor during future emergency department visits rather than the telemetry (chest stickers and wires) that you wore today? (1 = absolutely not, 10 = absolutely would) |
9.8/10 |
Table 2.Statistical analysis of survey answers collected following the study
Table 2.
|
Overall (n = 34) |
|
Comfort |
|
|
Mean ± standard deviation |
9.6 ± 1.2 |
|
Median (min, max) |
10.0 (4.0, 10.0) |
|
Missing (%) |
8 (22.2) |
|
Comparison |
|
|
Mean ± standard deviation |
9.5 ± 1.3 |
|
Median (min, max) |
10.0 (5.0, 10.0) |
|
Missing (%) |
8 (22.2) |
|
Preference |
|
|
Mean ± standard deviation |
9.8 ± 0.7 |
|
Median (min, max) |
10.0 (7.0, 10.0) |
|
Missing (%) |
8 (22.2) |
REFERENCES
- 1. Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet 2005;366:1026-35.
- 2. Kane RL, Shamliyan T, Mueller C, Duval S, Wilt TJ. Nurse staffing and quality of patient care. Evid Rep Technol Assess (Full Rep) 2007;151:1-115.
- 3. Ogero M, Ayieko P, Makone B, et al. An observational study of monitoring of vital signs in children admitted to Kenyan hospitals: an insight into the quality of nursing care? J Glob Health 2018;8:010409.
- 4. Asiimwe SB, Okello S, Moore CC. Frequency of vital signs monitoring and its association with mortality among adults with severe sepsis admitted to a general medical ward in Uganda. PLoS One 2014;9:e89879.
- 5. Rosen MA, Chima AM, Sampson JB, et al. Engaging staff to improve quality and safety in an austere medical environment: a case-control study in two Sierra Leonean hospitals. Int J Qual Health Care 2015;27:320-7.
- 6. Dagan A, Mechanic OJ. Use of ultra-low cost fitness trackers as clinical monitors in low resource emergency departments. Clin Exp Emerg Med 2020;7:144-9.
- 7. Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiol Meas 2007;28:R1-39.
- 8. Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act 2015;12:159.
- 9. Shcherbina A, Mattsson CM, Waggott D, et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J Pers Med 2017;7:3.
- 10. Ferguson T, Rowlands AV, Olds T, Maher C. The validity of consumer-level, activity monitors in healthy adults worn in free-living conditions: a cross-sectional study. Int J Behav Nutr Phys Act 2015;12:42.
