The validity of HRV results using wearables has been questioned in academia. Wearable devices usually adopt photoplethysmography (PPG), using the photo sensors in the device, rather than electrocardiograph (ECG) that is used in medical and strap based heart rate monitors (HRM), to measure HRV. The top wearable devices use algorithms to compensate for the inaccuracies using PPG.
We purchased the latest Garmin Fenix 7X solar to test how well these algorithms work.
We took readings with the Fenix while simultaneously taking readings with the HRV Health Pro app, using the Polar H10 HRM. The HRV Health app takes the reading for 120 beats, which in my case makes the reading a little under three minutes. When taking the Health Snapshot on the Fenix the reading takes 120 seconds, so in my case the Health Snapshot is a little shorter.
The rMSSD figures were usually within 5ms of each other for the daily reading.
The Fenix can be linked to a heart rate monitor for the Health Snapshot, and this is where the results started to vary significantly.
The Fenix overstates the results, often by more than 30ms. Here are the most recent examples:
We uploaded raw HRV Health data to the Kubios HRV standard application to test our results. The Kubios application confirmed that our algorithms have produced the correct results.
This finding is extraordinary. Both the Fenix and the HRV Health app are linked to the same HRM at the same time, and yet the results are very different.
We have reverted to using the PPG sensors for taking the Health Snapshot HRV reading. Here are the most recent results:
There are three notable differences between the Fenix and the HRV Health app.
On investigation, the results on 2024-01-16 were found to differ because the HRV Health platform adjusts for ectopic beats. It appears that for this reading there were two ectopic beats, which the Fenix does not adjusted for.
On 2024-01-21, there was a difference of 68ms. There were no ectopic beats in the reading, and the difference cannot be explained.
On 2024-01-25, there was a difference of 59ms. There were no ectopic beats in the reading, and the difference cannot be explained.
On 2024-01-30, the Fenix would not take a reading. Removing the watch, cleaning it, and then tightening the strap made no difference. The following day, it worked fine.
After 9 months of data with both the Fenix and the HRV Health app using the Polar H10 heart rate monitor and have uncovered some relevant anomalies. The average difference between the two sets of readings is 39.9%. But when the rMSSD drops below 50, where it matters most for measuring health, the difference rises to an unacceptable 58.8%.
The difference between the wearable and the H10 is inconsistent. Some days, the Fenix is spot on. On most it is not. We have observed a maximum difference of 277.7%. The Fenix’s HRV metrics are not even a reasonable resemblance of the actual measure. They are completely unreliable.
These results are consistent with scientific research comparing wearable devices with medical measuring equipment that found a moderate concordance of 0.6617 for rMSSD.
Another issue we uncovered with the Fenix is the measurement of resting heart rate (RHR). RHR is measured at rest, not while asleep. The Fenix measures RHR while asleep. This overstates the results for all metrics where RHR is one of the variables used in the formula.
The measurement of sleep is also wildly inaccurate. The Fenix records inactivity as sleep, and the movements when going to bed as sleep interruption. The resulting score is complete nonsense.
I have contacted Garmin tech support for comment. I will publish their response, if and when they reply.
Sleep quality influences health, and with that the HRV metrics. We have compared the Fenix sleep results to those produced with the Amazfit GTS 3 and found that both watches are reliable, do not treat inactivity as sleep, and correctly identify napping. Longer naps, especially after midday can have a deleterious impact on your ability to fall asleep when you go to bed at night. This in turn has a negative impact on sleep quality, and with that impacts both your sleep score and your HRV results.
Read more:
Can PPG be used for HRV analysis?
How To Measure Your Resting Heart Rate
Garmin HRV Status Explained: What It Means for Training and Recovery