Fitness trackers' calorie measurements are prone to error

“Fitness trackers out of step when measuring calories, research shows,” The Guardian reports. An independent analysis of a number of leading brands found they were all prone to inaccurate recording of energy expenditure.

Researchers recruited 60 participants to take part in a range of exercises while having their heart rate and number of calories burned measured by fitness trackers, as well as by clinically-approved medical devices used in a clinical setting. Seven fitness trackers were tested, including the Apple Watch, Fitbit Surge and the Samsung Gear S2.

The data from the fitness trackers was compared against the data obtained by the clinically-approved devices to calculate any errors in the measurements.

The researchers found that although fitness trackers are generally reliable in their ability to measure heart rate, they perform poorly when measuring the number of calories burned. The results showed that, out of all seven devices, the Apple Watch had the lowest error in measurements whereas the Samsung Gear S2 had the highest level of error in terms of heart rate measurement and the PulseOn in terms of calorie burning measurement.

For now, fitness trackers remain useful for individuals to have an idea of how many calories they may have burned over a day but it should be kept in mind that these devices are not always 100% accurate, as shown by this study.

And if you are trying to lose weight, focusing on an overall weekly goal for exercise and activity may be a better approach than trying to measure every single calorie burning exercise from one day to the next.

Read more advice about meeting your weight-loss goals.

Where did the story come from?

The study was carried out by researchers from Stanford University in the US and the Swedish School of Sport and Health Sciences in Stockholm. No external sources of funding were reported.

The study was published in the peer-reviewed Journal of Personalized Medicine. The article is available on an open access basis, which means it’s free to read online.

UK media coverage on this topic was generally accurate.

What kind of research was this?

This was an experimental study that assessed how accurately seven fitness trackers are able to measure heart rate and calories burned compared to a “gold standard” assessment of heart rate and energy expended in an exercise laboratory.

Commercially available fitness trackers are increasingly being used by individuals as part of fitness monitoring or weight loss programmes. The results from the devices are often compared to average population levels and in theory could inform decisions made by doctors. Prior to this study the accuracy of the data produced by consumer trackers hasn’t been evaluated in great detail.

This type of study is a useful way to compare the different trackers against a “gold standard” measurement and will help individuals decide how much trust they can have in their devices.

What did the research involve?

The researchers chose fitness trackers which:

• are worn on the wrist
• continuously measure heart rate
• have a battery life of more than 24 hours
• were available to consumers at the time of the study

Seven fitness trackers were evaluated:

• Apple Watch
• Basis Peak
• Fitbit Surge
• Microsoft Band
• Mio Alpha 2
• PulseOn
• Samsung Gear S2

60 participants (29 men and 31 women) over the age of 18 were recruited to take part in the assessment. Participants were selected to represent a diverse range of age, height, weight, skin tone and fitness level.

The individuals wore the fitness trackers while sitting, walking, running and cycling. They were simultaneously monitored using clinically approved instruments: electrocardiograms (ECG) and continuous clinical grade indirect calorimetry.

Indirect calorimetry is a laboratory technique for measuring fitness in which the maximum oxygen intake while breathing on the treadmill or bicycle ergometer is measured and a standard formula is used to estimate energy expended. It was used here as a gold standard for measuring calories burned.

The data from the fitness trackers was analysed against data from these clinically approved instruments. Heart rate from the fitness trackers was compared with data obtained from the ECG and the number of calories burned was compared with the data from the indirect calorimetry. The percent of error in relation to the clinical devices was then calculated.

What were the basic results?

Overall, the researchers found that most fitness trackers were able to measure heart rate fairly accurately, but poorly estimated the number of calories burned.

Of the seven fitness trackers, the Apple Watch performed best whilst the Samsung Gear S2 had the largest errors when measuring heart rate and calories burned.

Heart rate:

• The Apple Watch achieved the lowest error when measuring heart rate: 2.0% (1.2%-2.8%).
• Samsung Gear S2 had the highest error: 6.8% (4.6%-9.0%).

Calories burned:

• The Fitbit Surge performed best with the lowest error of all seven trackers: 27.4% (24.0%-30.8%).
• PulseOn had the highest error: 92.6% (87.5%-97.7%).

How did the researchers interpret the results?

The researchers concluded: “We assessed, in a controlled laboratory setting, the reliability of seven wrist-worn devices in a diverse group of individuals performing walking, running and cycling at low and high intensity. We found that in most settings, heart rate measurements were within acceptable error range (5%). In contrast, none of the devices provided estimates of energy expenditure that were within an acceptable range in any setting.

“Individuals and practitioners should be aware of the strengths and limitations of consumer devices that measure heart rate and estimate energy expenditure.”

Conclusion

This study assessed how accurately seven fitness trackers are able to measure the heart rate and calories burned of individuals taking part in several different activities. The data was compared against clinically approved medical devices to test the accuracy of data obtained by the fitness trackers.

It found that although all seven trackers were fairly accurate at measuring heart rate, there was a high level of error when measuring the number of calories burned.

The researchers hope this study will help individuals and physicians be aware of potential errors when interpreting the measurements obtained by fitness trackers, especially when using the data to inform treatment options in a clinical setting.

This is an interesting study but it is small and would require further testing of devices with a larger number of participants to verify the findings. For now, fitness trackers remain useful if you want to compare data on yourself over time, but they shouldn’t be relied upon if you’re trying to replace calories burned with a “treat”.

If your goal is to lose weight through exercise, then bear in mind it’s very much a “marathon not a sprint”. Increasing your activity and exercise levels on a long-term basis is more important than obsessing about exactly how many calories you may have burned during a single run or gym session.

The NHS weight loss plan is designed to help you lose weight, through a combination of diet and exercise, over the course of 12 weeks.  

Links To The Headlines

Fitness trackers out of step when measuring calories, research shows. The Guardian, May 24 2017

Fitness trackers ‘poor at measuring calories burned’. BBC News, May 25 2017

Fitness trackers may be incorrectly counting calories burnt, finds research. The Independent, May 25 2017

Links To Science

Shcherbina A, Mattsson CM, Waggott D, et al. Accuracy in Wrist-Worn, Sensor-Based Measurements of Heart Rate and Energy Expenditure in a Diverse Cohort. Journal of Personalised Medicine. Published online May 24 2017