Quite surprised by how bad sleep tracking is (at least on scientific level) even though I'm quite happy with their Nightly recharge numbers/evaluation. As for steps, keep in mind that Polar converts every move into steps so in their philosophy getting steps while washing dishes is OK. Finally for sports, there are only a few brands that are for sports: Polar (every watch except Unite), Garmin (Forerunner, Fenix, etc.), Coros and Suunto as they give you sports analytics. Others just give you data.
@@TheQuantifiedScientist Thanks for making the trip over! Had a great time chatting! And, also, some hilarious outtakes. It’s a good thing neither of us play any sort of handball games 🤾 🤣
I love it that you don't run but cycle! I also have an injury, but on my ankle. Also I think it is better to adjust to gps+galileo because you're from europe. Because I think people who are looking at review videos do take the time to adjust the settings.
I'm getting much, much better HR accuracy when cycling with Pacer Pro. It was very reliable for me. Better than my dedicated cycling HR sensor (Wahoo Tickr Fit) and quite similar to Polar H10.
Hey Rob :) Great job as usual :) One potential idea for improvement of the accuracy: take the plot from 3:31 as an example. I would allow for a constant shift in time, i.e. optimize the RMSE of watch vs. strap with one free parameter - time delay. And then use the time delay as another metric, the shorter the better of course. Imagine a perfect device, that is 100% accurate, but has a 5 minute delay. A plot like the one from 3:31 would be essentially two curves in opposite phases. A parity plot on that data would have a negative R^2 (worse than a horizontal constant), while in fact the watch would be quite good, only a bit slow in giving you that good information. I think it would be much more valuable, depending on what the potential user is expecting. I don't care at all what the delay is, as long as the curve overall is accurate, while some users (maybe when they do quick intervals of something like that), would much rather sacrifice a bit of accuracy for a nearly instantaneous response. And I'm pretty sure there would be some correlation between the fitted delay value and R^2, and those that fall below that correlation would be the really bad ones. One parameter that exists in these HR algorithms is the data acquisition time window that produces a single HR value. Larger window (reasonably large of course, so that the finer detail is not lost) would likely produce better results, but at a cost of a larger delay, while a small window would produce a quick response but at a cost of accuracy (false readings, HRV etc. would be caught as data and not averaged/filtered out). And then you would have the likes of the apple watch where the algorithm is smart enough to adjust that window based on the conditions so you would have zero delay device with a very high R^2.
Indeed, these devices can make time keeping errors when they record data. In theory, both red and green points should be equally spaced in time axis in HR vs time graphs. Rob can easily verify this by looking at the raw data. If that is not the case, i.e. time interval between adjacent points is not constant, then he can omit time stamps and just input counting numbers for x axis like x = [1,2,3,…,3000] for a 50 minute workout assuming 1Hz record interval for the device while y = HR. I know Polar H10 records HR every 1s. I don’t know record frequency of Polar Pace Pro but it seems similar to Polar H10, .i.e. 1Hz. Footnote: with Polar Sensor Logger on Android, it is possible to log raw ECG signal from Polar H10 at 125Hz. On the other hand, I totally understand Rob, because he is doing the whole thing himself, each of which requires an immerse time, do the actual physical activity, write the code and analyze the data, edit the video. All of these takes much time, I know. He is not lending the wearables to his friends and say, can you do these activities for me and bla bla. Ok, this is good in some sense, he is voluntarily doing sport, good for his health but man, it is too much of a work ! Some companies don’t even share user data with user easily in csv format. I once saw Rob extracting data from images with plot digitizers. Do you know how excruciating it is ? So, when there are companies around like Apple and Huawei who solved not only accurate HR but also time keeping issues, why would Rob bother? Like all of what he is doing not enough. It’s companies’ job to solve it.
Love your channel! Have been recommending it to others. Keep it up. Looking forward to your review of the circular ring 🇫🇷 and how that performs according to your assessments.
Thank you for a great review, focusing on the Heart Rate sensor accuracy. I was starting to think my PacerPro was defective due to the blatant lack of accuracy in Heart Rate monitoring (vs Polar ECG chest-strap) … no I guess I’m keeping the watch but will go back to wearing the chest-strap … or maybe test the veritysense(?)
I have the same issue with my vantage M2. The heart rate monitoring is way off vs. the Verity Sense especially during light motion like walking... Makes me wonder if the problem is with an overzealous motion algorithm cancelling out legit heart rate readings. Extremely disappointed given the reputation of Polar.
I didn't know that I needed this channel until now. Awesome work man, thanks a lot for share your research. Could you make a review about Coros Apex Pro? Greetings from Argentina
Nice work as always! I just can't understand why you didn't check the heart rate monitor during running as it is the most common use for these type of watches. You also have to specify what kind of weight lifting session are you performing each time for each watch. You have to perform exactly the same exercises for exactly the same time for each one. Otherwise it's not a fair comparison. For example you can't compare a session which includes dead lifts, squat and leg extension with one which includes chest fly's,push ups and biceps curls. At the first case every watch would perform better than the second (because of the reduced wrist movement). If you did so,I am afraid that neither your favorite Apple watch would perform such perfect as you claim. It's just a thought that may help you! Keep up the good work!
@@philth7587 That's why Physiotherapists exist. Come on my friend, that's an excuse. He could delay his review until he recovers. Running performance is a basic issue.
Wooow... that's creepy a went from the fenix 7 update video directly to this and I stopped the video and wondered if I've done anything wrong for a moment... trippy🤯🤯
Hi Rob, this was a great review. how come you never mention Muse technology? They are getting very big in sleep and meditation world. Particularly, MUSE S Gen-2 for Heat rate, breath, Oxygen, Sleep and meditation. Thanks @The Quantified Scientist for all your hard work!!
Agreed. Muse S gen 2 must be on the review list. However, there are issues with Muse. First of all, I own Muse S gen 2 and have used it for months. Muse is not investing on sleep tracking capabilities of their device. They are not improving it. Since it is an EEG device, I still believe it is tracking sleep better than wrist-worn PPG devices. However, sometimes it mistracks my sleep, giving zero time for REM sleep for example. Their sleep heart rate graphs are wrong. Sleep heart rates must have occasional spikes, they smooth them all out, sacrificing accuracy. They don’t show sleep blood oxygen saturation, although the device has pulse oximeter. They don’t push too much of an update, yet there are many areas to be improved. This is an EEG device, correct sleep staging involves sleep stages like N1,N2,N3, REM and awake. So they can show sleep stages in greater detail but they don’t develop it. While N1 stage can be taken as light sleep, N2 stage contains sleep spindles and K-complexes visible in raw EEG data. N2 is merged with deep sleep N3 and presented as a single stage. While this can be ok for a wrist worn PPG device, one can ask more from an EEG device. The main concern with Muse is that they don’t allow users to export their sleep data. This is totally unacceptable. Year 2022! How do you expect Rob to compare Muse with other devices if he cannot access his data ? I don’t like companies embracing this mindset. Ok, sleep data can be synced with Apple Health but not with Google Fit, but still how much sleep information is shared with Apple Health is a question mark for me. I am an Android user. So if Rob ever hands on with Muse S devices, he will have to deal with plot digitizing to extract data :) One final thing, I doubt if Muse S and Dreem headband can be worn at the same for a direct comparison. They will have overlapping sensor contact areas I guess, But still indirect comparisons are possible. Fitbit compares well with Dreem for example,
@@yildiraykomurcu WoW, I did not know this much about Muse. Hopefully, Rob can test it to some degree...I would too but I don't have a verified EEG device to use along with it and I do not have his expertise to know how to analyze the data! Unfortunately, Dreem 2 is no longer sold to general public which is a shame to say the least. But your note here was useful! Thanks
what a good review!! The sleep stage tracking was the best and most useful to me. I wish you could do the same with the top polar products as well! I'm using mine mainly to track how I sleep.
love your content. I mainly watch for your advice regarding sleep tracking. I am most interested in achieving optimal total sleep time - as I assume over time that getting adequate sleep will translate well to all the sleep stages. Could you provide analysis regarding how well the devices you assess accurately track total sleep vs awake time? I like the sleep stages but I really think that if you get the sleep time right everything else will fall into place.
You are wrong in looking at Total sleep. Deep sleep phase is the one that your body regenerates in. If you want to optimize your sleep, optimize for the deep sleep. For this goal only 2 devices are accurate enough: dreem 2 and withings sleep analyzer. All smarwatches at this moment are garbage if it comes to measuring deep sleep.
If possible, I think it would be great if you could get a friend to help you test these watches while running, as that's a pretty big thing missing from your reviews. :)
Great review .. dont u have a list where all watches reviewed listed by sleep/fitness/gps .. and 1 thing i am missing for the watches that you dont compare is the body composition.. still looking for the best overall. Thanx for the recomendations at the end.. only the whoop sounds great but theres no option without a phone and a membership is needed :( thanx and keep up the great work🙏
Are you planning on reviewing the watches with blood pressure features like the Omron or any of the other cheaper alternatives? I'm really interested in buying one for my older parents/grandparents, but it's so hard to trust if any really works 😔.
@@bulliwoody3674 I tried the Omron, but it wasn't accurate. I didn't even care about trying any cheaper alternatives if the most expensive one is inaccurate.
Good day Rob. Love all the content you are creating 🙏 Helps alot. Have you by any chance done any work on the garmin vivoactive 4? Would love to get a watch that is good on heart rate and sleep paterns,but unfortunately we can only choose one as the other is always lacking😔. Keep on man🥳
I think it would help if the tester also kept a (sleep) diary so that he could include additional metadata relating to things like stress levels, coffee intake during the day, alcohol intake during the day, diet, pre-bed screen time, time to bed etc. I realise that is potentially an invasion of privacy, but in order to make a valid comparison between devices, the testing circumstances need to be exactly the same especially if n=1
I really appreciate your videos, however I find it hard to find any good overview. Can you please do an overview over the overall best smartwatches to buy this year, all factors taken into account?
Hello, great content, watched alot of it. Wonder if you have plans on reviewing a fairly budget Oppo Watch Free which is advertised heavily as a good sleep tracking watch?
Previously you would mention how much deep/light/rem/awake time watch X detects and how much your sleep device. From what I remember previous Polar watch that you previously reviewed was the best one on this. When you put those results on a matrix to see how much of what was predicted as deep/ rem, etc. is actually deep/rem, things don't look that good, but as not a scientist, I don't care about that - if it predicts something more accurate than other watches (I don't know how) - that's good enough for me :D
Dang, I wish I would have waited for your review before buying. I probably would have opted for the Garmin Vivoactive, instead. As an anecdote, my sleep results match up with yours. I am getting a lot of random short "wake" time in my results. Regarding satellites, do you think the GPS+Galileo setting would have worked better since you are in Europe? Also, it seems like your Chapter timestamps are off. Chapter 3 should start at 10:31, not 9:40, and Chapter 4 should start at 16:20.
I'm very interested in the hrm cadence lock problem. I own a PPP watch and I have problems with cadence lock. In my case it happen mostly during walking sessions. It would be very interesting for a video based on your scientific approach regarding the cadence lock phenomena.
Hey Rob, would you ever do a video describing your academic journey/what you do? I'm pursuing a similar path as you (biological data), and would love to hear your insight.
thanks so much! always love your videos. Do you think it would be accurate for tracking HR during indoor rowing? i am trying to find an accurate option that allows me to disconnect completely for sleeping. The Apple watch does not seem to do that no matter what I do to the settings. I really appreciate you getting this out so quickly after release.
I am confused about your summary diagram wrt sleep ranking. According to the summary diagram, Fitbit Inspire 2 (worst sleeping score) is placed lower than Charge 5, but looking at the confusion matrixes the worst score of Fitbit inspire 2 = 64.4% REM is much higher than the worst score of Charge 5 = 54.6% REM.
Thanks! But I have to wear the watch extremely tight to get an accurate HR reading. Can’t wear in my sleep because of that. Do you wear it very tight? Thanks!
Two comments on accuracy of your results. You are comparing the watches with dreem2 device which is known to have accuracy between 0.55-0.82 (see 2 papers using DOI 10.1093/sleep/zsaa097 and 10.1016/j.sleh.2023.11.012). Ok understood this is the baseline. But looking at 13:53 of your video, isn't it strange that between 00:00 and 01:00 you had 3 deep sleep entries of couple of minutes each - I understand that this is not common, and might indicate sleep disorder problems with test subject, or error in measuring of dreem2 device?? Rem entries of short duration at d2 device around 02:00 and 04:00 are also strange and could also indicate an error in measuring. The second comment relates to duration of sampling. Is is possible that polar device learns our patterns and after some time could get more precise? I understand that they require a subject to measure heart rate for 28 days before they unlock some parts of analysis. I am not sure whether measuring of two nights my give representative performance of the device which adapts to a subject and learns patterns.
Hi, I have been waiting for this material about this watch because I am planning to buy it. Unfortunately, the results are not encouraging. Do you know from your experience or maybe there is a chance for some update of the watch by the manufacturer repairing the measurements? This is a new device and I hope the manufacturer will be able to fix it again.
unless your hands are 1:1 same as Rob's, his review is useless for you. watch a review by fit gear hunter and you might change your mind. then watch one more by somebody else and you might change your mind once again :D
Thank you for your helpful review. Would you consider the Vantage M' s OHR (and identical M2 OHR unit) more accurate than the Pacer Pro? And how would those two rank compared to the OHR of the Polar Vantage V2 and Grit X Pro? I like the Polar watches and would like to know which of the recent OHR watches is most accurate?
@@nomoshiti I only do running and an accurate wrist OHR watch is sufficient (don't want to wear chest strap) for me. Most Garmins are accurate enough. I also like Polar and unlike Garmin which gives their latest OHR sensor across their line as new models come out, Polar currently has several different sensors among their premium watches. If only for curiousity would like to see a "quantified" comparison.
Hi, I have been using a vantage M2 for a year. The heart rate monitoring is way off vs. the Verity Sense especially during light motion like walking and sometimes even at rest... Makes me wonder if the problem is with an overzealous motion algorithm cancelling out legit heart rate readings. Extremely disappointed given the reputation of Polar.
@@psionic6433 Yes, it often spikes without justification when running but most of the time it's seems good. That's why I bought a Pacer Pro which hasn't done that yet. Just got a Grit X Pro (at a good price) which seems the most accurate of the three.
@@GENE2A my M2 does the opposite, it will display 20-40 bpm lower than other devices like the verity sense or fingertip pulse meter whilst in light motion. the only time the M2 matches the bpm of those devices is when I stood still for at least 30secs. once I start moving again it will drop 20-40bpm which is why i suspect it has something to do with the motion sensor/algorithm. i really like the smaller and slimmer pacer pro design but based on my experience with the M2 and Rob's review i dare not buy it.
Hi Rob, have you updated or retested this Polar pacer PRO ? I ask because it seems very strange to me that it gives you such low accuracy in heart rate, I have seen other videos that claim that it is more accurate than the Garmin Elevator 4.0
Thanks as always! Great stuff as always! It's almost boring to comment your vids, there's never anything wrong to highlight... And nice subtle inclusion of some DCR, always really much in the right place when talking sports gears. Though... i am not sure if Ray was using his Garmin Tactix 7 ballistics functions the wrong way while aiming at your hand, or your hand/eye coordination is orders of magnitude below your scientific acumen... Either ways... Hilarious small inclusion, thanks for that as well :)
So what I know now is that you have the ability to go through buildings and also dip your heartrate to near zero at random moments. Interesting. I think the governments of the world will have something to say about that.
There is no exact device for those metrics. Use a heart rate strap for every watch and decide by the sport functions and metrics you like the most. Simple as that. it’s all about heart rate training and analysis.
Don't think this is quite as scientific as you claim. There is a sample size of one. There is no independent control of factors such as the tightness of the watch on your wrist, its positioning and the amount you are sweating, especially as compared to these factors when you test the other watches. You can tell us you think it's similar, but that's a subjective experience, not an objective scientific measurement. The graphs are not as accurate as they appear if there is no control of such fundamentals, and a sample size of only one. I'm not saying your tests mean nothing - just that they're no more reliable or accurate than those of other reviewers. Your outdoor cycling heart rate accuracy test in particular would be heavily dependent on the tightness of the watchband and its positioning, and a greater sample size would probably reveal significant variability among people with varying wrist and wrist muscle sizes, as well as different skin colours (an area where other reviewers have identified as a comparative advantage for the Pacer Pro, but which your data doesn't capture). You're making a worthwhile contribution, but you're exaggerating the relevance and reliability of this data for people different to yourself quite a bit, it seems.
Love your content. Do you have any way to quantify or evaluate the training load features of this polar or say a high end garmin. I’m mostly interested in making sure I get enough workouts in and don’t overtrain. I am trying to decide between say a garmin 955 vs something like this. I have a Apple Watch ultra which I love but it doesn’t tell me anything meaningful about all of my workouts even though it does a good job of tracking them.
Please never ever buy polar watches. I bought vantage and pacer pro. It is a nightmare to sync the watch with my phone. Random glitches . Got my watch replaced twice. Resets randomly where you lose all your data that is not synced yet. Horrible.
