The bizarre flashing lights on a smartwatch 

Check out Andrew's video here: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-fDj2kYIF-sE.html and his python code is here: github.com/ajsteele/faceHR The sponsor is KiwiCo: Get your first month free here kiwico.com/stevemould

The labels on the graph killed me, as someone whose studying to be an engineer I’m definitely gonna steal those for my charts

Thanks for having me! Honestly the most fun I’ve had strapping electrodes to someone and filming it.

If you've ever listened to a heart for a long period of time, it's kinda scary how irregular they are, but that's how they work

In case you are interested, there are other examples of pulse taking using cameras in the world of video games. The Kinect 2.0 was purported to be sensitive enough to read the color variations of the skin to estimate pulse readings, very much like what was shown here. But the Xbox fitness app that used it was discontinued before I'd tested it. Ring Fit Adventure on the Nintendo Switch uses the simple IR camera built into one of the controllers to read pulse rate at the fingertips.

2:31 “And obviously if the premature baby has a beard like this, you would shave it” I wheezed at this

You can tell Andrew is a data guy from the way he talks about a simple graph of numbers.

Six or seven years ago I was attending engineering master thesis dissertations in the Trento University and one of the candidates proposed a similar method to monitor - using a very low resolutions webcam - the heartbeats of bus drivers, as an extra safety measure for public transportation. I was very surprised because it worked!

Superb video. This heart stuff sounds fascinating. But don't think I'm not silently judging you both for not even attempting to say "photoplethysmography" once during the video.

A graph of "shrugging emoticon" over "time I guess" is so perfect. Thanks for labeling your axes, and using the correct units Steve. That made me very happy. It may take some time, but someday in the not-too far off future, I imagine every room will have some form of sensors built-in. It's going to take some clever data scientists, like your guest here, to turn the resulting mess of data into useful information. But that information could be used to do things like call the paramedics if someone's heart rhythm goes arrhythmic. That's pretty cool on its own, but it's also one step closer to a Roddenberry-esque post-scarcity eutopia, and I'm all for it. Well, at least perhaps until the advertisers get ahold of the sensor data.

Next question: How does a smart watch measure blood pressure? I think it also uses light, and no doubt there are a ton of scientific papers out there mentioning things like photoplethysmography but no layman-level explanation which you are brilliant at delivering 😊

I thought it was interesting, I had a phone app *YEARS* ago that would detect your heart rate based on video - long before the Apple Watch or Fitbit existed. Comparing to an actual real medical-grade heart rate monitor, it was extremely accurate. I have to imagine it used similar processing to this.

One correction for you Steve, for pulse Ox, the sensing still requires the ambient light measurement. The ambient component needs to be removed from the red and IR because it can induce a signal into the red and IR sensors when the environment is bright enough. It's also used as a way to offset the red and IR signals when near saturation and used to detect dark room environments vs bright room environments. Some sensors automatically remove the ambient component and don't report it, but it is still needed for proper measurement.

Ooooh. I've worked with some biosignal stuff before, and have been curious if this was possible too. Also fun fact, you can get the respiration directly from a PPG signal (instead of the heart rate variability) using a bandpass filter that removes the higher frequencies from the heartbeat. It works surprisingly well, though the sensor has to stay pretty still for it to work.

Super cool video! One note if people are curious, the change in heart rate with breathing is known as respirophasic sinus arrhythmia. It is regulated by baroreceptors, so when you inhale, the negative pressure decreases the signaling from these receptors to the brain, which is similar to a low arterial pressure signal, which activates the sympathetic response and speeds up your heart rate to compensate. When you exhale, your thoracic pressure increases, which increases the baroreceptor signaling and increases the parasympathetic response to decrease your heart rate.

This was really interesting, and also explained to me why the pulse ox meter is not accurate in cases of CO poisoning. Red bloodcells with CO bound are still red, that's why people suffering from CO poisoning don't look like they are suffocating, they have perfectly healty looking skin tones.

This is one of those few videos you come across that you feel genuinely deserves a like button. Extremely well done, very informative, learned a lot of new things about biometrics and data analysis

Your video amplification presentation was fascinating but this is something else.

I have one of those athlete sport watches that can measure HR and sPO2 24/7 and I never knew exactly how they worked before. I had no idea that watching this video would be so informative. Thanks!

About the color strength, have you ever thought about bayermatrix? Since regular cameras are often tuned with one extra green pixel over red and blue(RGGB). In astrophotography we usually remove green noise due to this. And this is also a reason why we instead use mono cameras to make use of ALL pixel from chip.

I saw a very similar use of color change amplification to measure pulse about 10 years ago, the code was written by some grad students for a computer vision class.

I get so excited when I see you have a new video posted. The things you talk about are just so interesting and well explained. Thanks for all the work you put into this.

The banter at the end of this was really enjoyable

The amount of complexity, calculations, adaptation to variations and ingenuity on "crossed scientific branches" in a single device is ... striking, totally awesome and inspiring! I wish to know more, even the mathematics behind it! LOVELY

Hey this is exactly what my startup does! The technique is called rPPG, which is remote PPG, or photoplethysmography, which is what the Apple Watch is based on. Super stoked you've decided to showcase this in your video!

Thanks for letting us know a little of what technology we have on our wrists

When I hold my breath, I always figured my heartrate would go down because it has access to a lot of oxygen. but i couldnt figure out why it would go up instead. but having less space makes a lot of sense!

Most fascinating video I have seen in some time. As someone who has worked in a refinery if loved the scene of the piping and pump. Most likely their is an application for analysis equipment failure. Great job Andrew on the Python Code.

Hearing that the heart speeds up during a breath taken in explains so much! For years whenever I'd listen to my dog's heart (either dog) I noticed that every time he/she took a breath, there would be about 3 heartbeats quickly in the same space of time they had a single heartbeat when not taking a breath. Cool info!

Human ingenuity is so fascinating. Another great video, steve!

Lol, the outtakes at the end was brilliant. The banter between you pair was hilarious 😂

One of the best video I watched on YT !!! Thanks Steve & Andrew ❤

I love how you meet up with other smart people to do all these interesting stuff i could never even understand

I had a Fitbit I wore for a few years that was recalled (and they refunded 100% even though it was a few years old) that Fitbit claimed had battery issues where it could get hot and burn the user. Well mine never got hot, and I only wore it during the daytime when I was awake, never at night sleeping. The first year or so, no problems at all, but about 6 months before the recall I developed this lump directly on my arm directly underneath the fitbit that looked like a RF radiation burn. The reason I said that, was because back in the 1980's I worked on 2-way radio transmitters and occasionally would accidently bump up against an antenna that was transmitting and would get burns under my skin. It isn't a burn by surface heat but deep heating due to absorption of RF energy by the living tissues underneath. Since the surface of the skin is relatively dry it isn't harmed but the living tissue under the skin suffers damage causing a deep sore, which takes a while to heal. I had this same strange occurance so I stopped wearing the fitbit and after 3 or 4 months, the lump started oozing pus for a month or so, then finally healed up and went away. There were some articles that claim it was cancer, but I really don't feel it was that, but if my guess is correct, I'm guessing a glitch caused the one of those high energy diodes to get stuck on (which also could explain the battery overheating issue Fitbit mentioned) and radiating the tissue underneath causing a nasty radiation burn. Regardless, I realize nothing is perfect, crap happens, and I feel the benefit of smart watches still outweighs the potential risk of new technology possibly malfunctioning as I think mine may have done (I have no proof that is what truly happened but clearly something was weird was going on) and they will save way more lives as the technology is perfected.

It made me so happy to see R being used for cool novel ideas like this. Great video!

I was always wonder why my heart beats slightly faster when I breathe in.. thank you for this and all the other explanations! You both are great together!

7:04 I love the axes. 🤣

9:37 Thanks, this will haunt my worst nightmares now

The fact that your smartwatch is seeing all of this information and transmitting it back to its parent company is truly terrifying. Why do people willingly give this to tech giants?

Finally, thank you. I sort of had an idea how it works but you really clarified it

Fun fact: you can see your heart beats by color changes in your own visual field. Easiest if looking at a white background, takes some attention and the ability to keep your gaze steady for a while.

Great episode. I recently bought a smart watch that was having trouble reading some of my vitals. Now that I know the science behind the sensors I realize my $18 watch may not be the value I thought it was. Also, that KiwiCo stuff makes me want to be 8 years old again!

Them lights be going off when my watch is just laying in my bed by itself

Those may have been the funniest outtakes I've seen in a long time, thank you for including them!

I always assumed the reason your heart pumped harder when you breathe in is that it just got a bunch of oxygen and needs to deliver it. Turns out i was proved wrong. Thank you for the cool information, and awesome video!

I really loved the Axis labels here. One is definitely 'absorbance at a given frequency of light' the other is 'time I guess'. :D

Awesome video. Love having a smart watch and always wondered about the green vs red light and how it actually worked.

Thank you for the related video!!! I returned a smart watch because it was terrible at tracking heart rate but I'm now using a better one. This is still great info to learn as I'm within the return window.

1:08 Red lights and invisible (IR) lights are used in pulse oximeters to measure blood oxygen.

I think I saw something similar a couple of years ago, but it was more nefarious, like using that same detection of the change of colour of the face due to blood pulse for spying.

I have absorbed so much of the information you have distributed through your videos. Such a wide array of knowledge that I will surely pass on to my son.

The reasoning on depth of wavelength is true, this is why you would use IR for deeper tissue imaging. However there is a potential in the case of blood it is actually to do with its auto-fluorescent properties, which for red blood cells are far higher in around the 450-550nm wavelength than others in the visible spectrum, it depends on what the sensor is actually detecting.

I really appreciate how you break your videos down. You make science approachable, fun and cool. Thanks!

I already knew how the sensors worked but your video still made it fascinating. Very well illustrated and some great footage. Too bad we didn't get the IR view motion-amplified as well!

I often thought it must be motion amplification via phase difference of light wavelengths, but boy this was mind blowing. I really could use this information.

As a person with a complicated cardiovascular condition this had given me some great insight into how some devices work and how the medical system can monitor and maintain my current physiology. I have a condition called HLHS.

One of the problems I have with the wristwatch optical sensors vs the electrical chest sensors is that while I'm trail running, unless the watch is uncomfortably tight to my wrist, the amount of signal processing to extract the heartrate from the constant rhythmic jostling of the optics against my wrist is going to yield a pretty rough guess polluted with rapid movement (low frequency noise, coherent to a heartbeat's bandwidth), whereas the chest strap, is relying on a more consistent electrical capacitance undulation despite vibration. Wavelet conformity during periods of high activity is easier to pull from the electrical measurement whereas you can induce perceived arrhythmia within the data of a bouncing optical sensor.

I think the flash green light could also be energy saving, as you just need a few samples per second the deduce the hear rate.

I love your videos. You break things down so well. Any other person's video would be twice as long with unnecessary fluff

I JUST got a watch like that. So timely Mr Mould. Thank you.

I like how he went to infants in a hospital as a potential avenue for this kind of tech. I could also see this kind of tech (after some levels of advancement mind you) being used at security check points like airports/customs to see who's heart rate starts to get elevated as they get closer to the check point, or maybe just have random officers walking around to see someone who just got their luggage if their heart rate gets elevated although you have to be very careful about that.

Hi @Steve Mould, please can you put warnings up before flashing lights (e.g. 5:35) in future? Thanks!

High quality information 🥸 Thanks Steve!

11:47. This would be the happiest moment to a father that loves science and wants his kids to experience it's beauty. I want to feel this happy when I become a father.

7:09 I did have a good laugh. Well, they always tell you that YOU MUST have labels on your axis. Yours make perfect sense. I am going to borrow this. I have an electrical consumption graph (a non zero derivative) that is "meant" to be viewed over 24 hours with window sizes of 1 hour. So it's units are kwh/h ... but if you zoom in or out it starts reading in units of -\_(:-))_/-

Let's get this working on videos of politicians lying when asked certain questions. No seriously. Let's.

That was truly fascinating! Thank you!! (I'm digging the camera gear also!)

2:44 love the voyager plaque on the back of the laptop

8:36 Will the ups and downs of the infra red line be reversed as compared to the red line?

I wonder if this works just as well with people that have darker skin. There is already a known issue with pulseoxes that are less accurate if you have darker skin.

I learned a lot from this video about how things work and what they can be used for! That's so cool!

This is so interesting. Fantastic video! Thank you for sharing all this! 👍

I'm not sure your heart has less room to beat when inhaling, since you're lungs only follow the pressure changes in your chest, not drive them. Also not a doctor, but I've been curious about this before and asked them, the answer is usually 'I unno'

What people really wanted was a watch that could measure blood sugar. The GlucoWatch accomplished this by running a weak electric current through your skin, calibrating against a finger stick, and then using changes in the current to estimate changes to the glucose concentration of the interstitial fluid in real time (a proxy for blood glucose). Too bad it hurt and never really worked well.

Great video! thanks for this! Andrew's channel is awesome 👍👍

Steve is the most serious and goofy person I know 🤣 I'm glad that I discovered this channel from Smarter Everyday

Wouldn't the infrared go down when the red light goes up if they react oppositely to the amount of oxygen in blood (i.e. there would be an intrinsic "phase" difference in the curves at about 8:20)?

9:18 the smartwatch is actually seeing a *two* pixel image, where it's using the difference between those pixels for interference compensation.

Yet more quality content. Thank you, sir.

I'm interested to know how you sync'd up all the data? I guess with the mic and camera data you could do a clapperboard style sync, but the chest sensor? I'm imagining a very tense, "OK, 1,2,3, go!" putting in of batteries or something equally as fiddly 🤣

❗ 0:05 How can I get this software to work. I tried it but only got errors and gave up. Is there a good tutorial somewhere?

Great video, although there has been a lot recently about this not working so reliably for people with darker skin. It would be interested to learn more about how it can be just as reliable in such cases.

Your Videos are so Entertaining!! Im laughing so hard several times in every one of them but at the same time some interesting phenomenon is explained and I learn something. Glad I found this channel 🙏🏼😄

Okey so this is actually kind of funny. I tend to have anxiety attacks from time to time and I’ve learn yo recognize them quite easily, to the point where I calm myself by just realize it’s a panic attack (and I’m not in any actual danger), except for yesterday where I had my biggest anxiety attack I’ve had in my life, where I even started to hyperventilate pretty badly. After a while I managed to calm down and breath semi normally again, and I started thinking that I should buy a pulse oximeter to be able to show my brain that it’s just a anxiety attack. The point is, I started to wonder how they worked and how they could measure the oxygen in the blood, and just as I was wondering that, scrolling though RU-vid (to distract myself), this video popped up. I didn’t watching then because I was scared to even think about breathing and oxygen and all of that, but yeah, what a coincidence 😂

My physics teacher is yelling at you for the axis labels! XD nice!

This is an unexpectedly quick answer to a question I have had for a really short time and I'm amazed with how this all works

That's crazy! Because I've seen that on people's faces before, but I didn't know what I was seeing!! This is so cool to know. I'm going to start paying more attention to this.

Shared the video with a friend (You're welcome! 😉). He had a good comment. If you folks can do this, then people doing surveillance can do it too. I'm picturing camera's all over an airport just looking for people who's pulse rates seem a bit too high... Or any port of entry, while the Border Agent is asking "Purpose of your visit?" and the AI is staring at your face to see if you get especially nervous.

The main reason why the green LED flashes is to save battery, that significantly increases battery life of the smartwatch. That's also the reason why these LEDs will only flash when the smartwatch senses movement, otherwise it just assumes it's not being used and enters battery saving mode. They can get away with the flashing because our heart rate is super slow when compared to the speeds at which the embedded electronics are capable of running at, so they can sample a few times a second as opposed to constantly reading the heart rate sensor input.

IVE BEEN LOOKING FOR YOUR VIDEO ON MOTION AMPLIFICATION FOR UNI I FORGOT I SAW IT IN YOUR CHANNEL AND NOT IN AN ARTICLE THANK YOUUUUUUU ❤️❤️❤️❤️❤️

I was curious about this for a long time. You never disappoint me, thanks!

"Reflected back"? Why not just "reflected"? Doesn't that already imply a "back"?

Bio status: mined

This is incredible, im really curious on the programming on those smartwatches and how they work

"And obviously if the premature baby has a beard like this, you would shave it" hahaha

film everyone and findout who are aliens

It's good to remember that these optics-based detection methods are often calibrated based on lighter skin tones- for example, people of color can show oxygen saturation levels that are erroneously high. More work needs to be done to make medical equipment work for everyone!

I was literally just looking at my smart watch last night and asking myself what the light does. My thoughts are starting to sync up with Steve Mould videos!

This video was very illuminating. Thank you!

3:09 So red goes deep under the skin, and blue doesn't? But why than X-rays can go very through the tissue? red - goes deep blue - doesn't go deep ultraviolet - goes deep again?