How We’re Fooled By Statistics 

One of your better videos It is unfortunate that your next likely will not be as good.

The book "the talent code" talks about the appropriate style of feedback according to skill level. A beginner needs more encouragement to improve, regardless of immediate outcomes, but relative expert is engaged and needs no encouragement, but they still require detailed feedback in order to correct their mistakes. Reversing those, I could see beginners giving up out of frustration, and experts rolling their eyes, feeling like their time is being wasted.

Can we all take a moment to appreciate how good Derek is at drawing bell curves?

Selecting both answers on the question "This statement is a lie" and receiving 1/2 points is next level IQ

So, theoretically, we could watch or not watch this video, like it or dislike it, share it or not share it, and the next Veritasium video will either be better or worse? :)

I appreciate that you had to film yourself 49 times for those classroom shots!

I read about this exact example in Daniel Kahneman's book "Thinking, fast and slow". It is an amazing example of a fallacy we're primed to believe since a young age, which actually proves detrimental to both the performance and mental well-being of a person always being reprimanded. It is just one of the many quirks of our mind that Daniel explores in the book, a big recommendation.

This is ALOT of effort to explain our last Golf outing, dude...

I wish my teacher would say "Great work!" whenever I got a 61/100.

<a href="#" class="seekto" data-time="133">2:13</a> Question #1 was really, really, really, really unfair.

“The negative feedback was a harsh reprimand and the positive feedback was a 👍🏻” Seems uneven but ok

I feel like regression of the mean is something I’ve applied so much in my life but never actually knew about.

I wish my dad was smart enough to know this when I was growing up.

<a href="#" class="seekto" data-time="142">2:22</a> Name: Jimmy Neutron Student # : 1.675 x 10^(-27)

I started a new job and my confidence in my skills isnt that high yet, but the guy teaching me believs in me. He sends me off doing stuff as if its nothing, even though it makes me nervous. That makes me proud, and I want to prove myself. So I try very very hard until I get it right. If he wouldnt believe in me, I'd be discouraged.

A great video. I kept wishing that you would clarify that it isn't that *the next* event is likely to be better (after doing unusually poorly) or worse (after doing unusually well). It is that *any* event is likely to be average (by definition). There is nothing special about the next event. Thanks for a thoughtful video!

That distribution at <a href="#" class="seekto" data-time="138">2:18</a> was drawn brilliantly

I have to say constructional feedback is the best I hate when teachers give me an a- without explaining why it's not an a+ just because they say there is room of improvement or it's a way so I would keep try harder. It's better to actually know what I did wrong and how it can improve.

I'm surprised he didn't bring up the fact that the test was poorly executed anyway. What they should have done was to give negative feedback to both groups and then, to other test groups, give positive feedback to both groups. That would help reduce "regression to the mean" as a factor.

Like your channel. As a manager and scientist, the most important idea is to maintain communication with direct reports. Staying out of blame and asking questions help staff focus on the important things like values and group culture.these studies are important yet they isolate behaviors without context to the whole. Thus the point is well made. And what is also important includes perception of and individual performance to the group. And if done correctly, group influence actually means more than just my impact.

This was such a serious lecture. Congrats for the work done

<a href="#" class="seekto" data-time="140">2:20</a> That's a nicely drawn curve, Derek. Well done.

It's interesting, I actually read Kahneman's book, but I still find your way of presenting this idea very fresh. You are a really good educator.

When watching your videos I usually like them, then start to watch in full screen. Then for 3 or 4 times during the video I will go out of full screen wondering if I already liked the video. I wish I could like it multiple times. Excelent video.

Man these veritasium videos from the ancient times sure are starting to surface again on my feed.

This is why I love science. I clicked on this video thinking... why is a physicist talking about psychology? Why? Because science has to take into account all sides, this is one many people probably overlook.

I love this video, it's amazing. Though i understood every word of it. I am replaying it over once again.

This book completely changed my outlook on life. It’s pretty crazy.

This concept is very underrated and is really hard to integrate from a raw thought into an explainable concept

"This statement is a lie" True/False LMAO dude you got me 😄

Thank you for explaining! This has helped a lot.

Wow! you've been at this for this (and more) longer!? no wonder you got viral, your content is clean.

I'm torn between my love for Veritasium and my love for Vsauce.

Most of my thought pattern on this video went from "Hey, this is like what Jeffrey Lin is doing for Riot and League of Legends!" to... "Woo, where did he find chalkboards?"

Having spent decades training people in both technical fields and martial arts, my experience is that it depends on the type of person you are. Most people are encouraged by positive feedback, some are encouraged even more by negative feedback while others are discouraged.

This is one of my favorite videos of all time!

Coming back to clarify some concepts, still an excellent explanation full of relatable examples. Thanks!

It’s important to qualify that the concept of regressing to the mean is only as powerful as the inherent variability in case-by-case performance. Human behavior is not entirely dictated by random chance so this concept doesn’t necessarily speak to the effectiveness on negative or positive feedback in relatively controllable circumstances (such as encouraging or discouraging behaviors brought about by conscious choice)

I was just thinking about this today. Positive feedback reactions may be inspiring for learning, but i think trusting someone to do a task will really boost their confidence and ability to perform.

I read about this like two weeks before seeing this video in Thinking Fast and Slow by Daniel Kahneman, good stuff.

I was told, by many psychology professors, that punishment is very effective at promoting or deterring behavior, it's actually too effective. However, negative reinforcement and positive reinforcement are usually better at keeping the desired behavior more habitual. When using a punishment paradigm you risk the chance the undesired behavior comes back when the punishment is no longer present. It's true to positive and negative reinforcement, I believe they call it extinction, but it isn't as detrimental as punishment. Just to clarify, negative reinforcement is not punishment. It is escaping a stimulus that is aversive, which ends up becoming a behavior. Ex. Some people put their seat belt on because they don't want to hear the sound of the seat bell alarm. They are trying to escape that stimulus but aren't being punished by it. Punishment is a stimulus given after a behavior that's meant to be aversive but isn't present before the the behavior.

An important video for people currently in university or otherwise writing scientific papers. You'll understand why when you watch it. Also when it comes to positive vs negative feedback to students and players, keeping someone's attitude and outlook on life feels more important than rubbing in their performance on previous tasks(both positively and negatively).

Veritasium your videos are ridiculously good to watch. It’s crazy to think this was published 9 years ago, when life was simple.

I love this episode- Awesome. Accurate. thank you for making this!)

THE DIFFERENCE IS GROUP VS SINGLE CADET.... a group is able to converse and feel an emotion towards something in unison, thus strengthening said emotion. A single cadet is alone and unable to feel with his comrades, making it an all around internal battle, rather than external,(as a team would do). It's really easy to think about if you just put your self into those situations.

"Not Quite my Tempo!"

Thinking Fast and Slow is a great book and the Israeli pilot training is a memorable analogy. I wonder what the studies say about the type of positive feedback and the longer term affects. If you give students a participation award and praise them for average or even below average work then maybe they do find encouragement in that the first couple of times but they might soon adjust and realise that the praise is empty.

connection! Regression to the mean begs redemption day by day. Thank you for your work, the message at the end was great too

What I learned from this video: Derek has the ability to take something which I found really dull (statistics) and turn it into the most interesting thing I've seen today :-)

One thing that could also influence the effectiveness of of positive/negative feedback is the personality of the people involved. For example, fighter pilots tend to be super alpha males with big chips on their shoulders. Positive reinforcement may not have much effect on them as they already view themselves in a very positive light. Getting them to change via smacking them down a peg or two may in fact be what works better. On the other hand, using boot camp techniques on a bunch of grandmas in a yoga class is probably not going to have a very good result.

I love the thumbnail because the time duration partially covers the text such that it reads "Regression To The ME"!!!

I've noticed there are so many videos on this channel about the book, Thinking Fast and Slow. I'm reading it right now and these are super cool to watch and recap!

This video literally just made my day. I've been struggling with depression a lot lately and it just makes me feel like things will always average out to be better. Like I'm getting all the bad days out of the way now, so I can live a happier life later, ya know?

This is a fantastic video...I know I'm late to the part here, but keep up the good work!

Thank you, i appreciate your videos. You have a great job!

I feel as though we as humans love to have shortcuts and clear answers that are either black or white, but in real life, finding a balance in the middle is key in almost everything. I feel like the positive/negative feedback debate shouldn’t be about which one is better or which one helps more. I think it should be about which one you should choose in each different situation that requires feedback. For example, if you have two pilots training for the same exercise and one is doing great while the other isn’t, you shouldn’t use exclusively positive feedback or vice versa. You should adjust your method of giving feedback based on the performance of the person. Because even though in this exemple, both people are doing the same exercice, they do not have the same needs in their understanding of the exercice for them to be better at it. I feel like for example, the one who’s doing good would benefit more from (not negative but) constructive criticism, so that the person can still find ground to evolve from and not get stuck in a lazy place because they were great from the start. Giving exclusively positive feedback to someone who’s doing good can easily make them fall in a confort zone because the mind stops working to find ways to surpass itself (and with time it can regress) and same goes for the person who’s struggling and how negative feedback can affect their ability to trust themselves and concentrate on doing better. Sometimes when you are too stuck in a negative cloud, you can’t see anything positive anymore and you can get lost in a desperate state where you think you are inherently bad at something because of that initial experience and response that you had. Finally, i think the danger lays in thinking that there is ONE better way, its either positive or negative but no, we should always adjust. (I havent watched the video yet so I might change my mind after this if I discover something i didn’t think about :3 )

What if it has to do with how much respect people have for the instructor? I know that for teachers I dislike and don't respect, negative feedback makes me contentious and positive feedback makes me annoyed, but with teachers I do like, negative feedback makes me want to try harder and positive feedback makes me lax. In the case of the fighter pilots, the pilots are trained to respect and not fail the instructors, so maybe they'll try a lot better to not disappoint and to not be reprimanded. Meanwhile, many students loathe their teachers, so they wouldn't care. However, for the sports team, it could be a case where the players don't want to let their coach down, and those who are castigated actually feel letdown and they perform better because they are depressed a bit. Just a thought.

Excellent video, man. You nailed it. Positive reinforcement with dogs has proven to me a lot better at teaching them new tricks, while negative reinforcement has proven more effective at interrupting unwanted behaviour. So for me it would be "if you are trying to teach how to do something, use positive reinforcement; if you are trying to teach not to do something, use negative reinforcement" .. Of course you could say "we are teaching pilots not to make mistakes".. Still I think my idea has a little something to it..

My cousin and I used to play a lot of backgammon, which has elements of luck and skill. We were evenly matched, having pretty much taught each other the strategy that we used. We started keeping track of our scores, including the use of the doubling cube. The results swung rather widely, with one of us being well in the lead one week, and the other being well in the lead another day. But neither of us ever got a consistent lead.

JESUS I LOVE YOUR CHANNEL YOU ARE SO DAMN GOOD I GET IMPRESSED EVERY TIME OMG

take a shot everytime he says feedback

Brilliant video. I felt like just liking wasn't enough so I'm commenting that I like it as well.

Inspirational and instructive video. Thanks.

Brilliantly presented.

Its simple. When you get good feedback, you think you do really good at it, and because of that, think you don't have to try as hard. But it can help you have some confidence in yourself. Negative feedback will make you try to get better to make sure you do it better the next time, but to much negative feedback can make you give up. The best is a mix between the two.

I believe in consistent honest feedback. Consistency is actually the key to whole thing. Also, the notion of one disciplinary technique is the best fit for all people is flawed at the most basic level.

I think something that's different but related would be the fact that we tend to naturally give more feedback in respond to a negative performance (from traditional skills to social ones) than a positive one because we naturally intuit that less feedback is needed when things are going well than when they're going poorly

This very interesting video could be even more interesting if it addressed how the mean moves over time. Since nothing exists in a vacuum, our attempts to measure and influence do, eventually, move the mean one way or another. The way this video was presented, and I'm sure that wasn't the intent, kind of made it sound like the mean eventually overpowers any attempt to derive meaningful information from cause and effect studies. One place where this becomes extremely obvious is in the movement of price for currencies on the foreign exchange market. There, there is a very strong tendency for price to revert to the mean and revisit previous price levels, but the mean is constantly moving, since price is constantly moving. It becomes much more challenging to figure when things are at an extreme, and when they are actually not because the mean has slowly shifted due to the repetition of certain events (price points) being over represented. Derek you are an excellent communicator and you make amazing videos. Even though this video is 8 years old, I would love it if you made one addressing the points I mentioned above, namely the movement of the mean and moving averages over time, as sample sizes increase and feedback loops are introduced.

So from watching your videos, I gathered that the most efficient way to success or improvement is to receive rewards but keep stepping out of your comfort zone 😊😊

as a highschool student i can tell you now that we tend to do better if promised a reward

This is fantastic! I have done many statistical studies, and you have challenged my thinking! Shawn R. Hughes, Ph.D.

Well explained. I had expected to be fascinated but partially mystified, so this was a pleasant surprise.

I believe that the only thing learned through reward is how to get a reward. I suppose it's all about what we are learning as well. When we learn not to do something, it is usually through negative re-enforcement and the opposite through positive re-enforcement. However there are so many variables that it becomes hazy to know when negative or positive re-enforcement should be used for improvement. It is also expected that by using both, eventually an equilibrium is created and each persons equilibrium can be different and can change from time to time, that equilibrium being the point where one is content with oneself at the moment where one doesn't feel the need or expect a reward for learning something, and is also aware of possible consequences which is reward in itself, because one then avoids that which may cause harm, be it mental, physical, or psychological. I realise as well, that those who always seek reward tend to be self-centered, and greedy and often are not aware of or care for consequences resulting in their need for reward. These people I believe to have great difficulty with the negative aspect of learning and have emotional imbalances. When I became aware of this subject, it was hard not to notice it, see it in action and the results created, in my workplace and with the daily interaction of people. This also allowed me to know each persons state of equilibrium and the amount of balance or imbalance, and how it affects the person which in turn seems to effect their personality traits and reveals the level of aptitude they may have.

This video was fascinating, but I feel like it wasn't really about punishment vs. reward so much as positive feedback vs. negative feedback. I guess I'm just curious to see how (for example) kids might react to discipline or candy.

As a trainer I found this very useful man, thanks a lot

I love you bro!!!!!! Just keep making interesting stuff!

"three kinds of lies. Lies. Damned lies. And statistics." -Mark Twain.

Both positive and negative feedback can be subject to the person who is receiving based on environment, history, lesson, application, outcome & desired results. In other words, both forms can be beneficial as long as the homework is done on the above criteria prior to lesson plan.

Nice explanation!! Thank u so much

I never knew I needed this. Thank you

I can personally relate to this statistical phenomenon. After not having done so for months or even years, I've gone out to shoot a couple rounds of Trap (its shotgun shooting sport) or had gone rolled a couple games of bowling. In every case, my first round of Trap, I would shoot 23-24 trap out of the 25 targets and likewise my first game of bowling, I'd score of 195 or above, however, any following attempts at either sport, always yielded a less impressive score....15 for Trap and 123 for bowling. After years of seeing this phenomenon, (I'm 60 yrs old) I discovered that there were two common denominators in each sport. Firstly, I had developed the biomechanical skills from years of practice. Secondly, I had experienced a sense of greatness with the results of the first attempt relative to each sport. As I became more intent in subsequent attempts, my desire to improve the results had shifted me away from my naturally developed skills. I realized that by not being trained as an professional Athlete, I did not have the concentration skills necessary to maintain the results and would consistently do much better by utilizing my naturally developed skills, rather then forcing myself to perform.

The difference between Regression to the Mean and the Gambler's Fallacy is sample size. In GF, it's only one random event, so the probability isn't changed between the past and present. But in RM, it's a collection of random events. So while one event isn't dependent on it's past, it IS dependent on the average performance of the present, because the probability will be that it is in the middle, regardless of where it was before.

Good video, I learned something. In real life, like the fighter pilots or team players, it seems that constructive criticism is key to improve skills. Highlighting good behavior can be a buffer for the constructive criticism "You did this very well, but you need improvement in . . . "

<a href="#" class="seekto" data-time="142">2:22</a> I love that Schrödinger's answer to the question #1.

when i had my last math course in school, i failed every test before the exam, if i failed the exam i would fail the course. but being me i was not affected by my previous preformance and stayed calm and positive during the exam. in the end i got better results than my friends who studied (i didnt study at all which explains how i failed those tests). in conclusion, preformance depends on how well you handle each situation, or atleast in my case.

It would have been so much better if everyone understood this. I've been thinking about this for a while and tried to explain it my acquaintances, but you can't change that in a day. Thank you for making the first steps to bringing that understanding to the majorities. A lot of people think that a 50% change of getting head or tails means that if you flip a coin 100 times it would result in 50 heads and 50 tails, but in practice that is not exactly true. There is even a small change that all of 100 flips it will result in only one of conditions. As they say - there is a big lie, a small lie and statistics. And true and at the same tame false this statement is.

Goes without saying that this video, like all his other videos, is so loveable! But I must say this: Holy Cow! that was the most beautiful free-hand drawing of the bell curve I have seen in my life! Just one continuous swipe across the board and viola!

I have experienced this a lot with my video games, if someone praises me i feel like i can do anything and it's now expected of me, but when i do wrong and someone calls me out i now feel the need to protect my pride by proving them wrong and doing what they thought i couldn't.

"How we're fooled by statistics"? show me some stats on how this is

both positive and negative feedback work. it's not a matter of which is better, they BOTH should be used for maximum effect. punishing failure in addition to rewarding success provides a better gain than simply rewarding alone

Awesome content. Thanks veritassium!

Whenever i become aware of faulty ways of thinking i always do my best to avoid them in the future... but this... this is gonna take a while to wrap my head around...

For me, negative feedback can really demoralize me over time. For the first or second times, it can make me feel like I'm proving them wrong and I do better, but after too long I just shut down and say "I don't need to please this person" and end up not wanting to do anything they say. On the other hand, sometimes with positive feedback I know I'll have to up my game to impress again, so I'll try to do so. Of course eventually that becomes difficult and I level off again to reestablish myself before trying again. Both forms of feedback are beneficial for me in small amounts. Too much negative and I'm less likely to complete the task in the first place, too much positive and I know I can't always do better.

Here's a thought from the webcomic prequel: You find a four-leaf clover, this increases your chance of finding another four leaf clover, which increases the chances of you finding another, and another, and eventually you build up so much luck that the only clovers you can find are four leaf clovers.

The exam questions made my day. So funny. The first question was a banger I really am very happy for all the references.

I'm currently reading Thinking, Fast And Slow by Daniel Kahneman. I learnt about regression to the mean from that book, it's seems Veritasium did the same.

As a general rule I mix and match positive and negative feedback for my students about their progression and not their level, for me it's always putting them in the mean of an unknow scale and judging how they progress, I also promote self-feedback so my students are more able think of their progression and to push themselve to the limit they define.

just because many of the data we collect fits a normal distribution, doesn't mean that the normal distribution is some kind of natural law that governs performances of humans. complex actions cannot be easily quantified in that respect- it's very hard to evaluate objectively which performances are good and which are bad. also: it doesn't violate any physical law, if a novelist is getting "better" with each book, or an athlete with each competition et cetera.

I think there is many factors involve in learning. But in the case of positive and negative feedback, the biggest difference between positive and negative feedback, is that positive can increase your motive to learn what your learning. Because positive feedback respond to the human's need to feel good, feel love, get attention and believe in our potential, even if all of this is intense, it is also more pleasant to be suround with positiveness than negativeness. So in general, in your group, the mean should increase if you use positive feedback because you augment the motive and the feel good feeling of the student. A student motivated or that feel good gonna put more effort, more focus and maybe gonna practice on is own. Feedback is feedback so if you have relevant feedback, you still gonna be listen by those who are motivated. But positive is in the most case the best. But why then the jet student has different results. I like the explanation of the video, but still, I believe there is a causal explanation. I tougth first, that the motivation of a jet class, since you have a hard process to be there, would not be a random sampling of the population motivation that you would have most of the time in other studies. But then a remember that in my high level badminton class I would see the impact of a coach with positive vs negative feedback. So that is not it because a pleasant environment is still really important. However there is an extern variable in the case of the jet learning. Driving a jet is dangerous, we all know that if you do a bad move, you can die by screwing up. So I think that a negative feedback when you do those class make you remember the relevance of the information that's gonna come if you do not wanna die. So in that case, remembering you could die, gonna put your focus on the highest you have. The think is a negative feedback in a situation so relevant like that doesn't event feel unpleasant and it increase focus. The idea actually come from my lesson from lead in climbing. I had that focus that I do not have as high as when I knew the feedback couldn't save my life. It would be interesting to test another jet class or another class of something dangerous to see if we observed the same behavior of if the statistique gonna win again ;) It is not base on any paper or something, and I'm open to change my tought :)

I really wanna say you do a fantastic job at making your videos timeless. I can never tell when they were made until I look at the date.