QI | Are True Or False Questions More Likely To Be True Or False? 

Your teacher was the absolute personification of pure chaotic good.

@@fleshhuman4522 he mentioned he'd been giving this style of quiz for 8 years and only 20 or 30 students had ever attempted and less than half of those succeeded

I did one stupid quiz during a job interview where we had 40 minutes for the quiz, a maximum possible score of 20 for 100% right ... and a bonus point for every minute early we finished. I managed to restrain myself and spent a few minutes answering questions before collecting 35 marks for handing it in early. Apparently no-one had ever been quite so blatant before.

@@mozismobile please tell me you got the job

That's actually pretty genius - the statistical likelihood of getting every answer wrong by random guesswork is 1/1073741824. Only the kids that knew the material inside and out, and thus would have gotten 100% correct anyway, are likely to get the 90 points because by knowing the right answer to each question they'll know the incorrect one by default.

B or C

More specifically, C4.

@@oqibidipo Always Plan C

A for always B for best C for correct D for duh

In America we say to choose B LOL!

So... you're saying they're always false?!

@@Mythraen no, he is saying they are never true.

They always say that but it never works...

Your sentence about certainty improbability, I just wondered, (off topic alert): If only your basic complotist/conspirationist would ever begin to comprehend that. Maybe they'd have a chance to surface to reason. But no, they've ruled out the necessary doubt and moderation in their journey to insanity. Only remains the conviction with the inability to weight any counter argument or appreciate a different vantage point. They live in a world of true or false, with no gray areas.

Depends on the teacher. I had some teachers who would have things like "plant cells always have a cell wall" or "sound deductive arguments are always valid" as T/F questions. They were to test your understanding of the defining traits of a term.

Since English is not my native language, i have problems with terms like isosceles even if I know the concept. But when asked how many sides in any triangle are equal, and the alternatives are 2, 4, and 6, it gets easy.

I had a physics multiple choice question that asked for the correct density equation for a point inside a planet. A few questions later we have: From within a planets surface, it’s density is this, etc… Thank you for the free mark

These sort of questions were the reason I always read through all the questions before answering. Sometimes later questions give away the answer. And also when I was in year 7 the teacher gave us a quiz on April 1st telling us to make sure to read the whole thing before starting, on the last page it said to not answer any questions and to hand it in to the teacher without saying anything. I think only like 3 of us read the whole thing and handed it into the teacher, everyone else started answering straight away. It was very satisfying hearing them all sigh out of frustration as they reached the end.

I teach kids to look for answers in other questions. It's amazing how often it comes up. There was a lad in Spain that I taught who was sitting a very simple English test - I seem to remember he was Moroccan - that involved writing out the name of an animal that was pictured. It was a giraffe and the name was written on the next page in another question. I saw him surreptitiously roll the paper down to see the word and then copy it out. He wrote 'giraffe' perfectly, but upside down.

​@rachelcookie321 Very competitive people will not read through the paper. Good examiners will make you read through by asking for only 4 out of 6 answers, which makes you read all. I developed a solution to this was always to read through to work out the order of answering.

Having either Inna multiple choice question is a cop out

That's easy. "all of the above" would never be True. I mean from purely logical standpoint.

@@VK-sz4it Well, it could be true, as long as you're willing to accept that every single logical statement is therefore true by the principle of explosion.

@@VK-sz4it If all you have is a) none of the above b) all of the above then it is valid to pick b). But that is the only exception to your statement (unless you do some really funky logic where statements can exist in some superposition of true and false. I am sure there is some formalized logic that does precisely that.)

That's why I always use "None of the other answers are correct" instead of "None of the above". When the computer mixes up the order of the answers, I have no idea what answers are "above" the others. I also tend not to use "all of the above", instead listing everything out again in this one answer choice.

Fool me once...

Feels kinda bad but when you teach a class you'll learn why you give students some tools and tips on how to pass exams such as alternate after a known correct answer, simple fact is it reflects better on the instructor if all the students pass the exams than fail them... so it doesn't really matter to a instructor HOW they passed... just that they passed. Although to be fair, what I do in teaching IT support is try to make sure that people always look at acronyms and understand their full meanings, that way when they see multiple choice questions with a bunch of acronyms they might be assisted where the Acronym would answer the question such as What type of Server is used for remote users to authenticate to a network? RADIUS SFTP BGP CIDR Where RADIUS means Remote Authentication Dial-In User Service so if you knew the acronym it effectively answers the question for you, others being Secure File Transfer Protocol, Border Gateway Protocol and Classless InterDomain Routing none of which would be relevant to dialing in and authenticating.

When I taught I made a few different versions of the test that wouldn't have the same answers as the same letters in each version. Made it really obvious when someone was cheating.

Yes. However, most of the time the student won't know if the test was randomized, so the strategy presented in the episode will still increase your chances of getting a good result. If the test was randomized then that strategy will only be as good as picking randomly (but not worse).

That's especially true when only some of the questions contain that as an answer. If the teacher is smart and includes that as a potential answer for all the questions, then the likelihood of it being correct goes down by quite a bit.

Sounds like a driving license exam. "What should you look out for in this situation? The car ahead, the bicyclist you are about to overtake or the kid behind that hedge?" whith the correct answer being 4) all of them

Except when you get into the medical field where it is invariably to throw you off.

My HS biology midterm led with a bunch of directions including "do not answer the last question, you will get no points if you do. Just write a holiday greeting." The question was bizarrely complex grad school stuff to drive the point home.

I had a lesson where a teacher was trying to explain the importance of fully reading instructions before starting a task. At the end of the lesson he announced a pop quiz, sixty questions with a time restriction of 5 minutes to complete the test. Everybody set about the answers with gusto. As time progressed student after student groaned and stopped working. Reason: below question 59 it stated that if you had answered any of the questions 1 to 59 you scored zero points in the test. It was a lesson that has served me well over the years.

@@Tulkas219 That's an example of a really great life lesson, but not the one the teacher was aiming for; It's actually teaching the students that they shouldn't trust authority figures, or people in general, to know what they're doing - because putting instructions after the thing they influence is so blatantly stupid (except in this particular context, where the point is to punish the students) but there might be people out there that do it anyway.

@@jenerix5257 Like top posting in emails. The stuff that cam first is last.

@@jenerix5257 “Don’t blindly trust the competence and benevolence of authority figures” is actually a very good lesson.

Yes. And question 3 is likely to be opposite.

I hate those, because I had to justify the answer no matter what I chose. Justifying why a true answer is true, without repeating the question, is a pain (granted, I probably should have studied more too...)

I went to schools in the US in the 70s and 80s and graduated college in 1993. Once the Scrantron was invented, if a teacher took the time to write a good exam the machine checked the answers and the teacher just wrote down your score in the grade book.

NERD Jk

Dont have multiple choice at all in Germany

@@rv_354 I think I maybe saw one or two multiple choice tests in my education. And none of them affected grading.

That’s evil. Kids will be second guessing their answers thinking it can’t possibly be all true.

One of my teachers did something similar to that back when I was in elementary school. She gave us a 20-question, True or False test, with every single correct answer being True. I was reasonably prepared, so I knew most of the questions. But as I was going through the test I was experiencing a myriad of emotions with ever question I answered (incredulity, doubt, disbelief, etc.) like, “would someone really make a test with the same answer for every one? I’ve never heard anyone do this before!” Question 20 was the only one I wasn’t sure of, but at that point I was incredibly paranoid. I thought to myself, “There’s no way she would be doing all True answers; the last one has to be a trick to mess with us!” So I answered False in the last one, and missed out on a perfect score for that test because I started to over-think it, and talked myself out of guessing the correct answer based on the overall pattern of the answers. Anyway, that was the one and only time I’ve ever come across something like that on a test. 😐

@@PaperclipClips in fairness, I'd probably only do it on a non-important quiz or something. Possibly as a way to give an example of something. Bias, maybe? I don't know, I'm not a teacher. (Which is probably a good thing.)

@@PaperclipClips I had a similar experience, but it was a FINAL EXAM in COLLEGE.

You called?

Haha genius.

Some infinitief are bigger than other infinities!

You could easily flip that around to get 1+1=/=2 is false 1+1=/=3 is true 1+1=/=4 is true Each true statement has a corresponding false statement, and vice versa.

@@ashla2594 well played my friend. However, that only works in mathematics. Translated into language true or false questions its not so easy. Double negative phrasing is typically considered a cancelation. - the boy threw the ball -the boy didn't not throw the ball These two phrases in English are redundant, completely interchangeable, and in terms of the latter, not a correct phrasing of the situation because there is no utility to adding the two negatives. In math, 1+1 and 0+2 have utility to being distinguishly different. In language there is no utility to differentiating the same concept. So I guess the real answer is, can we describe a bigger infinity with language or can we describe a bigger infinity with math. I dont know the answer to that.

Duke How does that example show that there are more false statements though? You’re right in saying that the logic I’ve applied to the maths doesn’t really work for language, but you wouldn’t need it for language in the first place as you’d simply have pairs of statements where you exchange out ‘is’ for ‘is not’ (or in your example, ‘did’ for ‘did not’). I suppose you could say that you get more false statements than true statements when you have multiple statements in the same sentence (e.g. a sparrow is a bird because it has 4 legs), but that’s not really a ‘true’ true/false question, is it...

@@ashla2594 OK, so we have maths: 1+1=2 true 1+1=3 false 1+1=/=2 false 1+1=/=3 true Those are all fine because there is utility to each example being unique and independent from each other Language: My name is Duke. True My name is not Duke. False It can't be that my name isn't Duke. True It can't be false that my name isn't Duke. False The latter 2 examples of language have no utility and are invalid ways of expressing words. Now, with that in mind lets back up to your and my original comment and reply. I said there are more false statements because there are an infinite amount of false statements for every 1 true statement. You proved me wrong there by showing you can create a new unique true statement from each false statement. Now we get into my reply to your reply. I showed that creating a new unique true statement from each false statement only works with math and cant be translated into lamguage as easily. Therefor making the question "Are there more true questions or false questions" effectively impossible to answer using our shared line of theory using infinite numbers because we cant prove a larger infinite in true or false math questions vs true or false language questions. I hope I explained my thought well enough. I'm starting to confuse myself. Edit: I guess the answer to the question is effevtively: There are more true answers in math, and more false answers in language.

Agreed and his voice is so comforting when he is narrating..

Agreed, and Josh's comment that Stephen is "everything that's wrong with British education" really hurt, because I wish everyone in education was more like Stephen Fry.

If only I'd had Stephen Fry sat beside me when I was taking my exams. 😀

Yeah, a lot of people make that mistake. The error is in misunderstanding the relationship between the probability of each individual coin toss (1/2) and the probability of every coin toss landing heads (1/2 x 1/2 x 1/2 x ....). They probability of flipping 50 heads in a row is very low, because of all of those nested probabilities, but each coin flip is still a 50/50 chance - and that is a very counter-intuitive concept to grasp.

I'd say that as the probability of 49 heads in a row is so low, it is likely the coin, or the way it is tossed is biased; thus it is more likely the next toss would be a head rather than a tail.

Your claim of being an educator just makes the mistakes in your post all the more glaring.

All letters are silent until you read them aloud. Don't forget stamps

In high school, while handing out a test, one of my history teachers casually said, "You know, they say 'c' is the most common answer on multiple choice tests, but I think 'b' is better." On the test itself, there were two questions that were from the book that we hadn't covered in class, the answers to both were 'b'. Only two people got both the questions correct. The one who actually read the chapter (yes, only one) and the one person who actually picked up on the in-retrospect-very-obvious clue given as the tests were handed out. I was not either of those two people.

I had a written test that included the question "Spell polynomial". Four people got it wrong

@@DavidWillanski It's a common mistake. Not a lot of people know that you have to separate each letter with a dash.

Even then, I've been able to abuse the fact I know the thue-morse sequence in the past :P

Yeah, because humans try to “hide” the correct answer, thus avoiding A or D. Now with those programs, I think the advice is to just pick a letter and run with it. Because if you hop about you could actually miss the right answer on every one of your choices. But I also feel that math people would say that random means it wouldn’t matter.

I feel like a lot of my courses now do that to curb cheating. So if someone chose A for question 6, and you also chose A, it doesn't mean you chose the same answer because the order is randomized

If the students knew you did that, they’d expect roughly 50% of the answers to be true vs false :P

Except, no, because he didn't say 'you answer the ones you *think* you know first' he said 'you answer the ones you *know* first. If you get the answers you know wrong then, by definition, you don't know them. There is a fundamental difference between *knowing* a thing and *believing* you know a thing. Knowledge by definition requires the thing that you know to be true.

@@smockboy But knowledge is a subset of belief. When you say you know something, that just mean that you believe it very strongly. And if you're not bright enough to know that you don't actually know something then you won't know that you don't know it.

True

Someone forgot to deinterlace

False

The question being asked to the panel isn't about the number of possible true or false statements, but rather what the answer in a test is more likely to be based on how question setters tend to formulate their questions. As is stated in the clip, question setters tend to use true statements in their exams more often than false statements because it is much easier to write a true statement than it is to invent a false one (because with a true statement, you rely on what you already no to be true without having to think about it, whereas with a false statement it requires a little more thought and effort). Essentially, the main point is that question setters use true statements more often because sometimes they are too lazy to come up with false ones.

As for the preference for alternating true and false, there could be two things going on here. The first is a genuine psychological bias toward alternation, which has been observed many times in many different environments. In particular, people tend to think long streaks of one or the other of two options are much less likely than they really are. The classic experiment is to give kids coins and tell them to flip them 200 times (or however many) and record all the flips. Some students will actually do this, while many will cheat by arbitrarily writing down the letters H and T 200 times. The latter group can be reliably distinguished by the former, because they will almost never include long streaks of heads or tails, when in reality, after so many flips, you are practically guaranteed to have some. The second possibility is that the analysis was statistically flawed in the same way as Vallone and Tversky's 1985 paper on the hot hand effect in basketball. They concluded that the "hot hand effect," wherein a player who has recently scored many baskets in a row is more likely than usual to make the next one, did not really exist. They demonstrated this by looking at a large number of shots (field goals in one case, free throws in another, and a controlled experiment in a third) and comparing two data sets: one set was shots attempted after a missed basket, and one set was shots attempted after a made basket. They found that there were an equal proportion of hits following misses as there were of hits following hits. But their conclusion was wrong. In fact, absent any "hot hand" effect, you would expect to find _more_ misses after hits and hits after misses than predicted. This is because the way they chose their samples was biased. By only looking at the shots after a hit or only the shots after a miss, they effectively undercounted shots occurring amidst streaks of hits or misses. It's hard to explain this in a comment, but you can demonstrate it for yourself by performing the following thought experiment. Flip a coin three times. Look at every flip that follows a heads and calculate the fraction of such flips that were heads. For instance, in the sequence HHT, there are two flips following a heads, one of which was heads and one of which was tails. Therefore this trial yields a calculation of 1/2. Repeating the experiment many times will show that the average fraction is not 1/2 but only 5/12, which you can prove by listing out every possibility. This is due to the fact that the HHH sequence contains two heads following heads but only counts for 1 in the final average. Even as the sequence of flips grows arbitrarily large, the fraction you calculate will not approach 1/2. So the same thing could have happened in this study of true/false questions. A 63% probability is still a little higher than predicted even with a proper null hypothesis, but it might not be statistically significant depending on the sample size.

Also, that study has a typo at the bottom of page 2. ("poeple")

I have taught math at university. To me it is more important to reason correctly than to get the right answer. The students who made calculations with no comments were risking it. If they made an error I could not know if they fundamentally missed the point or just slipped up on something trivial. Of course, if you make an error that makes the problem easier, anything beyond that error can't earn points.

I teach math at a college. I hate multiple choice problems. They just are not effective at determining whether or not a student knows anything about that particular problem. But we pretty much have to give a lot of multiple choice questions due to current circumstances when classes are either fully or half online.

Was it a driving test?

Yeah my roommate had a test where there were 15 answers in a row where the correct answer was B. The prof just wanted to watch everyone second guess themselves

@@PabloSanchez-qu6ib no it was a history test

@@TheTroystreet so he was seeing if you would break, not brake, then.

@@PabloSanchez-qu6ib yes thank you for pointing that out

That's kind of surprising. Of all the people I expected Germans to be one of the more fond of multiple choice testing, preceded by Japanese

I never had to do a multiple choice (or true or false) type of test here in Ireland either. We usually had to know the stuff we were being tested on. There wasn't many cases where you could guess, and even if there was, you would usually need to prove your answer or show more knowledge on the topic in some way. I've never seen any kind of test where you could get points for doing it without trying at all.