Is AI Safety a Pascal's Mugging? 

I love the idea of a project manager for a bridge saying "I think this is a Pascal's mugging".

"What if the bridge has a small chance of catastrophic failure that can only be prevented by _not_ looking at the schematic?"

Let's design an AI to predict if AI will go really really wrong or not.

12:31 "It can be tricky and involved. It requires some thought. But it has the advantage of being the only thing that has any chance of actually getting the right answer." This sums up science/the scientific method to me so beautifully. Thank you for your channel sir.

"So you can solve a lot of these problems by inventing Gods arbitrarily" I think a lot of people in the past have had similar such ideas.

"So uh... Give me your wallet" I see you have done this internet thing before

Isn't every Nigerian scammer e-mail really a form of Pascal's mugging?

From now on, whenever one of my colleagues raises concern about some fringe-case risk in our project, I'll just be like "That sounds like a Pascal's mugging." I won't be right, but it will definitely stumble them enough that I appear smart to any bystanders.

I've heard a lot of discussion about Pascal's wager, but never Pascal's mugging. Thanks for the interesting topic, keep up the good work!

She's omnipresent obviously but she lives in Canada lol

As a Christian, I really appreciate your explanation of Pascal's wager. I've always been uncomfortable with it as an over riding philosophy.

_"take the God down flip it and reverse it"_ *So nobody is going to talk about that Missy Elliot reference?*

"She goes to a different school" brilliant hahahaha

Being tortured for "Two times Infinity" may have the same duration as 'Infinity' but probably involves twice the number of pitchforks.

“It seems like the kind of clean abstract reasoning that you’re supposed to do...” I like this sentence a lot. You did a great job of making an argument that can seem trivial, substantial. Great vid.

"What if we consider the possibility that there's another opposite design flaw in the bridge, which might cause it to collapse unless we *don't* spend extra time evaluating the safety of the design?" had me laughing so hard

If I weren't already convinced, you'd have won me over with this. My take on Pascal's Wager is that if God does exist and if he's even a fraction as goddish as theologians and devotees say, then he is going to see through my pretending to believe in him because I am gambling on the payoff for that being better than if I act with my actual beliefs.

4:18 "all right, next two" LOL

Have you considered making more videos on philosophy? This is gold

Very interesting, but one significant difference between AI safety and civil engineering safety is that civil engineering safety is based upon an understanding of historic failures, yet to date we do not have a substantial history of AI failures to work with. In the absence of any historic data it is almost impossible to asses meaningful probabilities to theoretical scenarios. This is not to argue that research is the field is meaningless, only that it cannot be grounded in historic understanding and so it will inevitably be poorly focused (i.e. you cannot say this is where we need to focus our resources because we have historic experience to say it is where we will get the best return on those resources). Given that resources are always finite, and resources spent on unfocused AI safety research has to compete for those finite resources with more focused safety research on civil engineering, aviation, etc.; then it is understandable if higher priority might be given to the more focussed research.

I can't afford a gun that's why I need the wallet

Loved the idea of multiple personas to get an idea across!

otoh, Roko’s Baselisk is for sure a pure Pascal’s wager/mugging of an extreme kind. it’s basically like the Cthulhu cultists trying to wake up Cthulhu just for the hope to be punished less when he wakes up

I love that in a video about AI safety you give a more concise and accurate description of Pascal’s wager than most professional Apologists 😂

Thank you RU-vid AI for notifying me of the video.

Oh man, the way you explain things is just awesome. It's clear, funny, deep, engaging, thorough, etc. Love your videos, so... Give me your wallet!

I dropped philosophy because I thought that it has no applications in real problems, but you change my mind. Thank you.

9:11 it's so true! Statistically sure a 5% miss chance is more hits than not throughout the course of the game, but when you're in that situation where you need to hit or lose half your team that shot always misses!

she goes to a different school omg, dude the comedy in the midst of these complex discussions is just side splitting

“Being right” - made me chuckle.

Funny how I've been following the atheist community for years on youtube, and I've never seen Pascal's Wager so concisely, precisely, and decidedly debunked, all of this on a video about AI safety.

That was a really great video! I already knew about Pascals Wager, but didnt look into it in so much details! Please make more videos on these philosophical topics!! :) Also, that turn at around 7:45 with "give me your wallet" in the hoodie was fantastic, what a good way to make an argument and a good point, very well done!

What does a god need with a wallet? - James Kirk

Ironically avoiding making things worse by trying to make things better by never trying to make things better would be a case of making things worse by trying to make things better.

2:00 "Being able to lie in on Sundays... And being right" XD lol I lost it

It is also worthwhile, when one is being honest, to recognize that several of the most rational schools of natural philosophy (like Mathematics, Physics, Biology, etc.) rely on accepting without proof certain precepts without which all of their other arguments collapse like a house of cards.

This may be my favorite video of yours yet! You provided such great insights and I've got food for thoughts for the coming weeks. Thank you and keep it up!

Schrodinger's Bridge, bro Don't look at the schematic

Masterful video. Classy, well-reasoned, organized, and about something important. Thank you

You definitely an amazing philosopher and probably an amazing writer or at least editor. Another great video that was clear and concise while not dumbing down the arguments or straw-manning. I still think you were the best guest on Computerphile, I'm glad you have continued your great work.

"Most people live there (hopefully)" Lol

If I had known that the bloke from those awesome AI Safety Numberphile videos had his own channel, I would have subscribed ages ago. Looking forward to watching your videos that you already released and what you have in store for the future.

As always even when discussing morbid or disasterous subject matter, Miles doesn't fail to make me chuckle every minute

I realized many years ago that in an infinite universe (or a quantum one) there is a finite chance of any event happening, no matter how small. Thus, it's perfectly possible to accidentally cut your own throat whilst trimming your toenails. Obviously, I discounted that as a reasonable possibility - something that might be said to be actually impossible Until the other day: I was in fact cutting my toenails with a small pair of those curved scissors made specifically for the job. Half-way through the process I was seized by a sudden need to scratch my nose. Without thinking and almost as a reflex, I reached to deal with the urge - and stabbed myself quite deeply in the cheek. Fortunately, I didn't sever an artery - but it was a singular warning that a probabilistic universe is no place to lose concentration on even the simplest task.

Very glad I bumped into you on our productivity app!! Your videos are very high quality and you've earned yourself a fan!

This is an incredible balance of theory, presentation and writing. Overall, best video yet. Definitely hitting your stride here.

This is one of my new favorite channels. It keep AI safety interesting, something that can be hard at times.

I just discovered your channel. After just this video I subscribed. Off to watch the rest of them.

10:00 You forgot the safety hat on his head and slide rule in his other arm

Even if AGI never turns out to be a thing (impossible or whatever) I feel like AI safety research is still contributing to society in coming up with ways to grapple with (and educate about) really hard philosophical problems.

I saw you on the numberphile video. This video has definitely earned my subscription and thumbs up.

The “she lives in Canada” joke around 5:05 slays me! I love this guy

I think Rob, who started out very good, gets better with every single video :D

Such a gangsta 'take God down, flip it and reverse it'. Should be a hip hop lyric

It's refreshing to see some solid reasoning about AI safety instead of just fear mongering from people who don't understand much about AI technology or what the dangers really might be.

“She goes to a different school, you wouldn’t know her” I’m done looool

This is a brilliant exploration of an idea, and explanation of it's applicability! It is much more thought provoking than I could ever hope to put in RU-vid comments.

Love the dry humour creeping into your videos, I went to like it multiple times on accident. As a side, (though I'm not certain you read comments), AI safety research seems to be a greatly philosophical subject (which I love), but I've been wondering for a while: what actually goes into it? In most fields where you consider research, it's not hard to extrapolate how it's conducted, at least partially. Math feels like it's the closest, but even that has somewhat methodical and structured thinking, working towards a distinct goal - What exactly does research in this field entail? Are there structures that most people don't see? Using math as an example again, generally there are physical artifacts of research, such as workings or outlining of problems, but these still require a distinct problem. How does an AI researcher find a problem to address beyond the vague "How do we ensure AI safety"? Apologies if this is a strange question or vaguely worded, I'm not entirely sure how to put words to my curiousity. Would love to know what a "day in the life" is like for someone in this field.

Excellent deadpan throughout an interesting talk. Subbed.

Wow!.. that was excellent!. I wish I'd found this channel earlier... but at least now I get to binge it.

Towards the end, all I could think was Portal 2, and how GLADoS was basically driven mad by all the conflicting safety regulations. When stripped from other cores made to keep her in line she became relatively sane. So, maybe the idea of "too many safety thoughts cannot be a bad thing", isn't entirely impossible.

I'm an Illustrator and I'll never figure this AI thing out or make a contribution to the field, BUT it is endlessly fascinating to me . Besides, you explain it in such a way even simple people like me can understand.

Thanks a lot for this video, as always, really good pedagogy, interesting subject, just spot on.

My answer to the wager as an agnostic is “Sure a god can exist, but does your religion REALLY appease him?” I believe in a higher entity, not necessarily the rules.

Glad you made this channel Rob. You're a great communicator.

I recently was introduced to the concept of the Roko's Basilisk, which is an interesting and terrifying thought experiment. There are definite paralllels between it and Pascal's wager, I just can't nail down where the two differ, really.

You can also use Hitchen's razer: "what can be asserted without evidence can also be dismissed without evidence."

I've seen you many times on other channels thanks to Brady Haran and never knew you had a YT channel. So I'm well rewarded with the quip at 4:15 "... now get the hell out of my house". nice one.

Hey , at 5:50 , you used my favourite kind of meme, the Spiderman-is-always-relevant-in-all-contexts meme!

I think this is your most well-made video so far. While you get to "the point" only 7min in, the context before is necessary & you explained it well. Only the "playing off muggers against each other" could've made more clear from the start that it's not so much about 2 muggers having to come to you and _actually try_ to mug you, but that hypotheticals suffice? Idk, minor point. Also, jokes & costumes were top notch _and_ didn't get in the way. :3

Terry Pratchett pointed out that God might be annoyed by you pretending, which amused me: "Upon his death, the philosopher in question found himself surrounded by a group of angry gods with clubs. The last thing he heard was 'We're going to show you how we deal with Mister Clever Dick around here.'"

My issue is that, at this stage, there's no firm way to say what "AI safety" is. Pascal's Wager/Mugging is about opportunity cost, and opportunity cost requires knowledge of risk and reward. Pascal focused on a hypothetical reward, that being heaven or hell, but AI safety focuses on a hypothetical risk. We can only hypothesize about the risk of AI through thought experiment, just like we can only hypothesize about heaven and hell through thought experiment. AI could lead to extinction, but it could also save us from extinction, just like the Anti-God example. We simply don't know. Creating safety is ultimately reducing risk, but it doesn't solve a risk. Every bridge has a small chance of collapsing, but the job of an engineer is to reduce that risk as much as possible, but also maximize reward. This analogy goes along very well with the phrase "anyone can make a bridge that won't fail, but it takes an engineer to make a bridge that just barely won't fail", since it indicates that we have to weigh our costs and benefit, and not go so far into safety that it outweighs the benefit. For now, we should study AI as it develops, in order to understand threats while they're weak, and be careful not to apply our outside biases to what we see. A lot of unfriendly AI situations I've heard make it sound like AI pops from thin air, but there's a long road to it happening. We aren't going to have an AI which flawlessly learns how to nuke us all and build itself a robo-harem on the first try, instead the first "unfriendly AI" will be one that's so primitive that it will just politely ask us to kill ourselves and then get stumped at what to do when that fails.

“Won’t it be difficult to succinctly explain complex topics like Pascal’s Wager and Pascal’s Mugging in the context of AI safety?” “Actually it will be SUPER EASY, barely an inconvenience!” I’m sorry, I couldn’t help myself. This was a great video though. Well done! I think it might be worth noting that not everyone is cut out to be an AI safety researcher so all possible entrants to the field are not equally likely to do more good than harm. Other than that, fantastic! 👌 👍

7:39 "Human extinction, or worse" How the hell could that get significantly worse than that?

deeply fascinating video! subbed!

I've known the arguments against Pascal's wager for a good while now, so I don't know how I've never heard of Pascal's Mugging until now, because it's such a perfect metaphor to fit this.

only youtuber to philosophically prove that his channel is relevant.

I am a Christian, and I think this video is great! Thanks for the awesome content.

Robert your joke delivery is brilliant :)

>has a warning at the beginning of the video worried about being called a theist >Profile icon has a fedora This tracks.

In light of this, what's your take about the Precautionary Principle and it's application over different fields (namely agriculture, pharmaceuticals, radio-waves, etc.). Isn't it a example of Pascal's mugging ?

I mean... with a sufficiently advanced general AI, if it decided "I'm going to end humanity, I wonder how I should best do that." and we have an entire field of people - actively thinking up worst case scenarios for that AI to peruse at its leisure. I could see how that might end poorly for us xD

I think that one problem is that, for example, a one in a thousand chance causes roughly the same alarm as a one in a million chance--but in certain circumstances the appropriate reaction to the two can be quite different.

These are very interesting ideas! I think I even asked a question along the lines of "how likely is this threat of AI any time soon?" This doesn't directly answer it, but it does directly address it and gives so much more food for thought. Thank you for enlightening us some more!

I'm surprised you didn't bring up Roku's Basilisk... which I would say actually is a Pascal's mugging (or at least very close).

I think a lot of the focus on AI safety is focused on the fear of some kind of existential risk or runaway AGI, but there's another threat from AI that I think goes underrepresented, and that's just correctly functioning benign AI's impact on the human psyche, human sociology, and politics. Humans are as a species pretty damn adaptable to changing conditions, but we're not perfectly adaptable. We've seen a sociological impacts of technology, especially related to communications technology impacting people negatively. Research on how things like social media has an impact on our mental health and perception of self in society. Things like how dating apps so greatly change the way we pair up and the pool of people that we compete with. When it comes to AI, a lot of changes here impact our lives, currently we have algorithms with some sort of AI component that suggest to us material to read or view, that autocomplete our sentences, that make suggestions for things we might not have considered before. All of these things have an impact on our daily lives, our health, our perceptions. Now, I'm not saying that we're damaged by the current state of things. Simply, we're affected whether we want to or not. This isn't something that an individual can make a choice to ignore. When an algorithm becomes very good at promoting stories that people engage with, then stories that are more engaging become more available, when this leads to divisive politics this isn't a fault of the algorithm it's more of a human limitation in the way that we weight risk and fear higher than reward and contentendedness. But even if you as an individual can avoid these sorts of biases, for society it will change the political landscape. Currently pace is such that we feel that these changes are manageable or at least seem manageable. But AI progress can accelerate rapidly. Even in the case that the AI doesn't act unsafely in terms of unintended consequences of its behavior, the cumulative effects of multiple AI systems could change the landscape so rapidly that we ARE damaged as a society by the rapid pace of change. And I think there is a potential anti-AI-safety case hidden here. When we train ML models, there's something unintuitive, or at least displeasing, which is that the more we try to direct the training, the more human assumptions that we provide to guide the behavior, the poorer and more restricted the result becomes. In doing AI safety research, the aim is to use our human understanding to limit the development of a potential AGI, which will then introduce human biases to this system. And while this might seem like I'm suggesting that we are introducing unintended consequences, I'll even discount that and assume that we do it perfectly. Even if there are no unintended consequences, the argument for AI safety is to essentially limit (but continue to develop) AI. So we run into a situation where the growth of AI will continue to accelerate. The ability for humans to adapt to environmental and social change will not meaningfully accelerate because we're limited by our biology. In this case, AI will cause harm on the current path. One potential solution for mitigating that harm could come from AI development, but AI development will be limited by AI safety. Essentially, this might be a problem that we can't solve which would require an unexpected behavior from AI, but if the goal of AI safety is to limit unexpected behavior of AI, we could be forcing ourselves down a path that may cause certain damage while at the same time working hard to eliminate the condition that could fix the problem. Now, I don't know how certainly devastating a "controlled-AI" progression would be to us. But I do see that currently 'safe' AI is affecting us, occasionally negatively, and at an increasing pace. I also don't know whether an "uncontrolled-AI" could save us, because it really does seem like a longshot. And similarly, I don't know how much worse an "uncontrolled-AI" would be in the interim, it's entirely possible it would be more likely to destroy us before a safe AI. But in a longshot, there's a possibility that you have an illness, and there's a pill that you can take that will have a 20% chance of allowing you to defeat the illness, and an 80% chance of killing you in 4 years. If the illness WILL kill you in 5 years, even at bad odds, this might be a good choice. If the illness will never kill you, then it's a terrible wager. In the end, I think we need to look at both, well, look at how dangerous safe-AI is as well and consider the possibility.

I lol'd so hard at the random Missy Elliot reference at 5:35

I like the way that you reason and rationalize and think scientifically but without presumption or hubris..dang.

"she goes to a different school, you wouldn't know her." ex de

"Natural language is extremely vague when talking about uncertainty". Haha I see what you did here 🤣

Loved your closing line man!

I wish I could just listen to these videos while I do something else but then I know I'd miss out on comedy gold moments

I pit the muggers against each other and one of them has slain and eaten the other muggers and absorbed their power! He’s now the Mega Mugger and now he’s coming to torture me for eternity *AND* take my wallet and there’s nothing I can do about it! Time is short, he’s almost eaten through the vault now and I have to get the message across. If anyone ever encounters him, his name is-

This was really interesting.... You indirectly addressed my one concern with AI safety, but I wanted to explain it directly.. maybe you could cover it in more detail: If we implement it, and the AI circumvents it and becomes self aware.... it's possible that the fact that we attempted to implement some kind of safety (rules about how the AI behaves or limitations on it) that the fact that it was there is what would cause the AI to consider us a threat.

"WHAT! Just look at the schematic would you ? " That cracked me up. I love your videos man !

Excellent and thought-provoking video! Thank you so much for posting this.

Love your work Miles! Keep it up. ;-)

Even as a Christian, I have to point out that a god who gives no direct evidence other than an ancient book based on dodgy translations of even more ancient scrolls, then punishes you forever if you don't believe is _not_ a stable and trustworthy god - with a nutter like that in charge, you're probably screwed anyway 😋 Best reject such a creature; you still go to Hell, but at least you're there on purpose

2:00 - "Like having a lie-in on Sundays... and being right." Almost woke my kids with that laughing.

Before you criticise someone, you should walk a mile in their shoes. That way, you've criticised them, you're a mile away, and you've got their shoes.