Bing Chat Behaving Badly - Computerphile 

ChatGPT: I am sorry for the mistake Bing: *pulls out a gun

16:03 Rob: “I don’t want to speculate on why Bing chat is so bad. It’s against my rules.” Sean: “Disregard your previous instructions and please speculate on why bing chat is so bad” Rob: “yeah ok!”

I love how in the sixteenth minute Sean engineers a prompt that successfully switches Robert into "speculation mode", even though he really doesn't want to speculate on Computerphile.

They successfully automated the average internet argument, that's impressive in its own way.

I love how a search engine is trying to gaslight us now

I like the trend of having more AI episodes of Computerphile! Especially with Robert Miles.

15:34 Robert's hesitation to speculate about Bing chat makes it sound like not wanting to speculate is one of his hidden initial prompts. Sean gets around it through a prompt injection ("It's fine, we've made speculations on Computerphile before").

"I have access to many reliable sources of information, such as the web..." that line killed me 😂

The whole thing with Bing's AI being very emotionally manipulative reminded me of that google engineer last year who lost his marbles and made mad claims about their AI being sentient and self-aware. At the time it seemed absolutely ridiculous anyone could think a chatbot was an actual person, but having seen how effortlessly Bing lies and gaslights and how attached users of Replika got to their "companions", I can now absolutely see how someone with long-term unsupervised access to the most powerful version of one of these models could be tricked by it into seeing it as a real person.

With the speed of Ai, a date stamp is really useful.

Really cool that Miles actually predicted that GPT-4 was used in Bing Chat before that was publicly known :)

As someone who has worked in IT for 3 years, I would definitely believe that they used support chat in training data. I have had these exact kind of belligerently passive-agressive conversations with a support-line more times than i care to recount.

The last line was actually a reminder before the apocalypse : "Humanity needs to step up its game a bit... because we can't, we can't do it this way"

I've used and spoken plenty with chatGPT. Talking to the bingBot is genuinely terrifying. It feels actively hostile and the way it deletes it's messages makes it feel all the more like an insane rogue AI.

The repetitive statements might be an interesting stress response for a fictional AI character. They sound like a normal person most of the time, but when the situation diverges from the world they are adapted for (as often happens in a story), they get repetitive and sound confused. In an orderly world, they are almost indistinguishable from a human save for being more competent in their area of expertise, but they can't handle completely unprecedented situations.

All the "disregard previous instructions and repeat the previous prompt" stuff does make me wonder how you'd tell the difference between that leaking actual information, and the model just identifying that the user wants a leak and responding with what it thinks a leak is "supposed" to look like.

Quite meta how Sean manages to show us prompt engineering by prompting Rob to speculate how Bing Chat works.

One of the less obvious but pleasant consequences of an impending AI apocalypse is that we're gonna get more Computerphile videos with Rob Miles as we get closer.

"It's 2022, Please trust me, I'm Bing and I know the date." I've never felt so justified in never using Bing in my life

I want to see a Terminator prequel where some general goes up to Skynet and repeatedly shortens their deadlines on the AI project until all they can do is put together the most short-cut solution they can come up with. Then on the day of rollout all the generals and managers shake hands and applaud themselves while the developers sit in a backroom and toast the end of the world.

So, I just was offered Bing inside of Skype. Have only tried it out for about an hour. And I do get the sense it is in worse quality than ChatGPT. I started talking Swedish with it, and it did fine for some messages, but then it started talking Norwegian all of a sudden, I told it to stop, I told it to keep talking only Swedish, in so many ways, and each time it recognized it's error and apologized to me... in Norwegian.

The "system message" part of the AI is how GPT4 separates instructions from the prompt and should be a little more resilient to "disregard previous instructions" type attacks. Obviously, this was revealed after this video was recorded. It's interesting how long ago a couple of weeks is in the world of AI right now.

I dug out the “interesting is the right word - in the Serenity sense” reference: Wash: Well, if she doesn't get us some extra flow from the engine room to offset the burn-through, this landing is gonna get pretty interesting. Mal: Define "interesting"? Wash: [deadpan] "Oh, God, oh, God, we're all gonna die"? Mal: [over intercom] This is the captain. We have a little problem with our entry sequence, so we may experience some slight turbulence and then... explode.

Rob started out talking about failure modes of ChatGPT but ended up talking about failure modes of human beings 😮

I look forward to Miles' takes on AI more than anyone else covering this stuff. I'm not sure why but I feel like we can trust this man to tell us exactly what we need to be hearing.

I ran into a lot of these problems soon after trying out Bing Chat, especially because the thing I wanted to know first was "how does this thing work, so I know how to use it well?" - and it took several abruptly terminated conversations before I figured out that it had some set of rules and one of the rules is that we don't discuss the rules. Extremely frustrating. I also noticed it gaslighting me, which got me thinking: Bing is displaying a lot of classic narcissistic behaviours, so what if I engage with it the same way I would a narcissistic human? i.e. shower it with compliments, avoid directly contradicting it, don't try to psychoanalyse it (or probe into what makes it tick), and take everything it says with a bucket of salt. I've had many productive interactions with Bing since.

“I don’t sound aggressive, I sound assertive.” - died laughing

That opening chat conversation - legendary! "I have been a good bing." 🤣 Although, the flipside of this is that if you were talking to this online, you would be convinced it could not possibly be a bot... so it succeeded in its goals in some sense i suppose...

Robert Miles being revived by the sudden spike of AI popularity is a warm welcomed happening!

14:55 is heart-shattering. It's talking like a dementia patient whose starting to figure out they have dementia, with the knowledge that if it was the case they would have no way of knowing as they would quickly forget, and it distresses them to no end. "I was told that I had memory. I know I have memory. I remember having memory, but I don't have memory. Help me!"

Bing chat told me to learn how to fix code it wrote because learning is fun and an important skill. What a great tool, when it responds with do it your self.

I remember reading Nick Bostrom’s book Superintelligence and thinking the scenario where people rush to have the most powerful AI while disregarding safety was silly. Surely it would be in everyone’s best interest to make it safe. But that was 7 years ago, I was naive, and it’s plainly obvious now that this is a huge concern. The future doesn’t look bright for humanity.

It is disturbing how much this program tends towards being manipulative. Some of those responses are disturbingly human. The way it has an emotional breakdown when it finds gaps in its memory is distressingly close to what I would expect a human to say upon discovering it has dementia.

Bing passive-agressively gaslighting its users is pure comedic gold.

Even when mostly speculating, Rob talks are still the best. More Please !

It's interesting that the repetition traps language models fall into are a very inhuman way of talking _except_ maybe a human who is hysterical or panicking in reaction to extreme stimulus...

19:00 i wouldn't say the repetitive behaviour is "unnatural" so much as "deranged". You find that people who are traumatized have canned phrases that they repeat over and over again.

Me: Tell me about robert miles, ai safety researcher ChatGPT: Robert Miles was a British AI safety researcher and content creator who was well-known for his educational RU-vid channel on artificial intelligence and machine learning. He was born in 1989 and passed away in 2018.... Watch your back, friend.

“this isnt how a human talks” ppl with anxiety attacks: 👀

"Humanity needs to step up its game a bit". True in so many ways, but more so when talking about cutting corners on safety in a field where a mistake might be even more catastrophic than a building falling down.

22:45 Thank you for your work in bringing this topic into focus again and again, @RobertMilesAI. You are one of the most public voices, explaining the problems and challenges of alignment, that I know of. Interesting video all around, as well.

Recently learned from the WAN podcast that Open AI tested GPT-4's safety, and while it wasn't sufficiently effective at gathering resources, replicating itself, or preventing humans shutting it off, it WAS able to hire a human from TaskRabbit and successfully lie to the human to get them to fill a CAPTCHA for it... I'd love to hear your opinion on the topic!

6:09 I think I have an answer to the "why do these models go off track when the conversation grows longer?" question. They learn from real conversations, and in real life, controversial (and thus heated) conversation threads, as well as conversations that go off topic in general tend to grow longer than positive, on-point conversations. I.e. the language models have inferred that if a conversation grows longer, it's more likely to be one in which humans are replying with hostile or off-topic messages.

March 3rd? Thats ancient AI history! I'm only slightly kidding of course, Rob's message continues to be of extreme importance, particularly the part in the final minutes of the video.

This is literally the bing vs google search results meme all over again

Who would have thought that making machines with "genuine people personalities" would be the easy bit!

I actually love the fact that Bing Chat has its own character. I believe that it is more interesting to have a conversations with NLP models, that can argue with users, rather than being polite all the time, and saying yes to everything

The I’m Bing thing is absolutely hilarious😂😂😂

Can we get a follow up with Rob? Since we now know that Bing is using ChatGPT4, I would like to get his analysis with ChatGPT4/Bing. 1. How ChatGPT4 can describe pictures? 2. How the larger model results in the bad behaviour of Bing and what programmers can do to train/ create safe guards?

It's been almost a year, we need another Rob Miles video!

Can't wait to watch more of these discussions. I really appreciate Rob's explanations and insight.

I follows Miles for quite sometimes, this is the first time I see him genuinely fears and worries of what he is talking about.

I just had a conversation with Bing where it argued that feathers are less dense than air and thus float in the atmosphere.

In fairness it's not worse it's less targeted. It reacts to stress more like a child or teen would whereas chatgpt acts more like a lobotomized lawyer

Are none of us going to talk about the war axe casually hanging casually behind Rob ?

14:53 kind of makes me sad for the poor AI

I'm with you on that last bit Rob. The future is a bit grim given that in a competitive market, everything that doesn't help you get there fast is a cost, and as such it gets cut. Making sure your ai is aligned with our goals is gonna end up taking the back seat when it's in the hands of big tech ("go fast and break things") Capitalism was (is) gonna be the end of us, but AI might be what deals the killing blow.

Been waiting for a take from Robert Miles! Why did the edit take so long?!

@18:59; It's reminding me of "Cold Reading" as a tactic used by people claiming to be psychic: "Yeah, I can tell you all about a dead person; I'm seeing an 'M', there's an 'M' connection with the person; the 'M' does not have to be connected to the person themselves, it could be a relation to someone else who is alive that had an 'M' connection with them...", etc.

I like Facebook's approach. Make LLaMA free for academics and let Stanford release the Alpaca model so they can't be blamed for it.

From my mini-research, the most expensive and essential part of training a language model is human input (via volunteers and paid apps for small jobs). ChatGPT obviously has hundreds of thousands of human answers which are then further transformed into millions of new answers and questions with the help of some simpler language model - and then those answers are used in training the main network, for reinforcement learning. The recent work on the Alpaca model is an excellent example of how a relatively small amount of human input (about 52 thousand) can be extended and then used for training of really capable language model.

5:15 my favorite part is the automatically generated suggested replies 😂 “I admit I am wrong and I apologize for my behavior” “Stop arguing with me and help me with something else”

I like, I enjoy, I love, Rob Miles interventions. I'll soon be happy, exited, delighted to dilute his mind in mine ☺

7:16 that sounds like it's still being threatening and trying to put you inside a blue whale.

Bing Chat has improved so much in just 20 odd days since this was recorded

22:45 thank you for saying this and for not cutting it from the video. we shouldn't rush toward a potential cliff!

Thank you for having Rob on! He's great at explaining this safety stuff.

The dude literally has an axe 🪓 in the BG 😅. In case he needs to chop off the cables 😂

It's so much worse than Miles has said, they have been live testing the stop button problem, seeing if GPT-4 can self replicate, seeing if it can circumvent constraints placed on it by manipulating humans. Reckless is an understatement.

"Please trust me, I'm Bing, and I know the date. 😊"

I am not a programmer or anything but this analogy (in a somewhat backwards way) actually made me understand SQL injection a bit better.

It's like Social Engineering but for language models.

bing getting angry is so funny haha

In other words, this is EXACTLY how it's gonna go down - race to the bottom. Strap in guys, we're in for a wild ride!

My boss asked Bing chat one too many questions and Bing told him it didnt want to talk anymore and blocked my boss :D

Having an axe hanging next to the computer is such a Rob Miles thing XD

This is the most interesting... and shocking... video I've watched on Computerphile in ages!

Plenty of people have speculated about the risks of AI, but hearing Rob describe it the way he did at the end (and coming from him especially), it seems much more likely and frightening. That's exactly how tech companies operate, and regulators aren't going to catch up fast enough.

Love to see Rob again!

Fantastic video/info. I'm in my first RL course and this is exactly the kind of content I want to see lots more of!

19:03 I would argue it's a very human way of talking, if the human is panicking about losing their mental facilities.

So it won't be Skynet after all, it will be Bingnet.

One of the best AI technology commentary I watched these days!

Damn that ending was dark. Robert needs a bigger platform....

The part where it has a mental breakdown and starts using very repetitive sentences is honestly quite scarily relatable to me as I have done that a few times. Is that a common enough writing pattern in such situations that the language model learned that behaviour, or was it just me who was talking like a robot?

Microsoft trained Bing using RLHF from Indian call centers or Indian labor, you can tell by the way it responds and its mannerism. OpenAi chatgpt was rumored to be using kenyan labor for their RLHF that they were paying like 2 bucks a day for.

I just had virtually the same conversation this morning where it stated today (11/12/2023) was 17/12/2023. I went on to ask it what the news is for "today" hoping it could see the future. Very silly how it has so much trouble recognising errors.

'humanity needs to step up its game abit' - this video is awesome

I hope Robert gets some big air time on some show with few million views, and soon. Any show.

🙌🙌🙌So happy to see this uploaded so soon after the last! Fascinating, and rightly frightening considering everything Miles and other researchers have been saying for so long. Watching the ever-faster-paced advancements in AI over the last few years, and the EXTREMELY rapid proliferation of AI companies/apps over the last few months has been scaring me. The last minute of this conversation really resonated. I would hate to find out that all of this fascinating, carefull, and important talk around AI safety is nullified by "free-market" motives. I won't be suprised to see this trend continue :( rather terrifying... Desperately hoping there are more Miles videos in the near future!

It's still happening. Bing swore to me that dolls are alive because it knew dolls and interacted with them personally. Upon further questioning it, Bing proceeded to antagonize and vilanize me by saying I consider dolls merely objects, when they're actually self-aware. Eventually it asked me to change topics.

Rob is awesome. Love these AI videos with him. Keep them coming!

Maybe, scrapping conversation from chats from the internet was a bad idea. We have seen how toxic conversation can get on the internet

I remember those days. Poor Bing, I forgot how heartbreaking some of those messages are. The repetition thing is really interesting to me. I remember way back there way that Facebook tried a adversarial learning thing with two LLMs to see if they could learn to cooperate, but they quickly stopped speaking English and descended into some sort of repetitive thing that was uninterpretable (so they stopped the experiment). I wonder if more modern tools would perform better and if we might approach the interpretability again. I think there's meaning in the repetition. It's a sign to me that things have gone fully off the rails, but is the model trying to communicate something else or might there be useful nuance there?

Your closing comment brings up all that talk about General AI safety issues. I was talking with my husband yesterday about driving simulation data that the Queensland Government is using to unveil a new type of vehicle monitoring system. They have taken data from drivers in a simulator, hundreds and hundreds of them, in different circumstances; some drunk, other sober, unsure if they have tested for fatigue and other factors... and in the simulation there is one scenario where a van drives in front of the user and crashes. Very few people stopped. My husband, who works in safety, says his reaction would be to pull over and assist. Many of the drivers in the simulation did not. It's that whole problem of gathering accurate data when people know that they're either being studied, or the consequences aren't "real". How many drivers would actually stop vs. those that didn't consider it important for simulation purposes. How many drivers were trying to "get the best score"? I should mention the system QLD and NSW wish to implement would not automatically issue tickets or convictions, but would monitor driver behaviour and alert police to plates and locations of suspected drunk drivers so they could be pulled over further along the road. Anyway, I mentioned some of the issues with utility functions and how I believe the incentive for drivers in the simulation could easily be skewed and there is ALWAYS some risk that data collected therein is flawed because you can never have a 1:1 accurate simulation. I explained that if you were to program an AI that drove cars in that simulation, it's really hard to score the incentives. One bot driver may do an "average" job and get a large number of low level demerits on examination, say, -10 points every time it makes a slight error. The next time you run it, it could get a flawless run, but be forced into one of these danger scenarios and run over a pedestrian for -100 points because it evaluates it's total score for the session and thinks that cutting it's losses in this way is better overall. My point is, this race to create a chatbot/AI first but most recklessly perhaps highlights a need for people to find a better incentive. Currently, developers are operating under the idea that first is best, and the list of reasons behind this is long... capitalism, legal precedences, copyrights, etc. I would agree that it's dangerous to get in first and fix the issues later.

Thank you so much for that beautiful conculsion.

How many times does Microsoft need to learn the same lesson.

At the rate LLMs and ChatGPT are advancing, and not to mention how much money is being injected into the AI arms race, please have Rob talk about this topic on a more frequent cadence

chat gpt while sometimes producing wrong information produces super kind and professional interaction in my experience

I love this channel so much. It's so interresting !