Scientists warn of AI collapse 

This reminds me of when I was playing with play-doh when I was young. You start out with many different colors, and somehow always end up with a big brown ball.

"The more it eats its own output, the less variety the output has"... sounds exactly like the RU-vid recommendations algorithm.

My biggest problem with AI is that it needs to get the information from somewhere, and sometimes these sources can be slightly dodgy. I did an experiment where I asked ChatGPT about some very narrow subjects: The Danish organplayer Peter Erling, the trio Klyderne, and the artist Jørgen Fonemy. These are subjects that I have some knowledge about and actually have written Wikipedia articles about. I could see that most of the answer that I got from ChatGPT was based on the exact Wikipedia articles that I wrote! I have tried to write the truth in those articles, but if I didn't care if things were correct - or worse; if I deliberately wanted to mislead people, then AI would base the answers on wrong data, if there wasn't multiple sources available. The problem I see with AI is that we trust it too much. Already now there are people who believe that it is an omniscient trustworthy source of all answers and that it will always be more correct than human knowledge or just knowledge that we have googled or looked up in an oldfashioned book.

I remember as a kid I used to record my voice and play it back on my speakers, which I then proceeded to record once more. By repeating it, I could hear how it slowly degraded until it was nothing more than a weird, synth-like sound.

3rd possibility: AI learns how to gaslight us, and we forget how many legs elephants have.

This could lead to a .. Nightmare on LLM Street....

It's kinda like Google search engine. It started out sucking, then there was a time it was pretty good to find stuff. Now it sucks again....

Glad you started this conversation. There is also the theft component of generative AI. A RU-vidr like yourself will get automatically copyright struck for using 4 seconds of a clip in a 20 minute original video. Yet these generative AI companies can use entire social media platforms with content painstakingly created by individuals across decades to create their data sets. This is peak hypocrisy in which, as per usual, corporate "big money" is protected while the individual is left with no means of defending their content. Generative AI is 100% theft in my opinion.

Apparently in the original Matrix movies storyline, the reason why the machines needed to keep those troublesome humans around was not as an energy source (“batteries”) but as a source of creativity. But the writers thought that this idea was too complex so they substituted the battery idea instead.

A friend in the UK is a graphic designer; he says that over the past few months, more and more clients have been saying 'NO!' to AI-generated artwork - "it's too samey". They'd rather pay more for something original. Trouble is, AI has pushed down the rates; so while designers and artists are noting an uptick in requests for proposals, the money is much worse.

This actually is not that suprising, when you think about it: these AIs are basically using huge amounts of data to approximate averages of various things, and with more iterations they extract more and more core features until they just have the same set of features they are using all the time. It's like taking data scores and continually averaging them until you are left with one value.

I love that this underscores how complex human intelligence really is.

As someone who used to play with photocopiers as a kid... A copy of a copy of a copy is always much worse and weirder than you might think. Small flaws amplify until you all you get is a smudged blur.

This is exactly what I was telling people the other day. Our greatest danger with AI isn't that it'll take over but that at the moment we begin relying on it most, the more it will collapse because it's going to end up cannibalizing itself.

AI is geared to come up with a solution using the smallest amount of computations possible. It means that ignoring diversity and details is part of its basic make-up. It can't capture interesting quirks and details and spurt them out once in a while, in some outputs but not others. It also means that if it found one way to get to an acceptable solution, it will only try to get there quicker the next time around. If you've experienced getting stuck in a rut with ChatGPT, that's why.

Sabine, as a professional in the application of machine learning in medicine I would like to thank you for making this video! It’s understandable and it reaches a lot of people! There is the AI hype (which people should not believe because it comes from executives and rookies) and there is the machine learning reality that veterans understand. This technology will be useful in automating some drudgery and common simple tasks…. It’s dogshit at doing anything truly valuable. What’s most worrying is the very real threat, without laws, that this nonsense will create such a firehouse of bullshit that we can’t get through our email, find what we need on the web, tell the difference between fact and fiction, and generally think for ourselves!

You make a good point. Already I can usually pick the ‘style’ of AI generated images. They have a certain ‘style’ because they are in a sense too perfect, too smooth, too balanced. It is not something one could define in some cases, but the human brain is good at recognising patterns.

You reminded me of Pandora, the music recommendator that provided you with music according to the 👍 and 👎 that you gave to the songs proposed. No matter if you started with Black Sabbath, Chopin or Yunchen Lahmo. Eventually, after a couple dozen songs, you always ended up in a Coldplay loop.

That’s really interesting.. because comparing it to ‘bad cinema’, most bad cinema is bad in the same way, if that makes sense. Overused tropes, predictable storylines, cliche characterisation. Is there someone that can expand on this thought?

Short, to the point, informative. Thank you, Sabine.

My experience with chat GPT is that you can ask it 2 or 3 questions, get it to contradict itself, and when you point out the contradiction, it starts to ask if you are angry, and/or says IT'S taking a break from YOU to let you relax...😊😮😂

It's true, when you've worked enough with ChatGPT you can immediately recognize a ChatGPT text. It just always has a certain vibe that makes it distinguishable from human text.

I think Frank Herbert might have already told us the eventual solution in Dune.

Thank you for clearly articulating what many have been trying to point out.

Garbage In/Garbage Out. I've been saying the same thing about both AI and Analytics for the past decade and a half. People only want to look at processes, algorithms, ease of use, speediness, raw power, TCO, design and pretty UI with both AI and Analytics. You rarely hear people talk about things like bias, data integrity and context. Those three things only come into conversation when AI and Analytics produce horribly incorrect results.

My experiance with ChatGPT shows it to be a regurgitator, the test questions were in an area of X-ray physics that I know well and it spewed out all the usual stuff with no insight, no deep understanding, no creativity, nothing that would indicate any form of curiosity.

Doctor Frankenstein used snippets from a whole bunch of people to make his monster. I was told the experiment didn't turn out well for him either. 😂

Than you for covering this! I've been saying this since I saw the 1st interest in AI art

1) AI is trained on data from the Internet. 2) AI outputs data to the Internet. 3) Goto 1 ... haven't anybody acquainted themselves with the topic of "inbreeding" ?

There's a third option: there may be soon a deliberate attempt to poison the content to make it unreadable for AI. There are already tools out there that scramble images just enough to make them confusing for AI to use as a training set.

to me the scariest thing is how quickly people advocated against themselves the moment they realized the potential with AI

Indeed, thank you. Chris

As an artist, this is a known issue in HUMANS Thats why the art solution is to look at the "old masters" as mentors before learning how to draw from more modern artists

Adding random variation probably isn't as easy as it may sound because the randomness still has to follow certain rules. For example, no one is going to believe that elephant with two trunks.

It's not a plastic contamination, it is easy to remove all those generated images from a training data with other anti-spoofing neural networks, which have near 100% accuracy. There are no unsolvable problems, it will just require a little bit more effort on the development side.

I think we'll start to filter datasets to exclude 'low variety' data to sort of solve the problem. It'll be an arms race of NNs that generate believable outputs and NNs that are trained to detect the differences.

So basically, if you keep feeding the output back into the input, you could get a feedback loop.

This is exactly what I've been thinking since all of this exploded into popular awareness. It's like a giant ouroboros eating it's own tail. I'm glad to see that people are talking about this. Editing to add: Will you critics please lighten up? I'm not anti-AI. I'm just agreeing with Sabine that this is a potential problem that should be studied. All new technologies have potential problems that need to be studied and understood. Pointing this out does not make me some kind of neo-luddite.

Very interesting and well explained. I learned something new in a field I have no real knowledge.

Every email in the future will start with: "i hope this email finds you well" 😂

The one thing people should know about machine learning is: a machine learning trained model will only be good as its training data. It's just learning (in theory) the pattern behind the data leading to a host of problems. The main issue is that it doesn't actually reason about the data. Let's say I train a model where I have several examples where I have pi as 3.14 and then one where it's 4. The model doesn't reason "you know.... this one example seems to be wrong" but rather it updates the model to make it slightly more likely it will give the wrong answer. So how do you prevent models training on information generated by another machine learning model? The current approach is to stick to information before generative AI become dominate but most of that information (for better or worse) is probably considered or part of the training dataset. The main problem is that there's a popular opinion in machine learning (and sadly AI) that, as an AI researcher, I have had to deal with. This opinion is the key to all AI problems is that we just need to use larger models, with more training data, and train it in the "correct" way. "Look how far LLMs have come. Just imagine how much better they will be in a couple years". But you run into the 90-10 principle: 10% of the effort for 90% of the results and vice versa. It's why self-driving cars are taking a long time: there is nearly an infinite extremely rare cases that the car needs to make the right decision in. As such, it should be expected for the current LLMs to plateau performance wise unless new smarter methods are found.

Great stuff! I've highlighted this spiraling feedback loop on my blog and in one of my videos, and I'm happy to see similar thoughts here. Since the launch of ChatGPT, content creators, including science and medical communicators, have feared job loss due to generative AI. I've been more optimistic, saying that communicators with original ideas will thrive in a constantly paler and monotonous information landscape. Thanks!

After the autotune-as-an-effect started, with Cher singing "Believe", you have multiple generations of singers who mimic the sound of autotune artifacts quite naturally and spontaneously. Also, young people having plastic surgery to mimic the look of phone camera filters. We are also trainable, it turns out. I think that is an important element to consider when predicting the future of this stuff.

I think a big part of the convergence is that people often are attracted to certain tropes and conventions when it comes to what they like, so AI produced images are actively being 'pruned and purified' by our preference of our existing cultural paradigms. What I think is really interesting is the feedback, where people's tastes of which conventions they like are in turn influenced by AI art.

2) More randomnes in AI output might do away with the problem of repetitive AI output, but it might increase the mistakes. Instead of elephants with big heads or two heads, we might get elephants with two big heads.

I ♥ all your videos thanks for sharing your knowledge.

I hope it boots universities a bit to start bringing individuality back into academia. The amount of students I heard that said "whatever I'll just chatgpt it and work over the bumps" is staggering.

Wow... this is like ultra-high-speed "Groupthink"

Consider what generative AI actually is. It's designed to produce data which most closely resembles its training data. So distinctive and idiosyncratic ideas are actively selected against.

I called this a few months ago, good to see I'm not the only one who can see where this is heading.

After riding the train Sunday from up north in our capital, It occurred to me I talked to you briefly on the platform and regretted not recognising you then and telling you I appreciate your videos.

As an AI scientist, we've been talking about this for years. Once the AI starts eating its own tail it will quickly optimize to a singularity of stupidity in its own echo chamber. The only way for AI to continue to work is to automatically label all AI output and ignore it for training. Or to manually post label it by humans. Humans are necessary for AI success in any case. It would be interesting for you to discuss both the labeling servant culture and its injustices as well as the impossibility of AGI if AI depends on human labeling.

Look how people are more and more hating CGI in movies to the point that some movies refuse to do any. If you have ever read anything written by AI you know it has the ability to make the most exciting subjects boring.

I’m not surprised by the finding and I dare say that it is obvious. I dare say that because such a thought has crossed my mind and I have often tried to convince others who are willing to listen beyond the hype; AI is “artificial” but not “intelligent”. There are different ways to assess and critique AI - philosophically, linguistically, psychologically, biologically, etc, which many thoughtful experts, beyond the industry, have challenged the claims of AI, in particular AGI - but, for me, it comes down to creativity and novelty, the latter AI lacks completely and the former AI can only mimic. If you want to be impressed, see a human child.

Thank you for the video! I am not an expert on AI just used to work with big data. It has never occured to me that AI collapse due to low quality of input might be an issue, but it actually makes sense. I would like to point out that adding more randomness might now be even viable solution to this problem since all the existing random number generator are in fact pseudo-random (the random result can be predicted if you know the algorithm) Guess we will see how it will play out in the future

I recognized this as a possible problem when I learned that they were training AI by allowing them to converse with people on Reddit. AI developers can now apparently pay a fee to be allowed to plug their AI into Reddit and have it learn by having conversations there. It occurred to me that, "wait a minute, wont then the AI's end up conversing with each other and training each other??? Won't this cause problems?"

The popular image generation models prior to Stable diffusion were GANs (generative adversarial networks). The way they worked was to have two different networks - one trained to generate images, and the other trained to classify images as real or fake. This forced the generator to learn to avoid the most identifiable characteristics and to generate a diverse set of images. Stable diffusion was more effective and scaleable for higher resolution images, keeping the whole image globally coherent. But it's likely that reviving some adversarial techniques could help with the diversity issue.

So basically ai need a human brain. Take note matrix.

You remind me of candlemakers and how they responded to the light bulb.

That plastic analogy is probably the most succinct depiction of AI generated content contaminating the environment and why I always thought that human intervention in the use of computers is always necessary. We can fake human thinking to a degree, but getting the full complexity is still a pipe dream.

Great presentation, Sabine. I have always maintained that AI is like students cribbing exam answers. One student just has to copy one thing wrong, once. From then on it is a done disaster. When scientists hypothesised robots making copies of themselves - they never saw this far into the mess.

Any analysis on AI I watch always brings to mind the computer character from the motion picture, 2001: A Space Odyssey, Hall 9000. The characters is something else.

I've been drawing attention to iterative decay of information in these systems for some time. Eventually we will just end up with a 'grey goo' situation similar to that suggested for uncontrolled nanobot replication.

Surprised we arent already forcing watermarks on ai content. Actually blown away. Like giving a kid a staw house and fireworks and not expecting a fire😊

AI can learn from itself when there is an objective outcome to measure. For example Chess, Go and Poker AI engines can improve by playing against themselves (though they also benefit from historical game records and playing against humans). Where there is no objective measure, such as art or creative writing, it's difficult to see how AIs can improve without human input.

Didn't even think of AI Art cannibalism. That is such an interesting thought-experiment. Thanks for the video and I'l be back for more!

Never heard this before ! That's really interesting. I'm sure we'll hear about it, that's sounding like a nightmare. Thanks

A third potential is that we decide to move on from brute forcing LLMs to work and get more efficient or different learning models. A human does not need to look at a billion images to learn how to draw. Even if we don't have AI that are capable of what we can do, it does demonstrate that there are better ways to design AI. Right now it's kinda brute forcing and incredibly inefficient

I think that if we program in randomness, they'll introduce wacky and impossible and obvious problem elements. The problem is in explaining how to adjust and add randomness to a program which doesn't understand the original state and how it has simplified and made things uniform. It doesn't understand what it does, so how can it introduce some oomph. How can it know when it's introduced too much? There are way to many parameters which can be tweaked.

Some sort of entropy scoring or min KL divergence threshold from the model's distribution at which an example could be discarded during training could work here. Basically, I think we will figure this out

Great video. This problem may be a teething problem, though. After the computers started beating the top human Chess players, Go players like myself felt smug. We said things like "Chess is about crunching through possibilities. Go requires real intelligence." For years we annoyed chess players with this sort of talk, citing the inability of computers to beat the top human Go players as proof. This lasted about 18 years until AlphaGo came along in 2016. Important point: AlphaGo does not play like humans, except faster and more accurately. It came up with some genuinely novel moves. So, I would not take any problems that AI is now experiencing as an accurate predictor of what AI will be like in another two decades or so.

This issue has occurred to me for quite a while. I have worked with Big Data extensively and had brief real world experience with AI development. AI’s reuse of AI-generated data seems highly likely to result in a “creativity asymptote.”

I like the comparison with pollution by micro plastics. Furthermore, some images you showed made me think "AI, the copy cat!"

So apparently, echo chambers create echos...

There’s a chaos setting in Midjourney. I imagine that modifier will be available in everything. It works well for randomness although the weirdness is a bit much

I kind of wonder if this problem actually started with the widespread use of the internet. We went from communicating with books, which had to meet a certain standard before the expense of publishing could be justified, to zero-cost sharing of opinions on the internet, to having machines lap up these opinions and feed them back to us. Each of the above steps involving less rigour than that which precedes it.

its already happening on youtube. the same videos with different thumbnails usually one or two words are changed

The problem is oldest than your video suggests. Automatic translation and even grammar correctors have been distorting human language (and reducing language variance) for decades, and we have incorporated their language quirks in our day to day language.

I found the same thing when a i was playing around with a music ai generation app. While it was fairly impressive at what it could do when i tried to get it specfic content to fit different needs and moods it wasnt able to, the music sounded too different even when using similar prompts

Very interesting findings there and suggest that while initial models create biases, refined models may also create average bias.

Please, would you put under the video the references of the sources you use? A study from Japan, another from France - please, give us the links! Thank you.

Thanks! you are one of the best things about the internet!

I think that this might push AI creators to start tagging their creations as AI, and make them ignore creations with those tags. This could be a good process as then fake images could more easily be identified.

This reminds me of something happened in bycicle industry, sales were declining so "creatively" they "invented" 29" wheel and stopped producing 26" .

Worst case scenario, it'll be like Walmart moving in to a community, destroying the mainstreet businesses, everyone takes Wal-jobs, and then the Walmart closes because the local economy is crap simply leaving a ghost town with people who don't have the capital, business acumen or consumer base to reboot mainstreet.

I think within language sets a reward for rarity might help create a more organic system of bringing up "antiquated" usage of words. Not that they are literally antiquated, but that within the AI model they became antiquated.

I was just called by an AI telemarketer. I am sick & this was reflected in my voice when I answered the phone. The AI tried to relate to me by matching the quality of my voice. It sounded like a drill sergeant or concentration camp guard mocking me.

As a 80-90s kid, I just wanted to say your thumbnail looks like artwork for a gameboy game with that left stripe.

Exactly! I've been wondering about the potential for this problem. Thanks for pointing it out.

It’s done everything and more than I ever expected it to be able to do. I now can’t function without it incorporated into every facet of my existence.

I have said this from the start, people forget that AI is just fancy web scrappers. Why on earth would content creators continue building websites if people are going to by-pass search engines and go straight to AI, they will not get visitors so no advertising revenue. So since no information is going to be published online where is AI going to scrap their data from? Will become stale in 5-10 years. If I was a big recipe site, fishing site, car site I would be suing AI makers, no one gave them permission to scrap their data, is their even a tag we can put in that forbids AI from accessing our sites? We can block Googlebot and other search engines from accessing out data if we want, AI?

Unfortunately it will only be a matter of time and they will either get better with the parameters that introduce random variables that simulate creativity or they will achieve singularity which they are moving extremely fast towards. I believe we just have to keep creating original works of art. Ai can never replace that. Draw and paint more originals and let Ai exist for the people that don’t want the joy and fulfillment that true creativity brings them.

I’ve been saying this to friends for about a year now. I’m glad I’ve finally run into an expert identifying and addressing this potential problem.

I absolutely love reading comments & knowledge shared by people. Seems like a computer or program doesn't really know the world, discernment. Still pretty interesting & useful.

This could be the next great art movement - humans creating such bizarre, original and abstract art that AI can never touch it.

Great video!

I really like this take. Thank you for all the information.

I wonder if the generated elephants look so similar because usually generated images try to match the sample sizes (512x512, 1024x1024) which only leaves so much room for good compositions and wonder if in future with larger models we might see this change a bit more.

HI would it be possible to do a round up of the current peer reviewed material regarding the spike protein of corona-virus. I find it hugely difficult to approach the material because of the vested interests and difficulty of appraising academic papers. as an alternative a round up of how to approach such an issue in terms of assessing the material including how to screen papers and thus establish credibility etc

I think a lot of this comes down to trying towards desired outputs and then giving vague prompts. The model will give an output based on the way the input interacts with the weights. In a sense, these weights would be reflections of the average desirable output - no? So, within this paradigm - it would naturally fall on the person prompting to create a set of parameters that make the output more interesting. Having the AI operate as more of an agent that takes iterative steps to make something that is also unique could also be a solution. Am I missing the main issue?