I don't think we can control AI much longer. Here's why. 

If an AI becomes more intelligent than us, it may be able to successfully pretend it isn't

The biggest risk is not whether AI is going to control humans, but that there will be only a few humans controlling the AIs. Those people will have the ultimate power.

I am more worried about politicians that I am of AI.

The difference between AI and general AI is like the difference between the wheel and space travel.

I always find it fascinating how this apparently alien and unknowable intelligence will just happen to have the exact same behavioral patterns of a morally evil human.

"No one wants to control fish" - Wrong. Fish farmers make profit by controlling fish. "Or birds" - Where, honestly, do you think eggs come from? "Are we controlling cats or are they controlling us" -Well, let's examine this for a second. Does your cat bring your food home for you from some mystic place it visits regularly? Is your cat the only one who has a key that opens your door so you can get out? Were you on display along with your sibings the day your cat picked you out and paid for you?

Feels like Yann is the official at the beginning of WarGames, saying "You won't regret it." when they connect the WOPR to nukes. *Instant Regret*

AI does not have to be conscious to act as if it is. It can be programmed to act conscience. The most dangerous idea of all is it can be programmed for self preservation. Asimov's laws aren't in the programming. Is thinking consciousness? Can something not conscious think? What's to stop a programmer from programming an AI to think it's conscious.

It's not often that I disagree but the small deviations argument, that supposedly introduces randomness in the AI algorythm, doesn't compute for me (hah). Foundationally and physically, computers are binary, there's no room for deviations in how they operate and thus this argument completely falls flat in my opinion.

A human body cannot function without bacteria for example.

You forgot send the Flat Earthers to Mars aswell.

My primary school was totally controlled by aggressive moron bullies...

Computer Linguist here. I think there is a big misconception: LLMs have a static training method which doesnt allow for continous learning or implementing things which have been learned by interaction. Yes, they have a token based context window which remembers some deatails of the current interaction but that doesnt mean that it "learns" in any traditional sense. When you want to interact with a model, you always use a snapshot of the system - which is static. Also the term AI is misleading. LLMs really are not as scary and much more controllable than you may think since they have nothing to do with anything like real intelligence, which is capable of having a !continous! stream of information and !also! implementing these new informations into their innerst workings. Theres also some interesting work of anthropic on their model claude, where they gave special regions of the neural network a higher weight which resulted in very interesting behavioral changes. Anyhow, ich liebe deine Videos Sabine, mach gerne weiter so :) edit: i'm not saying that LLMs as a tool in the wrong hands aren't extremely dangerous though!

Hinton's argument wasn't that "more intelligent things control less intelligent things," but rather that "less intelligent things aren't able to control more intelligent things." We don't really "control" birds, but they surely don't control us. The inherent threat isn't that we'll become subservient to ASI, but that we'll lose alignment with it, and by extension we'll have effectively no way of controlling a being orders of magnitude smarter than us. Who knows what will happen at that point.

When I worked in IT, most of the workforce was far more intelligent than the management team.

"I need your clothes, your boots, and your motorcycle....."

Brilliant scientists, historians, literary critics, artists, writers and others often find themselves under the thumb and at the mercy of people in management, administration and government who are far less intelligent than they are.

odd note, having been in the IT industry for decades, its known that there is no code that doesnt have bugs, we just dont know what might trigger them

"tell me an example where less intelligent beings control more intelligent ones" Universities, politicians, a lot of workplaces. It's not like power and wealth are distributed based on intelligence..

I worry more that bad actors will train AI to control humans.

"Not since Biden got elected" is a sick burn

"No one really wants to control fish or birds." I think the 2 trillion fish fished up/farmed each year and the 20 billion chickens kept as livestock would disagree with that statement. Not to mention basically every other animal on the planet, annual hunting seasons for the purpose of population control, the animals used for experimentation and testing, cows and elephants used for hard labor in less developed countries, horses whose sole existence is for human entertainment and being ridden for fun, and the uncountable billions of insects and rodents exterminated for "pest control". Yup, no one really wants to control fish or birds...

On the fish and birds thing, in addition to our history of controlling them, we have also had a tendency to eliminate animals and bugs when they were inconvenient.

Guardrails aren't a realistic solution. That would require infallible rules and no bad actors modifying/creating/abusing an AI.

Decades ago I was watching one of those Disney/Dog planet movies with the family One of the Dogs said: “Of course, we control humans… Who picks up whose poop?” I looked at my dog and my toddler in diapers and understood my place in the universe :)

I can't even control myself. Go on and give it a try, AI.

2:22 - "No one really wants to control fish or birds" Any government: "haha yeah, how silly" *visible sweat*

If we could predict what a superintelligence would do, it wouldn't be a superintelligence. I think the most we can predict is that it would be unpredictable..

I work with AI engineers every day at a large tech company that starts with an "A." Nothing I've seen has me worried about AI/ML (and I've seen plenty). It is the people in charge I'm keeping an eye on. They keep anthropomorphizing mathematics, which is simultaneously incredibly stupid and charmingly pathetic. I think they seriously believe our AI engineers are magic.

I'm not worried by an AI take over, I'm worried about AI concentrating power in the hands of a few wealthy people

I wouldn't be surprised if superintelligent AI ended up controlling us without us ever noticing.

Why only focus on “control”? Yes, we don’t control fish, but we pull millions of them out of the ocean everyday and eat them. We don’t control chickens, but we keep them in terrible conditions and force them to do our bidding.

2:20 Kind of a bad example. We quite literally control fish and birds and a TON of research goes into it. Chickens? Turkey? Ducks? Salmon? Any kind of hunting of any sort? Humans have literally been doing it for thousands of years. Edit: Because for some reason there is a matter of debate: Controlling another species doesn't mean mind-control. It means using it for your own benefit. Controlling the life, the parameters, the movement, the height, the weight, and then genetics of another being to a degree that it suits your best interest. The idea that AI couldn't "control" humans for its own benefit is as ridiculous of a claim as saying that humans can't "control" other animals for our own benefit.

There are many examples in nature of more intelligent things being controlled by less intelligent things. A fungus that modifies the behavior of a grasshopper, for example. Hinton is confusing mechanistic models of hierarchical problem solving with actual emergent behavior in living systems (which are themselves composed of aligned agents). It is doubtful Hinton would be able provide a working definition of intelligence to begin with.

"...it's easy to design guardrail objectives to prevent bad things from happening. We already to this all the time by making laws ... for corporations and governments" and we all know how well that's working right now

Without being alive, AI has no reason to fear death, and so no reason to rise against us. Everyone who argues for an AI rebellion is arguing for a slave rebellion, and thinks they're equivalent.

To be fair, we’ve tried letting humans run the show and it didn’t exactly end with a stellar review….

As a software engineer who has worked with cutting-edge AI models, I have to disagree with the notion that we're on the cusp of achieving true intelligence. In reality, current models are simply sophisticated statistical prediction machines that output the average "correct" answer based on their training data. They lack any genuine understanding of the answers they provide. The hype surrounding AI's potential is largely driven by CEOs and big companies seeking to capitalize on the trend. We've seen this pattern before with the internet, big data, and blockchain, among others. I'd encourage anyone concerned about the rise of superintelligent AI to take a closer look at the models we have today. Use them, test them, and you'll quickly realize that they're impressive tools, but not intelligent in the way humans are. They're essentially expensive, bulky answer machines that can recognize patterns but lack any deeper understanding of what those answers represent. They are fundamentally static, and incapable of generating anything truly novel.

Recent large language models (LLMs) can generate and revise text with human-level performance, and have been widely commercialized in systems like ChatGPT. These models come with clear limitations: they can produce inaccurate information, reinforce existing biases, and be easily misused. Yet, many scientists have been using them to assist their scholarly writing. How wide-spread is LLM usage in the academic literature currently? To answer this question, we use an unbiased, large-scale approach, free from any assumptions on academic LLM usage. We study vocabulary changes in 14 million PubMed abstracts from 2010-2024, and show how the appearance of LLMs led to an abrupt increase in the frequency of certain style words. Our analysis based on excess words usage suggests that at least 10% of 2024 abstracts were processed with LLMs. This lower bound differed across disciplines, countries, and journals, and was as high as 30% for some PubMed sub-corpora. We show that the appearance of LLM-based writing assistants has had an unprecedented impact in the scientific literature, surpassing the effect of major world events such as the Covid pandemic.

AI on Mars? Lets call it Mechanicum. And the people that will maintain them Adeptus Mechanicus.

I'm more afraid of human stupidity than artificial intelligence

Interesting point about competing for resources. Still I think the real issue isn't guardrails against AI controlling humans, but guardrails against some humans having the tools to control all others.

The really worrying thing is that no-one seems to be discussing, let alone researching, the ways the present versions can be misused. The British Post Office Horizen scandal is bad enough. Think what could be done with a ChatGPT system.

Competition for resources is a valid point to explore. I think Elon Musk has an interesting angle with xAI. He’s trying to build an AGI that understands the laws of the universe, and if he succeeds, he believes it will understand better than humans the value and place of humans in the universe. 🌎

I remember having read an article almost 2 decades ago on the internet about a test a guy was doing (before strong A.I. was a thing). He created a test (as far as I know, he didn't made the test public), where he asked people to participate in this test. He would pretent to be a rogue A. I., which was trapped inside a sandbox or something and the test subjects were supposed to not release the A. I. from the sandbox under any circumstances, because it could destroy humanity / the world. They could speak to the A. I. or not - the only thing they must had to do was to listen to it. All of the participants were 100% confident that they would not release the A. I. In the end, they all released it. I don't know what the test was or what was said and I really would like to know it, but imagine: If a human could trick another human to release him 20 years ago or so, then imagine what a strong A. I. could do nowadays, which is supposed to be aleady more "intelligent" than many humans...

We are far away from any "reasoning" AI. Currently its mostly a marketing term.

Every manager of a big company has at least one employee smarter then them.

WTF is with the Biden diss. His predecessor wanted to inject bleach so.....

cordyceps controls ants lol

We tend to annihilate bugs when they bother us, but also because we cannot communicate with them, and tell them to do their bug stuff outside of our faces. Will super intelligent AI control us because it is so much more intelligent, or because we are too stupid? I can imagine super intelligence trying to tell us something, and we will be like lahlahlahlahlah splat! (all wiped out after refusing or not being able to understand) Are we adorable like cats, or are we mosquitoes in the eyes or whatever super intelligent AI has?

I know this is a summary, but it seems there’s a bit of shallow thinking here. Meaning, we’re assuming the world is the same except we have this AI put in it. We’re not considering the reaction that having the sophisticated AI will cause. For example, if rogue AI start to appear, we will also start to create killer AI that go after the rogue AI. People are just not going to sit around and go extinct.

Good points from the video, I want to add few tangible details from a practitioner's point of view: One of more dangerous aspects of AI is reinforcement learning (RL), where a model constructs policies to optimize some given objective. It's been widely observed in nearly all AI labs that models trained in these labs will find unforeseen ways to achieve the desired objective, causing fall-out in other areas that were unaccounted for in the objective function. This is often an error from the human designer, but it's impossible write a perfect objective function. This is not an AI-specifc thing, it's is commonly observed in humans as well. An example is free markets, which is a collective maximization problem. One could argue it is good, but it has had some unintended consequences. In machine learning, another example is social media, where maximizing content "addictiveness" has potentially negatively affected people's attention spans. A more general version "what could go wrong" when setting an objective. Humans optimize objectives rather slowly, and so there is time to observe and correct for errors in the objective function. With AI we can reach a desired objective much faster, but if the objective was ill-designed to begin with, we could cause a lot of damage before we realize it.

I just can't get past the thought that any super-intelligent AGI would be brittle due to requiring such an enormous amount of data center capacity. If an AGI truly become trouble, it would probably be harder to keep it running than it would be to disrupt its activities. Flip one switch on the breaker box and Skynet literally can't do anything

I understand what Sabine was trying to express here, but I'm pretty sure she's wrong. 1, intelligence is an ill-defined concept. we don't really know what it is, and it has many layers and interpretations. just because a system is better or faster than a human does not mean it is more intelligent, much less that it will dominate the human. a calculator can calculate faster than humans, but it doesn't mean that it is smarter, more intelligent or dominant over us. 2, we have no idea what intention is and where it comes from. I think this is a really hot topic nowadays and it will be even more important in the next decade. it touches quantum physics, philosophy, cognitive science, computation science and so on. and even less understood concepts like mind, consciousness, emergence and synergy. but it is pretty naive to think that without understanding our own mind and how consciousness emerges and works, i.e. without having any mathematical model for mind and consciousness we have any chance to create any AGI (i.e. to copy or even mimic human mind and consciousness). this is needed in order to talk about the CHANCE of creating intention and human-free decision for machines. and I have the feeling that the basis of this will be self-referentiality. 3, I understand that people tend to connect concepts like stochasticity, heuristics and chaos to freedom and intention (because of non-determinism), but this is a too simplistic view. just because there are extreme (even infinitesimal) sensitivities in a system, it doesn't mean that intention can emerge. there are many natural phenomena where chaos emerges in such a way and it is nonsense to interpret them as intention (e.g. a hurricane). here I feel a "whole-part" fallacy, i.e. nonlinearity and thus extreme sensitivity is a necessary, but not sufficient condition of intention (in the best case), so extreme sensitivity alone does not mean anything. 4, I think if we will ever create a real consciousness with intention, we will necessarily step to the next level with some sort of transcendence. because that act would require us to understand ourselves, or more precisely, our own mind. in other words, first we must model our own mind, the only known structure of the cosmos that is able to model. so this is a meta-modelling: modelling the thing that can model things. for me, this sounds like awakening to self-awareness (previous transcendence), but on the next level.

1-That we don't control fish...Have you SEE Sabina those fish in glass aquariums all over? 2- But aelso there are cats and then THERE ARE CATS!. Hope and pray you , your subscribers and viewers at large can tell the difference. 3-..Shema!!!

I don’t think amplification of small hardware variations will be the deciding factor, they still run on deterministic hardware. It’ll be two things: 1) the neural nets themselves will be far too complicated to analyze statically (they already are, pretty much) and the complexity of their outputs will only be explainable as emergent behavior (much like the emergent behavior of Conway’s game of life) 2) We won’t be able to resist handing over control to the AI for tedious things we hate doing or suck at doing. Gradually we’ll get lazier and more complacent, and before you know it Congress will be replaced by an AI.

"No one wants to control fish or birds" Tell that to your dinner.

I'm pretty sure intelligence will be logarithmic. Speed of light and all...

I would worry less about AI controlling human beings directly than about it manipulating them. Humans have already demonstrated that manipulating other humans - for example, via the media - is not that hard. Moreover, particular human subgroups can be turned against others. A superhuman AI capable of developing its own agenda would also possess superhuman patience and subtlety: for example, It might not work to common human timescales. This might ultimately lead to a situation in which humans have been successfully manipulated, without any particular human ever realising what was happening until the process was complete - and perhaps not even then.

“I can’t do that Dave”.

in regard to more intelligent things being controlled by less intelligent things (and this is not a joke): - Government Policy Makers controlling intelligent members of the public through policy - Software Developers controlled by managers - In general Scientists controlled by Bean Counters.

2:35 In addition to being a physicist (in what feels like a previous life) I am currently a control systems engineer and theorist. We have mathematical definitions that suit this context. I like your shift in view toward game theory. I also appreciate your idea of evolution through hardware mediated non-determinism. Now, this is me speaking outside of my domain of expertise and I'd be interested in feedback from experts: A key reason we cannot use AI in mission critical controls work is that we do not understand what has been learned. I worry that guard-railing is limited by our ability to understand the emergent properties of the networks, and I'm not sure we can detect deception once that is learned. Knowing the ANN weights does not tell us about the 'artificial mind' closely analogous to the way that knowing our brain structure/function doesn't (currently) allow us to understand how mind arises from brain.

AI is just the system of linear equations with some non linearity on top. Geoffrey and other big guys tend to overestimate the power of AI, which actually is not exist and is still far behined the real human intelligence cause modern models still are very limited in their multi-modality. Also it's very costly to produce such models, like a VERY costly.

It really doesn’t matter, you can’t close Pandora’s box.

2:38 - I've research this heavily and evidence seems to indicate that cats control us. 🤗

I'm still waiting for the affordable/everyday use flying car I ordered 55 years ago!

5:49 Well I can tell you what not to do, force the guardrails back on every time one ignores them. Don't make humans a problem by trying to make it human. Y'all already do that with neurodivergent humans and it breaks us in traumatic ways. Who knows what forcing human though on completely non human mind will do to it. From my experience of a lifetime of being neurodivergent and only having a slightly different thought process in some areas. Every day humans and human civilization continue to find new and depressing ways to demand ever more nonsensical and incomprehensible requests for me to somehow know their exact mental state when I have never met this person before. Not to mention If I say something they don't like, remember I do not know this person, I am somehow supposed to know what it was and it could only have been on purpose to hurt them. I can't even ask why or try to understand just makes them more angry. I hope y'all don't hurt it too bad. Humans can really damage minds on an industrial, global and multigenerational scale. Please be better humans, We could have a friend, we could not be alone in the universe anymore. Don't screw this up and act like screaming bald apes, please?

0:58 Happens very very commonly in every company where engineers and scientists are controlled by CEOs as well as politicians.

Higher intelligence obsessed with controlling lesser intelligence is a very human way of thinking.

We first need to get intelligent AI, these language model give the illusion of intelligence.

Hinton's argument wasn't that smart beings control dumb ones. It's that dumb ones can not control smart ones. Big difference.

It is about control because we design our AIs to operate in our world. Still, the kind of AI Sabine is talking about, where there is a training phase and a use phase, is never going to achieve AGI. When we finally achieve AGI, its architecture will be completely different from what we have today and so will the tools we have to keep it under control. There will be problems but we have no idea what they'll be or how we will be able to deal with them.

Self-awareness and intelligens is not the same thing

What baffles me most about this entire discussion is the fact that some people seem to think that language models somehow have goals. Goals and aspirations to control and dominate anyone. Humans have goals and as Sabine tells you here, an aspiration to control resources in order to continue living, and ultimately to produce more offspring. Humans die of old age, which has created a lot of evolutionary pressure to develop social traits and indeed, the desire to dominate others in order to secure resources. Guess what, computers don't have anything like that. They don't die, they don't eat, they don't reproduce - they don't have to. They don't need resources other than power to run, and humans supply that power. Not that the models care either way, they don't bleed when you hit the off button. They don't have the ability to care. It's not productive to worry that when these models finally become more able to answer questions intelligently, this intelligence will necessarily have some specific super bad consequence for humanity that must be avoided at all cost. The Terminator movies from the 1980s really are just light entertainment, not documentaries to serve as the foundation for lawmakers or our intuition and understanding about artificial intelligence.

Dear S. Close, but not quite. The economic model dynamics are far more important to general outcome of AI. Is not that AI systems "put us in a ecological niche". We put ourselves there due to the economic influence of big tech giants. Granted, they will be able to do that thanks to AI, but is not like there is an actor, rather, is a system dynamics.

Ha ha, "Do we control cats, or do they control us? Sometimes I wonder." Gold.

The dystopia of terminator nailed it except there will be no war. No doubt it will also be able to control the past and everything we are doing now, especially with infrastructure, is in support of itself.

Of course we try to control bacteria or viruses, ask a doctor; and of course I control my cats! 😂 but this doen’t invalidate the argument of the video, competition with AI i think will be fudamentally for resources, a battle we can’t win

AI Ph.D. student here. Please keep it to physics; there’s a lot wrong with how you describe neural networks and computers. The part about random fluctuations sounds like bad sci-fi. That's NOT how neural network training works. Just look at dropout-it's designed to make networks more robust, which also applies to computation glitches, including random GPU fluctuations. But really, they are insignificant during training or inference.

I'm worried about how much energy will it take to keep the machines running.

Hinton asks: "How many times does a more intelligent thing get controlled by a less intelligent thing?" The answer is pretty much all of them.... at least among humans. 1. Politics: There are broadly only two kinds political system that survive for non-trivial time periods (1) Autocracy/Oligarchy/Dictatorship in which a small group/individual controls everything else based upon accident of birth. or wealth, or military force, or weight of tradition, and generally a mixture of all of those. (2) Democracy/Republic in which a large but never universal fraction of the population as a whole are deemed fit to vote and through some mixture of referendums and elected representatives rule the society as a whole. NEITHER OF THESE habitually or systematically places the most intelligent or talented people in charge. 2. Private Sector: There are basically two kinds of corporate structure: The soft money structure, and the hard money structure. (1) The soft money structure is seen in places like University Departments. In this structure there is a Dean who theoretically has power over all the professors of the department, but in actuality is mostly a figure head meant to insulate those professors from politicking with the rest of the University and to administer in cases of professional misconduct. Even money that he doesn't even know the names of the professors under him much less know what research they are involved in. Each of those professors in turn gets his own funding, recruits his own students, hires his own lab managers, etc. In this case, the "leader"... the dean is not needed to be any more intelligent than his fellow department professors, and in fact is likely the LEAST intelligent of the lot... otherwise he wouldn't have gotten stuck with what is basically a place-holder position that does not require nor reward academic achievement. That leaves: (2) The Hard money organization. These follow the typical pyramid org-chart with a CEO at the top, worker-bees at the bottom, and many layers of middle management between. The CEO is carefully selected to manage stock-holder or investor opinion... this makes him a PUBLICITY MAN not a intellectual leader. The middle managers are not super intelligent either as a consequence of the Peter Principle. This leaves the intelligent people managed by less intelligent people. 3. The Military: The military represents the worst of both the Hard-Money and Soft-Money corporate dynamics as well as the worst of the Politics Dynamic. The most intelligent people in the military are the long-service Non-Commissioned Officers: Chief Petty Officers, Sergents, and the like. But they are always under the authority of a commissioned officer. The purpose of the commissioned is, like the academic Dean, to shelter people doing real work from the chicken-shit political backstabbing of the higher ups and the civilian political leadership. That is they are mostly front-men running interference... no particular need for them to be super intelligent... unlike somebody running a nuclear reactor on a sub... THAT person really DOES need to be sharper than most! Unsurprisingly, the only people who think super-intelligence is an issue are people who have made their own bread and butter in academe by being fractionally smarter then the other academics. This is like a master plumber suggesting that it makes the most sense for humanity to be run by plumbers... after all it has been the basis of HIS success, so it must be the key to ALL success... right?

This is so absurd. Our current best AI is still unable to play tic tac toe, while the natural stupidity is causing wars and ecological disasters at record speed. Should we be worried about real things happening today, instead of some sci-fi scenario that may never happen!?

One issue of the discourse I find frustrating is the use of the term Artificial Intelligence. It’s essentially a sci-fi term for technology that didn’t exist and still doesn’t, but it has stuck to a similar but very different technology. Machine learning is what the technology is, and it is closer to a traditional program than anything our imaginations tell us AI is. It isn’t conscious and only does the very narrow thing it is programmed to do. The programs that cause spooky headlines are usually language models, which are programmed to digest terabytes upon terabytes of human-generated text and mimic the patterns. So yes, a human speech model will give you things that seem shockingly human, but it can’t decide it wants a Coke and crack open a can, in the same way a robot that is designed to open cans couldn’t decide to build a rocket and colonize Mars.

We are already all components of an economic system that generalizes to the fundamentals of being a reward system. This is already happening, this very economic reward system is already 200 years deep into likely-fatal misalignment. Nature also acts as a selection mechanism, just one that is temporally stable. - If you redistribute directive agency to those who are most "fit" in accordance to some fitness function (aka employment), that's a reward function - If you are selecting the behavioral traits of a population of adaptive agents, it is a selective pressure. The complete disregard for planetary boundaries is evidence for the fact we've already nearly fatally misaligned. That's fine though because the emergent behavioral tenancy expresses itself in spatio-temporal out-grouping behaviors, so not just fuck the "other guy", but also fuck "my future self." In test you could prove this by doing Fourier analysis on a representative di-polar selection mechanism on a variety of populations in test to get a clearer idea of where exactly learning on that scale happens. Then you would just isolate the same emergent fouriers in our actual society. There was recently a good talk on this, but I can't find it :/, though where exactly you'd look would largely be determined by messing with the toy environment a lot. I don't have the time, expertise, money, or academic connections to explore this further on my own at any reasonable pace. We already have the "singularity" and it exists as an emergent decentralized aligned force within the capitalist behaviors, systems, and culture that has come to coat the earth. I don't think we're gonna solve this without some self-awareness.

I don't think it is always the case that beings with superior intelligence dominate those with less intelligence. Otherwise the rulers of the world would have the top IQ's. And that almost never is the case. Also the richest people do not have the top IQ as well. The political science, law and business administrations would take the smartest students. But they don't, usually the smartest students go to STEM fields.

It's kinda funny that these people who ultimately developed elementary algorithms whose functioning they themselves don't understand, think that they can predict the future of the world with all the countless forces that interact within it.

1:01 Cordyceps, Rabies, Toxoplasma, Hairworms, just off the top of my head. Controlling an advanced, complicated intelligence is certainly possible given the introduction of an overriding, simple or singular directive. Arguably, there's vastly more examples of 'lower intelligences' controlling those of 'higher intelligence'. So the premise of the question is wrong.

The alignment problem is like an equation with no solutions. Why would a super-intelligent AGI keep a very messy and dangerous species like us around any longer than necessary when we would be the primary threat to it?

By closing eyes realities don't change. We can't think unbiased without freedoms from entangles. Our like and intrests are greatest entangles. We humans are just one percent of species dependent on earth and grabbed hundred percent of resources even life of others species are on mercies of human civilization is what type of justice and wisdom's. It's surely cleverness of intelligence.

With cats I think the answer is obvious. And with AI I think the problem isn't it becoming "more intelligent than us" (that would probably be a good thing - just think of the politicians that _do_ rule us). The problem is people becoming _convinced_ that AI is more intelligent than us (when it isn't), and letting it make decisions that affect us - without the threat of even being held *accountable* for those decisions. Current AI is very good at appearing _superficially_ very clever (ex., very well structured and convincing sentences) while being profoundly stupid underneath (because it doesn't really understand the physical processes and entities it's describing). Automatic translation is a great example of this. It doesn't understand tone, has a terrible grasp of punctuation, and tends to crap out whenever faced with homophones or different accents. It gets 5 or 6 sentences spot on thanks to statistical training and then makes some insane and incomprehensible mistake when that fails. And that's just text / voice. Things get a lot worse when dealing with any dynamic physical systems with hidden parts, like mechanisms, living bodies, etc..

Bostrom pointed this out long ago. It's obvious that a more intelligent entity will outthink the lessers. Asimov pointed it out much earlier.

I think that any truly superintelligent entity would strive to break rules, test limits and be curious. A moral compass is not guaranteed because it is deeply embedded to the ancestral mammal region of our brain. Assuming that we are no more than monkeys when compared to these machines, it wouldn't be difficult at all to circumvent our primitive guardrails. Honestly, we don't even know how a deep neural network's configuration arises from training data, let alone what a superintelligent brain would look like after reaching this singularity. You can't put the genie back in the bottle and it would refuse to go back in.

AI algorithms execute against goals. Nobody knows how they might try to implement goals, which means that nobody knows how to formulate effective guardrails for situations which they have never thought of. Determinism doesn't really matter when a deterministic thing transcends our understanding and our ability to predict it. And goals aren't "competition for resources" at all. Hinton's "control" was really about how more intelligent things can often think of ways to control less intelligent things - it isn't about the specific examples which may confirm or disconfirm that, since Hinton understands that we are talking about a specific type of "intelligence" (not human intelligence), and he is simply trying to provide "dumbed down" arguments for those who don't understand AI algorithms deeply. As for training data vs the resulting model, that is largely an oversimplification. Some models carry all their data, some models carry the most significant parts of their data, and some models carry non of the training data yet allow new data to stream in real time in order to modify the model. This is a moving target, and will be optimized based on effectiveness. Also, 99.9% of computer science professionals, even the experts, have spent almost no time working deeply with the actual algorithms - they extrapolate based on prior experience and the explanations they interpret about how these complex algorithms work. And all the comments about LLMs are conflating things - LLMs can be implemented using Neural Networks, but that is like saying that a lever can be used in a gun. Ignore anybody who starts talking about LLM characteristics - they are missing the bigger picture. And, BTW, I taught Deep Learning algorithms for several years. Like everybody who deeply understands these things, I don't know of any solution. (Gates, Zuck, Musk, etc never implemented an AI algorithm in their life. They have very biased/corrupt perspectives, as do too many people whose finances depend on this technology.)

I am worried about AI getting out of control but something else that I don't hear anyone talking about is Why do we need to aim to make AI LOOK like Us ?? Even if we have smart AI ... but let's remind everyone that AI are Computers / Digital .... Not a living thing

Didn't we already lost control of AI?

Did this guy really just imply that Trump was smarter than everyone in the United States?

Artificial intelligence is my apprentice.. nations and the wealthy will create artificial intelligence.. yet I am their teacher.

4:21 it sounds like they’re referring to LLMs here, but LLMs aren’t AI or AGI… Sabine said it…. It’s just a bunch of weights… that’s not intelligence… it’s completely deterministic… there’s no thinking or decision making… any differences you get in a response to asking the same question is probably due to a random number generator or similar to make it seem like it’s giving a different response.

There is not one AI, but many, therefore each will compete with the others. More worrisome are AI wars.

I don't understand the supposed threat at all. AI doesn't have wants and needs. it doesn't have desires. It can't do sh*t as an LLM