224. Superintelligence Assumptions 

Coincidentally, this is quite similar to how I feel about Bostrom's simulation argument---that the alleged 'inevitability' of hyper-complex simulated universes and minds which would quickly outnumber the real universe and minds . . . boldly assumes the _feasibility_ of hyper-complex simulated universes and minds. He treats us to this elegant statement that skips past the assumption: "later generations [. . .] computers would be so powerful, they could run a great many such simulations." The only assumption he acknowledges at the outset is that "enormous amounts of computing power will be available in the future." Why we are to assent that enormous amounts of computing power would necessarily entail _sufficient_ amounts of computing power for the simulation task he describes (let alone "a great many" simulation tasks of that nature) is left mysterious.

I was meaning to watch this video and then it showed up again 😅

Great as always.

I haven't thought about AI in a while but this does remind me of a story. When I was in New Orleans my economics professor would hold seminars every other Friday during the school year that were open to the public. We would discuss a book or article and then have dinner at a nearby restaurant. I would usually ride with my professor to the restaurant afterwards but one time his car was full, so I ended up getting a ride with a character of a physics professor from Tulane who recently started attending the seminar. On the way there he begins talking to me about how AI will eventually become very advanced and will try to wipe out humanity. He said he was certain this would happen. In addition, he said he could also mathematically prove the existence of God. I listened with an open mind but didn't really know what to think of any of it. After all, I'm just a humble philosopher and economist and this older gentleman certainly knows more about physics than I ever will. I looked him up when I got home and it turns out he's a very controversial physicist. His name is Frank Tipler. I don't have much to respond to his comments but meeting and talking with him was an experience I'll never forget.

I feel like I’m a little late to the party on this one but it feels worth saying anyway. When Bostrom (or more importantly, the AI alignment doomsdayers, among whom I count myself) think of intelligence as little more than the ability to select actions that lead to the outcomes we want. It’s not like an IQ test or the ability to adapt or whatever, it’s the general fuzzy set of things like world modelling (building up an idea in your head of what the world is like), coming up with actions you could take, and accurately predicting the results of those actions. The basic idea is that all the goofy stupid AIs we might make along the way aren’t a guaranteed-apocalypse-by-default like some eventual superintelligence will be (unless we figure out alignment first, ofc). So for the most part we basically say humanity will probably survive whatever else happens in between, and the next extinction-level-event we seem likely to face (in the sense of being a thing that seems highly likely, as opposed to it just being a thing that could possibly happen) is a misaligned superintelligence. If we get a misaligned not-so-superintelligence while on the way, congrats to us we might well survive it, but we’ll be dumb and build something more dangerous afterward, and THAT is the entity we want to make sure has the same goals we do. Which seems hard, given we don’t even know what our goals are ourselves 😅

One other assumption that bothers me is that preferences are assumed to be transitive. The evidence seems to suggest there are animals with intransitive preferences (e.g sphex), we know there are computer programs with intransitive "preferences" (get stuck in a loop), I sometimes seem to have intransitive preferences and I'm not the only one (e.g I like film 1 more than film 2 which I like more than film 3 which I like more than film 1), why would we assume that AGI will have transitive preferences? Even if thats true by default, couldn't we program intransitive preferences in as a safety precaution? (This comment is a reupload from the unlisted video)

Never blame the tool. Always blame the person using it.

Bold of you to assume that dogs aren't planning the absolute annihilation of their bipedal masters.

All of my silliness will go on until… it stops. Responsible

Hey! I've been watching (and enjoying!) your videos for a few years now and am wondering if you have any sort of profile or public list of books on Goodreads (or some other way of tracking books you've read / liked). I feel like there's a pretty high chance I'd be into many of them :)

So yeah your channel is pretty awesome btw. Did these supersmart theists make super cool general AIs?

We really are doomed aren't we

Nice double pun to open this one.

I'm not a philosopher or a super intelligent person ( some might say not that intelligent). I would suggest that humans relative good at a lot of things but not anthropomorphizing every thing is not one of them. Inherent in what I understand of his thesis is that any and all movements are either advances or regressions based on our current C21st perspective. A very basic understanding of evolution since the primordial life is that its survival is conditional on it's environment and circumstances (i.e. not withstanding a restarting of the clock by a "natural" disaster..... a few mega volcanoes or a god awful large space rock.). The point of life is its self. Hence what ever conditions life tends to refigure its self to reproduce and prosper and if a version doesn't it disappears. There is NO GUARANTEE that future evolutionary movement would be an advancement ( from our C21st perspective) or a regression. Much less what our C23rd version would adjudge. Ergo it stands to logic that similar factors may/would apply with an AI It may conclude it's best option is to simply stop and destroy it's self ( there is no guarantee that super intelligent AI would have our biological drive to either replace us or have self consciousness and therefore emotions or drive to continue to evolve) as the logical best solution . Then back on the conditions for existence what isn't considered is the limitation of physics and the availability to create the resources to make it's next iteration et al and then there's the inevitable parts constituent failure. In short this is a pointless mental exercise as is worrying about The practically unknowable future. I'm more worried about the almost unfathomable level of human beings' internecine capacity for self destruction.

🐕👋👋❤

Why the reupload?

that bostrom guy seems wack man