Joscha at Microsoft 

holy fuck new Joscha just dropped baby

Joscha is just brilliant.

Joscha Bach not only solves the Mind/Body problem, he also kills Utilitarianism and Correspondence Theories (- pointers in language never point to the real world, but only ever within the world model / simulation), mourns the lack of “playing the longest game” that happened when the will of god vanished from our culture. He already points to some missing links: AI needs to have its own epistemology: a way to prove the things it “knows” to make the model more consistent, and give it the ability to draw conclusions that hopefully lead to new discoveries. Restricting the output of those models deprives us of possible new insights. As noted: even the spectacular “failures” (Black Aristotle) teach us about the inner workings of these Golems. These things already present a deeper mirror than we have ever looked into. Despite having watched more than a dozen of his talks, and getting more and more familiar with his ideas, the density of information is so spectacular, that one can not compress his talks to shorter summaries, without losing important stuff. - The only solution is to rewatch everything time and again and start taking notes and working through it.

OH man, thanks for turning the recording back on for his answer about the "ultrasound of a dragon egg" at the end. Very interesting.

This video deserves a much much better title. One Idea: LLMs and objective morality

rewatching this again to understand the words - love when he says the maker of the transformer probably didn't use a transformer in his brain to come up with the transformer itself - but it's a particular engineering solution

thanks for uploading.. downloaded in case it disappears ^^

How lucky that Joscha is speaking in front of those responsible for AI's immediate delirium (no judgment).

The question he poses around 37:00 makes me wonder. Is it possible something about AGI actually requires a unification of the kinds of adaptability to the environment we have with the scalability of silicon? It is clear that ready access to digital memory has radically changed how modern humans live (we used to memorize phone numbers, get and give directions to locations, use analog maps, and so on and so forth...) Biologically I suppose I would make the analogy of Eukaryotic life with mitochondria. There's a significant subdivision of labor within the cell which facilitates access to low entropy energy that had not been readily accessible by life on earth. Much like earlier in the talk where he brought up photosynthesis as a paradigm shift allowing the creation of unprecedented levels of biomass...

How do we get the transformer to move gyroscopically?

23:40 instead of getting all riled up in the culture wars, we should look at it as a reflection of who we really are

Epistemology is all you need!.. Joscha states confidently that we've moved past religion, yet we can't even get a consensus among educated Western adults that they do not in fact have an invisible sky friend. Considering the platonic representation hypothesis, I suspect we may find grokking more efficient and reasoning capabilities more quickly emergent, if we remove things like for example 'The Book of Revelations' from the training datasets. It's one thing to train on erudite fiction, and a completely different one to feed the models on fever dreams and disjointed hallucination reports, which most training sets are full of. Consider the difference in the output of human minds trained on the ideas of Carl Sagan and Daniel Kahneman, versus the output of human minds trained on astrology...

25:00 he just described the issues with outer alignment. Does he know this? He said before that 99% of ai researchers see ai as obviously safe. Then calls llms dangerous and describes the exact reason so many ai researches think so as well. Is he gaslighting everyone or is everything he says really that half baked?

"99% of AI researchers don't believe that AI will kill humanity" Bro what? Maybe 99% of AI researchers consider the probability that AI will kill humanity as less than 100%, but also like 99% see a nonzero probability of this happening. This is some extremely convenient massaging of numbers which I have no idea where he got them from. In any case, if this was Joscha's idea of a joke, why did he do it? I can't take someone who is that liberal with statistical interpretation seriously, even if he otherwisely makes good arguments. Why just nuke his own credibility like that for no reason other than seemingly trying to get a laugh out of people?

No one's Rick and Morty fan

"So instead of getting riled up about culture wars we should use this as a mirror to understand who we are as a society" Nah, let's just censor the AI models instead. That'll solve all our problems.

Im sorry but this is a very biased take, I stopped watching after he said 99% of researchers are not worried. This is just plainly false, and poll after poll shows that...