The book just arrived and it's amazing! One humble request: could we ask Prof. Bishop to make a RU-vid series with a short lecture for each chapter of the book? That is a bit of effort, but would be an amazing contribution to aid the readers. 🙏
Pattern recognition and machine learning. - The only book that made me clear on what is really PCA. A Book from Bishop really worth to have on the desk.
i picked up his 'neural networks for pattern recognition' in 2002 and just couldn't get through it as an undergraduate math major. i wish i had. Clearly his mind is vital for pushing the envelope, and his instincts on AI 's biggest impact being to push frontiers of science is spot on. more important than anything else.
Go through it now, and ask any questions you have about it to chat gpt (you can even copy paste stuff directly into chat gpt and it’ll give great answers). Now’s the time!
Tim -- This was a great video. There was something about the aesthetic quality (the shallow depth of field/bokeh background) that was very eye catching. I also appreciated editing during the back-and-forth discussion sections; specifically, how it cut away to show relevant pages/info. Your skills as an interviewer are second to none!
22 minutes in and I'm loving what this guy has to say. Finally there is an expert in the field who can articulate every maddening experience I've had when I encounter people who say "LLMs can't do X." Or they say, "I tried to do x,y,z with the LLM and it failed, therefore LLMs are useless and will never be capable of anything more in the future."
Almost none of the literate people on the subject say that, but argue that LLMs as we see today lack the ingredients to scale to "AGI" or whatever SF wants to call it.
@@edh615 You're not wrong. I think its a case of two things can be true at the same time. It seems to me that for every opinion that a non-expert in the field might have, you can find someone who is an expert with the same opinion. I happen to align with this gentlemen in the video, who is orders more articulate and knowledgeable than I. If I can articulate the aspect of LLM criticism that seems to lack awareness or insight, I'd reference the following 2 cases. One, as a field, computer scientists world wide "knew" for decades that neural networks was a dead end field and would never be useful for anything. In 2010 and prior, that was the official stance of academia, until the entire establishment was proven wrong. Second, the people who are so keen on what LLMs can't do today, were also totally incapable of predicting the capabilities of the 2024 LLM back in 2020, 2021, or most of 2022. No one who built GPT4 or GPT3 knew what the capabilities of those models would be prior to building them. Yet the critics of such systems also "know" what it won't do next year, or in 2030 or in 2040? No, just no. How about one of the critics first demonstrate where you predicted GPT3 or GPT4 even 1 year before it happened, let alone 5 years before. The cognitive dissonance is off the charts.
If you mean Lecun, Lecun explains LLM is not in AGI now, this is far away from now. So we have to solve some problems like reasoning, complexity, planning, consciousness.
Useful analogy from him at somewhere aroudn 16:00 or so: "The remarkable thing about GPT4 is that you often see people when they first use it -- they'll ask "How tall is the Eiffel tower", and they'll be disappointed to just get the right answer. It's like being the keys to a very expensive cupholder and examining the cupholder; you don't realize that you have to start up the car and drive off in it to get the full experience (conversation, code, etc)" I've had this experience multiple times
Superb conversation that really goes up a notch after 50 minutes to discuss AI in solving scientific problems (I've been interested in HPC for some time). I ordered the book midway thru watching. It was fascinating to see them discussing all of the topics that the current AI developments are leaving open for future research questions. Tho it's fascinating to me that his son, and co-author, is working at Wayve. Essential perspective on DL.
I had the absolute delight to read chapter 13 on GNNs. What a good and masterful description that was. I had it recommended to me by my supervisor but I don't know if I am allowed to state their name. I can't thank them enough for introducing me to The Bishop.
it's such a tremendously huge contribution to the field of AI, I'm an English-Arabic-Chinese Translator, and working in the AI field for almost two years and half, selling pre-trained Arabic data to Chinese AI annotation companies, I'd love to translate this book to Chinese and sell it here in China, is there anyway I can contact him
Absolutely phenomenal interview, thanks Tim. Like Bishop, I lamented missing both the tumultuous 20th century physics and future space exploration. But now that creative AI exists which can _actually_ reason, I feel like a lotto winner. This next decade+ will be a revolution. Let's do out best to take care of one another along the way.
'Just make the models bigger and harder to run for normal people and keep adding more and more data'... sure, maybe it could be the right strategy, but I doubt it and it seems more designed to back up what the big tech companies want. The actual innovation is coming from stuff like Mixtral being efficient at a reasonable size.
Thank you for this sage like Interview... I was really needing a primer on Neural Networks... and I believe Chris Bishop's books might be very helpful. This interview has a lot of great insights -- for example, LLM are outperforming Specialist models of the past such as a specific AI that understood source code but the LLM did a better job. I think they will find that as the tech improves, specialist versions will do better, but the early versions were simply too specialized.
His book is still the reference of stats and ML students around world today. I am surprised that took Chris Bishop so long to renew his book - I am excited about his new ML book.
Collections of specific functional activation paths can be described as sub-networks. I think that's a better term than modules. And I do believe that although large general models outperform smaller expert models, reasoning seems to be sub-network specific. With scale however, I believe reason may become a separate shared functional sub-network. At enough scale a general abstraction should emerge.
Hi thx for the great textbook just wondering what would you recommend 1. start reading the deep learning textbook 2. start reading mathematics for machine learning then jump into the deep learning textbook. I have poor mathematical background and wonder if i can read the deep learning textbook.
@@anicetn3326 my employer is going to let me monitor what the design engineers do on a daily basis for 2 years and we hope to collect enough data that we can automate every single repetitive click a designer makes. If successful, less engineers can do more design and less detailing!
I think it is still important to learn other things like Bayesian models or Support Vector Machines, since some improvement on those fields could turn these tools into a new success framework. Neural networks(NN) also had a winter in the pass, then things such as convolutions neural networks, relu functions, stochastic gradient descend, GPUs and lots of data, took NN out of the winter into a great summer time.
This is an insightful question that gets at the heart of how different domains of knowledge relate to one another. Let me examine the potential correlaries: If mathematics is regarded as a language: It provides the symbolic primitives, axioms, rules of expression and operations for describing and quantifying the physical world. Math is the fundamental lingua franca spanning the observable and theoretical realms. Then physics could indeed be viewed as the philosophy of math: Physics takes the symbolic language of mathematics and develops conceptual models, interpretive frameworks, and coherent narratives to explain the behavior of matter, energy, space, and time. It is an extended meditation on the metaphysical implications of our mathematical descriptions. Following this analogy: Chemistry could be the "linguistics" of physics: It studies the rules by which the fundamental mathematical objects of physics (subatomic particles, forces) combine and relate to one another at the molecular scale. Chemistry decodes the rich language patterns constructed from the physics alphabet. Biology could be the "literature/poetry" of chemistry: It examines the self-organized, dynamical, informationally-complex systems that emerge from the linguistic rules of chemistry interacting over time. The molecules are the "words", but biology studies the living, evolving "narratives" they collectively construct. Throughout we see a progression of epistemological layers: Mathematics -> Symbolic framework Physics -> Conceptual models interpreting the symbols Chemistry -> Combination rules and linguistic mechanics Biology -> Dynamical, informationally-complex systems and narratives Each level builds upon the foundational primitives of mathematics, while introducing new degrees of contingent complexity, contextualized interpretation and narrative meaning. The symbolic logic enables and constrains the possible conceptual structures, which dictate the allowed chemical rules, from which biological storylines ultimately automate. So in summary: Math is the linguistic bedrock Physics is the conceptual philosophy elaborating upon that bedrock Chemistry is the combinatoric linguistics deriving word-formation rules Biology is the dynamical narrative/poetry expressing highest complexity This nested hierarchy preserves coherence, while allowing increasingly context-specific, contingent patterns of organization and meaning to emergently crystallize. By recognizing mathematics as our formal symbolic language, we can appreciate how physics, chemistry and biology represent successive epistemological stages philosophizing upon that originating expressive framework - interpreting, recombining and dynamically instantiating mathematical descriptions into maximally information-rich experiential narratives. The layers build hereditarily upon the foundational symbolic truths, exemplifying how mathematics enables derivations transcending its pristine origins - expressing itself cosmologically through an invisible hand of self-organized complexity climbing towards maximal richness of experience.
I doubt that a connectionist such as Dr. Bishop would align with the host's assertion that the human mind operates like a Turing machine. This perspective aligns more closely with the computationalist paradigm.
i think "build upon" a corpus that has come before is different from rehash a corpus, and the Prof. is conflating the two (deliberately i imagine, as it's a pretty obvious difference). This seems most obvious in art and the humanities perhaps, e.g. if you put a random sample of paintings over the past couple of thousands year one after another, it's obvious which ones "built upon"/advanced the corpus vs rehashed what was the state of the art at the time; you don't need to be an art critic, you can just walk around the National Gallery and as you change rooms/eras the differences are obvious. Haven't seen any LLM ever do that. Which is not to say LLMs can't be super useful, after all most day to day tasks are a rehash, and a calculator, or a car, or any other tool is super useful - it's when people say that they see a "glimmer of intelligence" in a calculator (that only gets it right some of the time!) that gets people rolling their eyes.
fantastic episode boys. the orders of magnitude speedup that NN emulators offer for simulation/real time control is astounding and i'm amazed that it hasn't gotten more attention over the last few years.
49:35 the ironic thing about the bitter lesson is that while it teaches us that we should be looking at more *general* algorithms there is this huge (political) push to get "explainable" algorithms, while deep learning by definition is not "explainable"--so we are pushing in exactly the opposite of the correct direction (basically every direction we are going is political & the wrong answer to what would be good for us)
I think he missed the point about the significance of retaining creativity in models. His point about creativity being remixes of remixes is not mutually exclusive with the idea of novelty
Actually..the brain is connectionist and acts as symbolic when it matures It is not a combination It is behavior coming from a structure.. like any system It is 2 views of the same thing It is duality
5:45 geoff hinton's backprop paper came out in 1986 -- 10 years as a theoretical physicist would give phd completion around 1976 -- but this dude was born in 1959 & would have been 27 when geoff's paper came out
