The Cognitive Revolution is a podcast about AI where hosts Nathan Labenz and Erik Torenberg interview the builders on the edge of AI and explore the dramatic shift it will unlock over the next decades.
If you love The Cognitive Revolution, head over to Moment of Zen (www.youtube.com/@MomentofZenPodcast) for weekly episodes from Erik Torenberg, Dan Romero, Antonio Garcia Martinez, and a revolving slate of guests talking tech, business, and AI.
1:15 Albert says that doubling the state size in Mamba 1 doubles the wall clock time. Also, that in Mamba 2 much of the computation is not contingent on the state size. Why the latter? Computation time is constant because it’s one matmul happening in parallel as one step on the GPU?
When Albert says, multiple times, that they avoid to materialize the state, it sounds that they don’t materialize the state at all during the forward pass in training. Does he mean that exactly? Or that they avoid to materialize the full state at once, but materialize the whole state incrementally, chunk by chunk?
The comparison of AI with domestication in the video is deeply wrong.and thisis why ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-o5fd_6RjDdc.htmlsi=tW1JJmg4OgQJbqg1
I can't believe that there is intelligence in criticism praise just share that the creation of energy that can not be created or destroyed is just perfect by the Creation of it for sometimes being wrong and bad mistakes we may make. If God is the creator, then everything must be respected . How big are we allowed to believe in the impossible.I want to nominate for creating a solar system that would reflect his beliefs.
Great chapter, as always 🎉! Loved the AI SDR example, and the AGI/ASI "not really worrying" rationale because of fit/sexual status seeking human behavior.
If everyone has to tune dialectical visions of specific weights & measure in models that really handicaps much of the world I'd assume. Elon evidence of gen "x" based on the fundamental feature in our reality 3 lines of measure = eqaulibrium/ Optimus that tunes English law American pragmaticism of "grok" is the same weight & measure longitude & latitude self drivers likley needs hooked to Austin giga factory as a independent network. He said this ,and his behavior on x makes sense from a weight measure longitude latitude perspective. While x ai would need completly tuning. Just a random guess here. But what I do know strongly is what we can't continue doing. 😢😅😢 The narrowing threats as evidence in our world brings everyone on par with each other removes excuses and avenues of what is successful social behavior that can be tolerated or defended against because it garunttees that eqaulibrium can't be achieved can't be a guide or reference tools. It really annoys many genx & older that evidence in the world now has ill informed basian marduk mind models on par with why nasa math maps dark matter spirits in the sky being hardheaded and leaning into ease of access everything is physicalism as opposed to objectivism which has created situations where everything argue over slight variations thank to population species perception mind dope of the great debate . On astronomy I'm not a fan of mond theory lazy light it's all the methusela equation of ancient world. All the arguments against darwin and others have been discovered and mapped while everything he and the naturalist humanist euro, India, parliaments monarchs socialist ,jew ,ortho catholic of that age specifically argued against is proven through merit. It's come and gone to the point today's evolutionary lecture would get them blackballed by their peers just 40 years is spoke as they do today. Now the prenticious value of over time manmade hierarchy knowledge of good evil equations has value but it can't overcome horizon paradoxes and it 100% destroys the flesh as current cog in the wheel wokism nilhisms proves. It is terrible guide or tool for reference in this . The horsepower robot slaves and utility serfs are anylitically trained in the exact same way our ancestors was taught about bottom up nature and how we have ingrained within it value as eternal cosmos soul agency 1 atom individual liberties to drive 2 lattus structure and body personal actors free will inertia responses to change our 3 frame of reference critical extreme states or environment how we see fit. Anylitical mind that separates us from animals is a gift by God subjective properties of space nature building breathe of life giving emerging energetic actors. You can't have anything but a Statistical anylitical dead space within unless we fill that blank with ourselves lol
Hey shout out to this video! another Dr Michael Levin video down for me going to watch everyone I can find. My mind drinks his thinking like cool ice water. His courageous, honest approach to science, seems to have collapsed the rigid prison bars of science created by male ego, and in doing so, unearthed, revealed a plethora of current and ongoing information, finally, discovered and beautifully expressed by Dr Michael Levin. Not to mention he has flown open millions of doors that until him have been shut tight; this is exciting and encouraging because it predicts much more tested information. thank you for this video I subscribed to this site I really like the site it's very interesting I found it really fascinating some of the correlations the two of you talked about in the communication between that which we call entirely machine, and that which we call entirely human I personally have been of the opinion that we are neither, that the human we call ourselves, are already a combination a biological materials and materials that are other than biological. Simple, honest, approaches like there is no such thing as binary I've always found that to be simple logic and yet the culture I've experienced found the binary perspective to be stable I guess I don't know..." nothing is binary, what kind and how much", that's my favorite saying so far inundating myself with videos of Dr Michael Levin thank you❤GEM ❤
I am grateful for Michael Levin because he shows Darwinian evolution is seriously flawed because he shows it is not necessary for change -- he does this by showing change can happen in the lab in different ways - I never for a moment believed in Darwinian evolution (it has been shown to be seriously flawed) and always favored a finely tuned universe intelligently designed by God who in the last 10 years, has allowed scientists like Michael Levin to make leaps of advances -- the sheer progress in the last 200 years and in the last 50 years and in the last 10 years is amazing. Amazing in terms of shattering people invested in lifelong theories proven 100% wrong, amazing in terms of revealing some things will never be seeable or experimentally verified (dark matter, what is beyond the edge of the universe). Amazing in terms of how everything boils down to things like Pythagorean Theorem, Euler (differential equation), Newton (gravity, motion), Planck's constant, Maxwell, "i" in imaginary numbers, bosons, quarks, neutrinos, etc., etc. All of these are not random, not chaotic, but are elegant, simple, and beautiful fundamentals. To think there isn't a God behind all this is a choice we all have to make. I choose to glorify God (wisdom starts with fear (respect, awe) of the Lord). I'm sure someone will vitriolically insult me here - this is the normal course of action for some when God is mentioned -- geniuses like ML cannot and won't touch the topic of God in order to promote their studies and be the least offensive to the most people and I understand that. Yet brilliant people like ML indirectly show how amazing God's creation is at the cellular level, the neural network level, the electrical network level, etc. And physicists do the same at the math and physics levels. I think philosophers are getting pushed away in areas of theories of consciousness as God directs physicists and biologists and cosmologist to acknowledge their must be a Creator who created all of this finely tuned system and life as we know it. But philosophy remains important and was a stepping stone for most of the brightest scientific minds in existence. I appreciate and am in awe of ML as he tries to remain neutral and focus on the experiments and amazing results without giving God the glory. That's okay. Every experiment he conducts puts God's creation on full display. It's not random, it's not a chaotic fluke after billions of years. The tendency for some materialist scientists to be dismissive of God has been going on for centuries and they will continue to try and confuse people who believe in God. But ML just neutrally and brilliantly and subtly, perhaps without even realizing it, gives God all the glory by revealing His magnificence. Amen!
True self-driving is a long, LONG way away. People are impressed with (say) Tesla's FSD, but there's a vast gulf between that and true "robotaxis". Musk constantly makes outrageous promises that it's only months away, but that's a LIE.
Really great interview, Nathan. You do a great job with interviews like this one. I would really like Martin to speak to qanon or instagram pushing anorexia positive content to young girls. What harm is he willing to accept as part of the process of innovation? This point of view is particularly hard to accept from a person who has the resources to manage every aspect of his family's experience of these risks. Also his 'AIs produce median human output' is way too simplistic and demonstrably false. You get the sense he hasn't spent much time using capable models and perhaps sees no reason to because his paradigm precludes the possibility that there is value to him there.
Ironically I disagreed with a lot of Martins positions and don’t think he understand a lot of the comparisons though I totally am in agreement and aligned with having no liability for an llm model creator and stopping any drive at regulation currently. I’m more often disagreeing with the host who I really enjoy but I think is bias from his red team time, and is driving at the regulation hoop these days way too hard. To be liable if any llm use results in criminality based on an answer is absurd. There’s infinite ways to trick a model and a huge landscape of ‘criminality’. These guys fear mongering are eating out of Altmans hand, helping build his moat, are doing America a huge disservice.
The most powerful models will need heavy regulation. Unless you want people developing super viruses in their garages it's a necessity we regulate the leading edge models. Why would you think otherwise?
@@LiquidRR yawn. 🥱 all the info you need to do that exists already. Has since the advent of the internet. Motivated ppl will do bad things. I suppose you like and believe standing in TSA security theater keeps us safer too. I don’t want Sam Altman or the SF woke brigade deciding what I can and cannot use or see. Look how that worked out for Twitter.
"If we identified one mechanism that has massive destructive power" The AI lab would identify pre-release, and chose to race internally and harness that power. Who is going to protect us from that?
Oof. It didn't feel that Martin was actually engaging with Nathan. It was basically 2 hours of "This is a stupid conversation that's not worth having, and you're stupid if you don't think it's stupid."
This conversation was a tough listen. He repeats him self over and over. Its almost like he's in denial. And man, the ways he talks. As if everything he says his fact.
Yeah, I can't say for sure if there's "denail" but I am seeing that kind of thing a lot with AI. No matter what it does it's a joke. He also reframed shifting goal posts from "it's not ai it's not ai it's not ai" to "it is ai (nope), it is ai (nope), it is ai (nope)" I suppose if I wasn't a ChatGPT enthusiast at this point I would still be able to get work done with my head in the sand. But because I try to use it I see use cases everywhere that someone who didn't try never would.
Okay so just as a random example (out of a population of probably 100 worthy examples this month). I needed this code in python. Now before you judge me I can easily write this in C# but this saved me probably an hour in python. If you aren't coding or have your head stuck in the sand you're going to miss how beyond words level of amazing this is. "can you write me a python function that takes in the path to two images, originalImage and portraitMask. you should know that originalImage and portraitMask are guaranteed to have the same dimensions. your function should replace every pixel in originalImage with a zero opacity pixel where the corresponding pixel in portraitMask is < (200,200,200). finally, your function should take a third argument which lets me tell it where to save the output"
It’s not like he’s telling OpenAI to stop scaling. He’s clearly excited about new uses enabled by scaling. He just doesn’t think it’s going to create god.
Excellent and insightful discussion as usual! 🙏🏻 Martin presents the most coherent arguments for his position that I've heard so far. However, it seems like every conversation I've encountered recently about the future are dominated by arguments over hypothetical AGI/ASI/p-doom timelines, with relatively little discussion of the impact and progress of smaller models, the possibility of unlocking new capabilities from existing models (Ilya at least has stated repeatedly and publicly that he believes there's massive untapped potential here), and the paradigm shifting performance optimizations that are on the way. These improvements are not trivial or even (for the most part) hypothetical, and some of their obvious short-term implications are IMHO the most exciting, profitable, and potentially problematic areas we're likely to actually encounter in the coming months and years. Just a few off the top of my head for example: - Mojo - Mamba - 1.58 bit - photonic hardware - thermodynamic hardware - quantum hardware... Many of these have already demonstrated stunning performance improvements, which in some cases may have the potential to stack multiplicatively, not to mention the potential for massive reductions in hardware requirements including total GPU memory, memory bandwidth, or even 1.58 bit outright eliminating matrix multiplication...
Material sciences simulating elements in critical extreme may help extract low concentration mineral resources at cost minus is faking molecular bonds of nickel to decay or fatigue slip micky in titanium supply chains. Bio weapon is dumb when you can subvert xray machines or normal qc process American founders knew this it's why they warn us we all work with the same atoms but must interpret them radically differently to maximize benefits ( customers always right liberal power, individual responsibility of your sins) Moral law ,higher ed & beaucracy deals in complexity. Horsepower beast of burden robot slave & utility of serfs for free but owned and operated by local citizens being able to spread out for better quality of life multi skillset wholistic liberal individual power & responsibilities of our sin triality of self as selfless actors gradually break down gatekeeping agency and institution with quadrupled logistics new farmhouse has 3d printers, cnc machines instead of horses..universal operating systems training entering workforce young as the old days. We are witnessing it unpack both past successful social behavior that was created very esoterically convergences of image & llms nodes Is exactly what I think should happen to a certain degree optimum plus or minus concave etc etc. Super intelligent Is puritan movement encoding English to know it what would be in a future beast of burden worthy of a pilgrimage to search roots of civilization to get it right from the alphabetical exodus sources abap. It's multiple genrations and centuries and not puzzle peices of choo choo train time einstein when needs & demands causes countless to discover at once. This is typically not even longevity as future gens may see it as metaphysical novelty. We know anylitical mind separates us from animals it doesn't lead to life. Lol
Wonderful podcast. I sent you my resume via email , but ill check out that form you mention in the intro. Keep up the good work! They keep getting better and better.
Thank you! The form is here : docs.google.com/forms/d/e/1FAIpQLSf_YUQnwJKDejosGsiHU6lUy1VicleQ4PXevfrmRFJM9wdpNQ/viewform Also you will find them always in the website.
An excellent episode, the FSD discussion is so pertinent. Elon can spout BS about how close FSD is, and he could be 97% right. But the problem is that the remaining 3% still happen everyday and any failure would be a killer for FSD.
well, this was a deeply frustrating watch. Reasoning by analogy to historical events when we are dealing with the most significant revolution in human history feels pretty asinine to me. brilliant, but I think he just doesn’t get it.
awesome conversation ! truly thanks for getting into the details , so many vectors to research further . the multitude of clever optimizations in the field , and that they were getting such results on 10k parameter model trained on the quantum chemistry synthetic data .
I'm extremely excited about researchers applying AI to green chemistry / materials science + making products and materials that do what they need to do without being toxic to humans or other species. Lots of amazing things becoming possible, along with some significant challenges. Interesting times!
If it's the famous virtuous cycle of how science bolsters tech and then tech does it in return for science. The World Wide Web got invented as a result of research at CERN, and though it's not the only contributor to all the data we have today, it was a major factor to it, and now all that data lead to training AIs that will in turn help in research.
Fascinating! Tim Duignan’s work exemplifies the synergy between AI and human ingenuity, where neural networks serve as a catalyst for deeper insights into molecular dynamics. The discussion on the self-ionization of water and the generalization capabilities of neural networks is a testament to the potential of AI to uncover phenomena that may not be immediately apparent to human researchers. It’s a reminder that while AI can be a profound tool for discovery, it is the human curiosity and interdisciplinary collaboration that drive these advancements forward. Very excited to see where similar methodologies take us in the exploration of topological photonic and phononic materials!..
On children: they go through their, "terrible twos", learning that they're not the centre of the universe when they're tiny and adults can override them. Robots that think they're the centre of the universe will have meaningful tantrums...
