Jonathan Blow on the "exponential" growth of AI (also Gemini fiasco) 

"could be a really long low exponent" Lmao!

recircling their own farts LOL

AI progress looks impressive when you compare it to the typical Silicon Valley rate of delivering features.

This is exactly why I keep telling Jon to stop schizoposting on twitter. He's so much more reasonable and articulate when talking out loud

These are awesome. Can you also link to the original streams or full videos?

Even an exponential increase wouldn't change the fact that LLM can't innovate, they can only recycle what they know. Also in programming approximation isn't enough. You need to be 100% precise or the whole thing won't compile, or weird bugs will be waiting to happen.

The thing is that the AI problems also scale exponentially the more you try to refine LLMs in general. That means that exponential scaling of computing will make progression linear if both are on the same pace. What will make it exponential would be another way to approach things, as JB more succinctly said

anyone know where i can read his argument

I hate how thick his taskbar is.

AI has come to be a marketing buzzword to push unimpressive technology and products, like chatbots, which have been done like 10+ years ago. Most people think that these "AI" will just suddenly improve on it's own. AI will never mean anything unless the underlying system/s are well thought out and designed, it's a broader design issue that people don't seem to mind. Noone seems to be upset at the fact that whenever I prompt a image of a cartoon cat, I always seem to get a practically identical style, color palette and lighting.

Where does this clips come from? I don’t see anything related on Jonathan Blow’s main channel

How are you accepting tips for Jon Blow's content? Aren't you worried about him coming after you?

If AI keeps getting better and better, but you still "can't really control it," shouldn't that be precisely the main cause for alarm?

Can someone tell me what game he is playing?

I have yet to see an example of exponential growth in nature that didn't wind up just being a sigmoid function in the end.

When was this broadcast? I see his last broadcast on twitch is his convo with Casey 2 months ago.

Seems very possible that we "run out of levels" and that's when the bubble bursts.

Everyone needs to remember that a) your savings account interest is also exponential growth, it's all about the coefficient and iterations b) Tesla selfdriving has been cllimbing local maxims for years now, just growing your training data (and Tesla has the best one) does not guarantee constant progress. c) hard part of machine learning is that when it fails at something, you can't fix it with more if-statements like the rest of software development. There might ceilings you can't punch through with your current approach.

Current AI (Chat-GPT and LLM's) cannot program something it has not seen before. I've given it problems from game programming that have been somewhat lost to time, but just aren't really used all that much or the documentation that was out there somewhat rotted. I've not tested this on Gemini or GPT-4 but GPT-3 couldn't nail it. It was reasonably close, but unfortunately close doesn't cut it in programming, either the code is correct or it's not. The basic idea is to take a unsorted list of co-planar 3d points and sort them into either a clockwise or counter-clockwise order. If you've ever taken the quake map format and parsed the brushes into 3d faces, depending on the method used, this is something you will need to do. While the method is straightforward, there is an edge case you had to take care of in order for things to work 100% of the time. GPT-3 never accounted for this edge case. LLM's (unless something has changed recently) are nothing more than probabilistic text prediction engines. You give it a series of tokens and it'll tell you next token in the sequence. It doesn't understand or care what the code is doing. If this were the case, you could have this stuff recursively self improve itself, but it can't... It cannot create new content. Will things get to that point? Maybe.. but it would be drastically different than what we have now.

When the AI starts reinventing its own source code than exponential growth is exactly what is going to happen.

Yet another S curve hyped as exponential. Sad!

Nice to see Balatro being played everywhere, it's a cool game.

My thoughts on why computers cannot truly understand valid program syntax: 1. Encoded strings is kind of a bad programming format to begin with. It relies on our ability to visually parse structural patterns, but the format is just a sequence of characters which is a far cry from ASTs, which is what code actually is. Creating a general AST representation file format for code would probably help a lot. To bad the DION project has seemingly been put on ice. Yes i know that this point is an unrealistic, unpractical pipe dream. 2. Current LLMs are pretty bad at reading things that are not a sequence of words, bad at understanding context, and bad at reading a lot of it. For more information, see: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-SbmETE7Ey20.html. I think AI agent programming has a future, but current tools definitely cannot get us there.

Mysterious generic exponential increase = deep learning

So true. The current tech is a dead end. LLMs are text completion software, they are missing the logic side of brains. They seems to understand what they do, but they understand nothing. We could see a jump in IA capabilities in coming years, but it would require a breakthrough, not just a scaling up. And it is entirely possible that we won't see this breakthrough in our life time, if it ever happens.

he is right

Exponential fiasco.

AI is currently plateauing hard. We had our iPhone moment but it's like everything in technology they try to scale it up or it ends in folding smartphones. I assume that there will be no crucial jump for a certain time.

Just got through the "intro" and you are "dead on". The people making claims are looking for praise, or not informed. This is still incremental and will be until "take off" which might not ever happen. AI will take your job, its unlikely it can "escape" or "steal compute" - only people who dont know how hard it is to build and deploy these systems say this shit.

Upvote if RU-vid deletes your comments.

Framing is off in this video. Majority of work, especially in web and computer software is technically solved. The issue that would be preventing the replacement of work, is having access to each companies proprietary code base. So, even in the framing of auto completion, the only true barrier is an artificial one. But since the main barrier is an artificial one, then the evolutionary factor likewise means some method of logic that is able to reproduce work. Without knowing the internals. What Google's Genie accomplished, was recreation of a video game based on video input rather than code base. Meaning there is a workaround to the artificial barrier that does represent some Intellgience, or patterned understanding capable enough to recreate gameplay from screen capture. If we are speaking on exponential improvement, the change from generating video to a lifestream would be an exponential improvement. Lastly, what LLMs constitute is some pattern recognition. That can likewise be refined by a logical thinking process. Therefore, what we have in comparison to a human mind is one-half of the equation. The next step beyond the ability to have pattern recognition that is filtered via a logical process. Would be a moment to moment improvement of the underlying network. Seeing as 2^2 is 4, and 4^2 is 16. Perhaps being in the second year of a certain something may be too early to call what is available, just an interesting tool. As if we do get streaming generation next year. That would just be 16^2, 256. If we are speaking to bits in a computer, this is when computers get interesting. 256^2, 65,536. Would be some network of models that would be capable of this generative streaming process. So when we get to: 4,294,967,296. Would be a network capable of this streaming process that can likewise have that moment to moment improvement. The real issue here is figuring out the curve fit alongside the time step. As for this to truly be an exponential process. That the process itself would merely be coexisting with AI that are improving moment by moment capable of any job. That halving time step may not be that far away. Especially if you throw out perfection and expect an AI capable enough to bumble around on a problem, then give the output when it has determined a solution. What we have isnt capable of that time series yet. But may sample high quality output from the Nth output to place back into the next training run. Most likely when we get to that exponential, it will be far more mundane than anyone could imagine. What comes after though, that is where things get interesting.

IMO we've been seeing exponential decrease across the board with Stable Diffusion, ChatGPT, etc. Every single release is more stagnant, less impressive, more the same as the last. There's something fundamental in the marrow of these things that is preventing them from improving beyond a fairly low ceiling. Look at Stable Diffusion. 1.5 was its cresting point. 2.0 was a resolution bump, but did not improve image coherence. All releases after 2.0 have been them fiddling with making it output faster. These systems are dead ends. They're already hitting their limits.

based

It's so over for non-STEM people

as bad as Gemini was, the underlying ramifications of generative AI are so imense considering the current structures of power and socioeconomic incentives that it surprises me that this is what catches the attention. And I said this in another Jon Blow vid, but it doesn't have to be exponential growth for us to see unfathomable results. In the ways that impacts the world ae most fields of study/crafts/jobs it is far more significant the rate of adoption than the actual rate of progress of the tech.

Hahaaa, now you can't train your AI in a feedback loop with a compiler because all your training data is in python and javascript and that's so slow to even start up that it's essentially impossible. And noone sees this. We could be progressing further, and now we don't, because people didn't care about their tools. Again.

I can’t believe this channel has patron donors. You literally upload someone else’s stream without any additional commentary

"I do not forget" Nor do I forgive :D This crap about "it will scale"... which one would you choose if you had to bust up a wall - a crowbar or a trillion q-tips? There's your scaling argument.

I've never seen anything approaching "reasoning" done by ChatGPT or any LLM. Whenever you spot something that looks like reasoning, you can slightly modify the inputs without logically changing the problem and it will fall on its face, which proves that no sort of "reasoning" capacity is emergent: it just looked like it was reasoning because you stumbled on something that was already in its training data. I think that wedding an LLM with some sort of logical feedback based on rule-governed behavior might end up working. It would certainly fit well with the "system 1/system 2" model of cognition, where what the LLM provides would be the system 1 non-rational fast intuition and the system 2 would be the as-yet-undeveloped rule-based system imposed on top.

Most cases of exponential growth aren't truly exponential. Logistic growth is more common.

Guess what, "being good at bullshitting" is leaps ahead of "not understanding anything". There's an emergent property of modelling the world and humans. Heck, it learned to do photorealistic rendering just by looking at photos. How many humans can paint at that level?

Exponential what? Do you know how much energy and water ChatGPT consume to seems just slightly competent? Everybody continue to pump up this "revolution" while what we have now is just a glorified search engine and statistical optimization functions.

AI is just a buzzword now.

Those who believe ai will solve any problem should learn some computability theory.

A parrot can talk but it doesn't know what it's saying.

This man really has a god complex.

Why do people value what this man has to say? He's only shipped like 3 games. IF THAT. Why are people acting like he's a voice worth hearing on game development when his actual real-world experience is so minimal.

Perfect example of why experts in one field shouldn't make the mistake of thinking their opinion holds as much value in other fields... Jon needs to stay on the topic of software and tech (where his ACTUAL qualifications reside). His takes on society and culture are... cringe, at best, and are generally a discredit to himself and his other well-informed takes on tech.

What's with all these posts fellating this mediocre game dev?