I'm (still) calling BS on the ".. basically trained on ALL of the internet." The 3 pillars is a strong metaphor, to be sure. Alex is clearly very smart.
classic.... "... society wide impact... (without the burden of credit or compensation of any kind)" Sure, why not? That's exactly the formula that got us to where we are now, right? The problem with blood sucking as an m.o. is the people with the blood eventually wise up. It just isn't sustainable. Is that the basis of HIS company, or is it for profit? Or maybe the "rules" are just special for him and his employees. :-P
I completely agree with Alex. It's all about the data for the most part. I find it interesting when I force the AI to cite its sources, that they are often very high level fluffy ones. Its difficult to get the AI to get down into the weeds because the data in its training set often doesn't contain it. Combine this with context window limits and the very nature of the LLM's (billions of parameters being calculated on single precision GPU's (most of the time) means they simply can't retain the detail by the very nature of the model and hardware its running on). He is also correct about the fact that most of the knowledge we have is not written down in a manner that makes it available to train an AI. The JP Morgan example he mentions is a good example (which I helped design for JP and a lot of the World's largest banks). The other classic example I always give people is software architecture. While we see a lot of AI copilots these days, they can only write very small pieces of code (and struggle to do even that). But they are not capable of architecting a larger, coherent system. Even if context windows were 10000X larger, there simply is no effective data to train the AI on that will allow it to learn how to design a complex piece of software well.
I'm calling BS. Scale is a data company, its not that we hit a data wall but we need better LLMs and architectures. Facebook's V-Jepa is the way to get there (at least the concept) but its no where good enough. We are not limited by the data, not compute, but we are limited by the breakthroughs...
As someone who is training for data analysis….he’s just repeating the most basic principles. But the fact of the matter is, it’s the function of the data set itself that the LLM’s can’t get around. Weight and CONTEXT of the data to itself and the real world environment. In lay terms: we ask stupid questions, and get stupid answers. “We’ve already used up all the Internet data.” Red flag. We haven’t scrapped the surface of HOW to use the data properly or more efficiently.
Even talking in the same breath about 150p of corporate (financial transaction) data and 1p web crawled data to train an LLM as if the size comparison has any meaning whatsoever is just so much foolishness. Even calling all of it "data" is about as helpful as having a discussion about a horse and a stairway because they both have "atoms."
Scale ai is nothing but a data annotation platform that hires people from low pay countries. Company is ticking time bomb😂😂😂 Also Alex you need a better stylist.
If you are passing on the recommendation of 25-30% of your team for hires, that means you aren't hiring Seals. It means you have created a culture where others will be afraid to challenge in push out of fear of rejection.
This is a pathetic interview. Getting CEOs who talk about AI as if it's black-and-white is an utter waste of time - please interview an actual comp sci researcher or someone who knows what they're talking about. This guy is a fund raiser, and a top-line manager.
Wow, this guy is worth billions of dollars? So not impressed by his level of sophistication and understanding of AI. Just goes to show, you don't become a billionaire by being smart, you become a billionaire by getting lucky.
His perspective felt thoughtful to me and demonstrates practical aspects of AI in building everyday Intelligent applications instead of just theory … I was not expecting a class on deepening my understanding of backward propagation and fine tuning 😅
