Full podcast episode: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-cdiD-9MMpb0.html Lex Fridman podcast channel: ru-vid.com Guest bio: Andrej Karpathy is a legendary AI researcher, engineer, and educator. He's the former director of AI at Tesla, a founding member of OpenAI, and an educator at Stanford.
It's great advice. It took me 5 years to become a competent design engineer. I thought I was competent before that but looking back I mostly wasted other people's time with my nonsense.
Einstein's quote, "if you can't explain it simply you don't know the subject well enough," is truly profound. I teach digital skills to youth in Africa. Every time I think I understand the material thoroughly enough, an apprentice asks me a question that demonstrates I don't.
This truly is a way to know if you have mastered a subject. I find that if I could explain it to someone non-technical in an analogy that they'll understand then I know that I understand it as well.
His intuition about "scar tissue" is eye opening. I've always been concerned about wasting time on the wrong things that it paralyzes me to take the initial step. But now I view it as a way to grow to know what is not a good way of doing things. 10,000 hours, here I come.
Lets do some basic math. If you studied machine learining for 3 hours every single day (which is highly optimistc) then you need to do that for 3333.33 days to finish 10000 hours. That equates to more than 9 years. 10000 hour statment is unrealistic and non-motivating if you ask me. Secoond thing he said is "it doesnt really matter where you put that time". This statement couldnt be more wrong...
I am not a teacher but I believe writing up is the equally helpful test to identify gaps of knowledge. I realise my mind is tricky that let me believe I know something to comfort myself. I realised whenever I want to test myself I need to try to write things up. Many times I actually couldn't finish my notes. Also, imagining I explain a topic to someone else, e.g. my father, is a test if I truly understand the concept.
You're not alone, I just realised that myself. I realised Writing with pen and paper helps me cover gaps. because apparently, I'll never write down something I don't understand. It's very helpful.
Same, sometimes I might feel like I don't know a subject really well or can't remember it but when I take the time to think about it with a pen and paper stuff, just start to pour in and dots connecting.
I believe spending 3 to 4 hours a day for a year should be enough to start with and after you get a job as a junior you will be able to put 9 to 10 hours at work which works towards your improvement. 10000 is a put off to many people but i still understand where he is coming from esp if you are undergrad, you have all the time in the world to study. But if you want to learn machine learning while you are working to pay rent and other financial commitments, that is next to impossible.
@@xWarEternalx that will take you 13 years my friend. By that time, the world will be moved on. The best think is to do 3 to 4 hours a day and after a year you get a job where you can do 9 hours a day using those skillsets. You won't miss anything as you will be in the game. 10000 is ridiculous
10000h is quite general number of course. But even he says it's the time needed to be an expert. But the reality is not every employee needs to be an expert. So no need be discouraged by 10k hours.
@@alphar85 10k is alright. And after 10k you become expert. You don't have to be expert to start doing the job. You become expert doing the job. While you're learning you're a student not an option xpert
10,000 hours is the equivalent of working 8 hours per day for ~3.5 years. But given that no one is going to consistently work at any one thing for that long, we're probably looking at 6-8 years, on average, to become an expert.
Even being a data specialist for 10 years I was so confused by machine learning models to choose regression do I choose classification, etc. in reality that is taking up so much of my time
Hi, I extracted text via Whisper AI and GPT-4 summarised it: 1. Advice for beginners in machine learning: - Focus on putting in 10,000 hours of work - Form a daily habit of working on the subject - Compare your progress to yourself in the past, not others 2. Overcoming paralysis by choice: - Don't worry about making mistakes; they help you learn and grow - Focus on what you have accomplished and the work you have done 3. Teaching and its benefits: - Teaching can be frustrating, but it helps others learn - Preparing educational material is hard work (about 10 hours for 1 hour of content) - Teaching strengthens one's understanding and reveals gaps in knowledge 4. Code as the source of truth: - Building things and using actual code to teach concepts is more effective than using slides or mathematical symbols This structured summary highlights the main points discussed in the podcast, making it easy to understand the key takeaways.
Me: I want to get started with machine learning. Do you have any advice? Andrej: Spend 10,000 hours to become an expert Me: Ok, but what about getting started Andrej: You'll need to spend 10,000 hours
I guess what he is saying is that it doesn't matter so much what you do, e.g study at uni, get an internship, do projects that you are passionate about - as long as you get in the hours. Whether it's correct, I don't know and probably depends on you and your past experience. His experience is somewhat biased by being around Tesla employees and MIT students...
He is right. Im 1.5 years in and spent the first 6 months "wasting my time" 2 hours a day. I didnt really waste my time and was learning, but what I mean is I should have learned other things first. You just have to have the mindset of putting in x amount of hours a day and the rest takes care of itself usually.
I would like to remind people that ml is not just coding. You have to learn mathematical concepts like statistics, probability, algebra, calculas. You don’t have to go deep but you would have to learn at least basics to be able to understand how things work under the hood and then you will have to learn libraries to be able to implement theoretical concepts. I think after a hard grind of 6-8months(4hrs a day) you will be able to get a job in the field but it won’t be easy. Good luck
10,000 is required but not sufficient. Four additional factors: 1. A learnable skill in a valid environment. 2. Quality feedback. 3. Push your limits. 4. Repetition to create automatic behaviour / fast brain response.
I also heard somewhere that most people at the top of their field across all fields tend to dedicate 55-60 hrs each week towards their craft, either through direct work or supporting activities (e.g. a top footballer could spend so many hours in match time, so many hours training, so many hours in the gym, so many doing analysis of games, so many visualising and so on...). Do the math yourself but to reach 10,000 hrs even on 60 hrs a week (excluding PTO and bank holidays) you're looking at 3-4 years. And that's assuming your day job isn't filled with unrelated tasks. As Andrej says elsewhere in the interview it's also good to have some chunks of these hours together, like a few long-hour days with minimal interruptions, so you can really make some good progress in a particular area.
So is it possible to reach 10k hours for multiple fields? My son is 8 and has been playing soccer for 3 years, but of course who knows where that will go. But at least if you start young you can get in the 10k hours much more realistically. But as an adult I have too much life maintenance to do, I don't know if I can do 10k hours in anything even though I want to. The non-stop groceries, cooking, cleaning, laundry, etc is a lot once you have a kid.
@@nofurtherwest3474 Yes and I'd say it's best to build skills in multiple fields, it will exercise more of your mental and physical faculties, which in turn leads to better health, happiness, success and so on. Then over time you can lean more into what seems to suit you best. If you're struggling for time remember just spending 5 mins a day on something (e.g. learning a new language) will lead to results. If you're awake for 16 hours a day that's 192 chunks of 5 mins. You just have to be very organised, efficient and disciplined. Can even apply it to other tasks like responding to work emails or playing with children. Most things can be done in 5-10 mins, even including meals and breaks.
@@nofurtherwest3474 You don't need to get 10k at anything, you just need to be interest and keep your interest about something alive. That is how you become master at anything. You you teach your son about commitment and support him at what he do. Not build a path and walk him through it. Let him decide.
10,000 hours is a good average bet, but we're not all equal in intelligence, memory etc. Some people will only need a few thousand hours, some will need more.
@@Mstorac1990 Yeah, it is a known idea from way back and it has been disputed, so that is why I made the comment. Just that he (an accomplished deep learning expert) is confirming the 10,000-hour concept. I should have worded it differently.
This is such a generic advice applicable to any topic. No directions whatsoever on where to start and which areas to focus on. As someone who wants to learn ML, I got nothing from this video.
10000 hours is a hell of a time so picking a random thing and grinding to that will certainly put you right up there whether you like it or not. But it's just better to invest time in something u r better at so it will be fun along the way. However u can pick any random stuff and get better at it it's very much possible regardless of what anyone says.
Even the person who first pushed the 10k hours thing (K Anders Ericsson) backed away from (and clarified what he meant, limitations of the studies, etc. What we DO know about repetition practice, “putting in the hours” is that the nature of the practice/activities plays the key role. It’s the cliche difference between someone who “practiced” for one year, then repeated 10x and someone who has 10 years of highly novel, diverse, challenging activities. The 10k hours idea led a lot of people to fragile knowledge. Like “deep and strong attractors” but not so much adaptability. There’s no way around needing lots of time and effort “practicing”, but I think of it like the way some AI algorithms are dramatically more efficient and robust, given the appropriate context. Most education is based on a linear process… a RL toward objectives. Given humans are complex dynamical systems, something more like a Novelty Search, Exploration, MAP-Elites (but for humans). In movement sport skill acquisition, there’s been a recent push toward NON-linear pedagogy. I first explored those ideas for teaching programming, but then left computer science to do horse rehab 😁.
10k hours => 416.6667 days of 24/7 studying/working => 833 days of 12/7 studying/working => about 2.2 years of studying/working to become an expert, seems pretty realistic
"I spent a lot of time working on things that never materialized" I wonder if he's talking about his years at Tesla... In the meantime his old colleagues at OpenAI were perfecting chatgpt.
IMO it’s just a way to say that becoming an expert requires huge dedication and time. That being said, I’m not sure about it, what if after 10,000 hours I still suck? I mean for sure time is very important, but sometimes I feel that those gurus are suffering from the survivorship bias (because it worked so well for them): even putting an extremely long amount of time doesn’t guarantee any accomplishment , it may be just one of the important factors.
i think at that point you are an expert, and you will have a path in front of you for success given your extensive knowledge in your field. then youll know how to monetize it or get it in front of the right people. i think 10k hours of perfecting your craft is better than 10k hours of just sitting and thinking of what your next move is.
if you spend 10k hours on something, you probably love doing it and THERE WILL BE A PATH for you, either one you make yourself or one that others will help you carve.
10k hrs is not correct. Do you think a 4 year old kid that is better at piano then most adults practiced for more than 10k hrs. No. They had a straight path that was guided by a mentor that has already been through the rough bits and know now what is right right direction, plus with instant feedback they can skip all the fluff. Imagine if everytime we learned how todo something we had todo everything from the beginning. Say you want to build a car and you used the same logic as the 10k hrs rule. You would need to rediscover the physics behind an engine, melting metal, manufacturing, reinvent the wheel, the calculus behind the science everything. You see how long it would take it would take forever. So now say that you followed the guidance of the people that came before you. You learn and follow the right path they took. Say it took someone 20 years to discover known and what they know but they have this sense inside them that if they where todo it again they could do it in 10% of the amount of time it took them. You see how it doesn't take 10k hours to become a master of something. Follow the path already paved don't try to make a new one when there already is one right next to you.
10,000 hours is bullzhit. You can fly an airliner with only 1,500 hours of flight time. Are there more experienced pilots beyond 1,500 hours? Yes, of course. But setting the standard at 10k hours is not true. You will be an expert much sooner than 10k hours.
I agree with the 10k hours idea, except for Linux. I feel like I've spent 10k hours trying to get a damn printer driver to work in Linux and that shit still won't print ;)
It is like if you spend 10k hours on a game, for this example I will chose CS:GO. If you would spend 10k on it, you would know everything about it but, yet, you still need to improve in some areas. Same goes to programming.
I love that concept of 10k hours. It resonated with me because of some of the things I do in my own life and career, but it also made me realize I've been stuck on some other aspects of my life, for fear of not making "the best choice." Like choosing a programming language. The right choice is to dive in, learn and correct as you go but just work at it. Great stuff.
The consept of 10k Hours works for closed learning systems. When the output is close related to the input and the feedback it has no delay. Like learning golf. But usually the learning systems are open and needs more general knowledge about how the world works
@@nsyll yeah, it feels like computer science/programming is a "wicked" system (not neat). Forget where I read about these terms. CompSci features the WORST of math AND language. Or, it seems like that to me. Haha. Can't "memorize" anything really... has it's OWN logic. I am gonna try to "chunk" things into my long-term memory. Need to get to 10k hours!
1 hour a day = 365 hrs 3 hours a day = 1000 hrs 6 hours a day = 2000 hrs (5years) Learning is a slope. and if you get a full-time work position, in half a decade you will be an expert, at the top of your class.
In the last few years, I've realized that the path you pick doesn't matter. With consistent effort, patience, and time, you can do anything you set your mind to, so don't get hung up on the fork in the road!
takeaways: Advice for Beginners in Machine Learning Focus on Quantity of Work Believer in the 10,000 hour concept Pick something you're interested in and put in 10,000 hours of work Form a daily habit to maximize likelihood of reaching 10,000 hours Avoid Comparison to Others Only compare yourself to yourself from some time ago Progress is motivating Don't Get Paralyzed by Choice Wasting time doing something wrong is not dead work Accumulate scar tissue and learn from it Focus on what you have done last week Why Teach? Love happy humans Not necessarily love teaching, but love the outcome of happy humans.
10000 hours of training is what is needed to train our brain neural network to get good at machine learning. That's a lot. But, with daily training, we can get there.
its very true. jsut about anything you pick, programming langauges for example, its all about knowledge, skills, practice and experience. Hands down that always wins.
For those butt hurt about the advice he’s just saying you need to put in the work. Start some where, make mistakes, learn from them and become smarter. Eventually you will be immersed in the subject you were previously curious about and before you know it you will be an expert
I have a bunch of small niches where I’ve just become the #1 resource for that thing and built communities around that thing. I help people to an extreme and they seem to think I’m overly “nice”. In reality I’m just practising my learning techniques in a way that benefits both of us
The consept of 10k Hours works for closed learning systems. When the output is close related to the input and the feedback it has no delay. Like learning golf. But usually the learning systems are open and needs more general knowledge about how the world works
For me personally the actual 10,000 hours is not the important part as mileage may vary depending on people’s experiences. I think the much more important thing it enforces is to just start doing it. Yes you may fail as they say in the video but you gotta just do it essentially. This is kinda just a universal truth, if you wanna get good at math, just do math. Programming? Do programming. It really reinforces the need to just do something, anything, and learn; rather than spending 2 months trying to figure out the best way to learn something you could’ve learned in 2 weeks, something I’m very guilty of from time to time. If you think about it the education system with its deadlines and such is more of a tool to force you to just do something, but outside of that one of the most important disciplines is actually just starting and trying things.
10K hours are basically 5/6 years... if you consider 8h/day for 5 days/week for basically 9 months.... BUT you should always consider a stimulant environment --> it is a bit of a gross approximation if you think about it
oh goodness, the 10,000 hours is so famously misunderstood and taken out of context, especially by Malcolm Gladwell. Karpathy seems to be aware of the actual statement, because he casually throws the word "deliberate" in there. Unfortunately there's a bunch of other conditions on top of the 10k that need to apply to become an expert. Mindlessly playing 10,000 hours of Counter Strike or chess will not be sufficient to make you an expert. And for some areas even 10k hours is not sufficient, e. g. you won't get into the NBA even with just 10k hours of deliberate practice (and the other conditions).
You should totally invite Daniel Schiffman, who is the founder of the Coding Train, on your podcast. IMHO, he is the best RU-vid educator on the platform.
only watching this made me realize we didnt take enough appreciation for Andrej 's work. All these good courses for free! I wish I also spend more time learning from him and start practicing.
Becoming an expert does not mean landing a job, because greedy businesses want 10000s of papers "aT tOp ConfErenCeS" and a PhD not from anywhere but "tOP TIeR" university.
Ok not to knock what hes's saying it does apply to so many things in life. But to this I'm not so sure because if he's saying 10,000 hours (we only got 8,760 in a calendar year)...... Then how good is A.I.???
This is the most unproductive advice ever. So much of problem solving is having the right intuition, and without approaching the subject methodologically not only do you waste time but end up not being able to apply effectively if at all. Say you are a eager learner and the first step you wanna take is to get the probability theory fundamentals right. But you pick up a Probability textbook but measure-theoretic. If you have the math background it will take you a year, without then bg, that’s 2-3 yrs. Tf did you learn about machine learning? Now let’s say you wanna learn some actual ML so you pick up idk probabilistic machine learning. If you are a genius cool that’s 1-2 years. If you are not, good luck lol. Assuming the learner has infinite time budget is just… stupid.
10,000 hours realistically is 3 hours of study every weekday plus 6 hours each on Saturday and Sunday for the next 7 years...and by 6 months in it mostly likely will be outdated information 😢
US people like to put some stupid points at the table like: "if you bla bla bla 10,000 hours you're gonna be an expert bla bla bla" stupid point, measure knowledge by "hours of studing" you can have 20,000 hours of knowledge e know LESS than a 10,000 hours guy.
The advice of 10,000 hrs is actually pretty harmful to beginners and it discourages people from even starting. 10,000 hrs equate to 15 years if we put 2hrs every day. 10,000 hrs theory was mostly given for pro athletes, grand slams & ultra successful people. Most people who are looking to get a job in AI or want to get good enough to develop products should be able to do so within 1-2 years of regular learning. Please do not discourage people by posting such clickbait videos.
I think an idea of spending 10 000 hours in learning of one thing is ridiculous. This is literally 10 years of work. Let's be realistic, nobody works or studies more than 5 hours a day. Of course, you may spend 8+ hours at work, but not all this time will be dedicated to learning activities. There will be a lot of routine. So let's assume we spend 5 hours a day actually doing something educational. This is going to be 2000 days. People work 5 days a week, 20 days a month, which results into 100 months. Let's say you take a month of a vacation every year and spend another month not actually learning to avoid burn out. This is 10 years period. Really? You need 10 years to learn something new? Well, if you want to become a medical doctor from zero to hero, than yes. Even longer. Otherwise, if anything takes 10 years to learn it just doesn't worth the effort. Learning to program (reasonably well) for example takes no more than 3 years. Learning a foreign language using the same amount of effort every day takes no longer than 3 years. Learning to fly an airplane probably also doesn't take more than 3 years. And there are a lot of things which take much less time to learn.
Why machine learning guys say things like "maximize your likelihood" when talking about ordinary things ? That's like: "I have to wake up and adjust my routine parameters to increase the rate of convergence of my tooth brushing algorithm"
Am I giong crazy??? 10,000 hours equates to 5.2 years thinking of AI, thats 8hr a day Mon-Friday. Why would I start ML if its gonna take that long to get good at it????
Hard disagree. There's a reason anyone who does a PhD, which is a degree in becoming an expert on a specific topic, requires the guidance of an advisor/supervisor. He's right that people over index on unimportant details. But there are important details and they aren't immediately observable from the perspective of a beginner.
