George Hotz | Programming | rewriting linearizer (tinygrad) | Day In The Life Of A Software Engineer 

ASMR stream | A Day In The Life Of A Software Engineer | 01:45:00 that is how real work on tinygrad looks like Bounties for tiny corp / tinygrad -> docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/ ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-lnVQsJJFcdg.html Hiring entire stack for tiny corp join if you are interested | ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-lnVQsJJFcdg.html work major source of value in your life Pre-order tinybox buy.stripe.com/5kAaGL6lk9uX9nW144 more info on -> tinygrad.org | github.com/tinygrad/tinygrad comma 3X comma.ai/shop/comma-3x | best ADAS system in the world openpilot.comma.ai | from $999 comma.ai/shop/body the future of people Support George by subscribing twitch.tv/subs/georgehotz | Follow George on twitter.com/realGeorgeHotz to be up to date | Read George's geohot.github.io/blog/ Chapters: 00:00:00 intro 00:00:30 ASMR typing 00:08:37 Lana_lux raid! 00:10:10 context missing, real work accessibility 00:12:50 linearizer 00:14:40 tensorcore implementation 00:15:10 tvm schedule 00:20:30 upcastmid 00:25:40 optuops 00:28:20 tvm parallel vectorize 00:30:35 future design 00:33:30 reorder 00:37:20 option space searchability design 00:41:40 git commit 00:44:00 test_float32_unary 00:46:00 fix_metal_tests 00:50:20 UOptOps(Enum) 00:54:00 group_for_reduce 00:59:40 yellow and purple 01:06:45 vin[0].arg out of range 01:07:45 tensor cores nightmare 01:10:40 metal_wmma 01:12:10 tvm tensor cores 01:15:40 problem 01:16:35 tvm implementation 01:17:30 thread_local_aliases 01:19:20 can fix this 01:25:00 device required 01:26:20 TensorCore.__init__ missing 01:27:00 specialize dataclass, perplexity 01:28:20 dataclass inheritance 01:30:00 dataclass child class with default properties 01:36:50 git commit 01:40:50 tvm flexible interface 01:43:20 cool test 01:45:00 that is how real work on tinygrad looks like

"real work doesn't involve talking about real work" amazing quote lol

Great project. It looks complicated! Wondering about future cuBLAS support. But also pondering whether recent acceleration in development of NVK, the open source NVIDIA Vulkan driver, will evntually eclipse it. Or at least make an appealing case for keeping everything open source.

George Daddd!

I resonate with that first line so much. (can skip for now)

Does anyone know whether HIP/ROCm are going to support AMD's desktop APUs (processor + iGPU + AI Engine)? Like the 8700G or 8600G (desktop SKUs). I saw recent news that they'll support Strix Halo APUs, but that's a laptop part coming out in late 2024 or 2025...

love the the real work stream! I can get some work done myself with out George doing epic rants every 20 mins

New keyboard? Type-S now?

A world without hotz is no bueno. Where art thou?

rant for the win.

holy crap dude its a player micro controller from old time coming to life it have play, stop, pause, skip, prev, repeat, shuffle. and then now its a function for everything, and then it turns into a class Weapon, with all the member function on it, and then in video game its a master control with player character with human body. now its an AI brain, now it need class Body and then all limbs and micro controller and actuator

ASMR typing lmao

linearizer? I barely knewer

*what most people didnt know is george had just done 48 hours of straight programming leading into this video*

can anyone tell me what linearizer is ?

does your program have enough intelligence to do basic, coding like audio programming, graphic OpenGL or Win32 API or oprn stl file and modify or manage file compression ?

first

Not a single distraction in an entire coding stream? This is unheard of! Plus boring, see ya on the next rant. :(

Tenstorrent emailed me today - looks like anyone can buy their card now. ~200 or 300 TFLOPs for their two cards ... of FP4 :)