One thing that you might have missed, and what likely resulted in the worsening performance from your own implementation is that the researchers only replaced the best model if the newer one won by a margin of 55%, otherwise it would be rejected and train for N games again.
please make more videos man, the quality is great. Would you consider exploring some of root concepts in this video in a bit more detail? like maybe a dedicated video on convolutional neural networks? I'm a beginner programmer and I'm super interested in this stuff.
AlphaFold 2 is a very impressive domain-specific feat. The matrix multiplication advancement is more of a meta advancement... it has the potential to seriously improve the computational efficiency of training models like AlphaFold 2, so in my opinion the matrix multiplication improvement will have a much broader positive impact than AlphaFold 2 has had so far.
At 2:35 you mention that we visit the root node (of the subtree) 9 times. I dont get this. Dont we just visit it once and then continue our DFS(depth first search down the tree). So essentially, dont we just visit it only once and not 9 times??
We visit it 9 times in the sense that we've experienced 9 different game branches so far as a result of visiting that game state. If you're simulating games one at a time, then you will pass through that node 9 different times. If you simulate games in batches, then you can do what you're describing. For the sake of MCTS, though, I think the "visit count" essentially refers to the number of leaf nodes that result from visiting a particular node. In that sense, it doesn't matter if we simulate one game at a time, or if we simulate games in batches.
Where do you think the energy required to run an entire datacenter worth of TPUs comes from, exactly? Ethics, climate science, and machine learning are inseparably linked when we're talking about a project of this scale, and we can't escape that.
Really nice video! One minor gripe is that at 5:30 you introduce a bunch of terms the layers/steps of what's done with the data that are not explained further than the abstract graphics on the screen. This is sort of fine, but when the next section at 6:06 begins with "So now we understand all the pieces of the puzzle..." it feels more than a little hand-waved. I'd had liked either at least some more sentences on explaining what each of those steps in slightly more detail, OR an acknowledged that we're not going to get into the weed of those things in this video. So it's more the disconnect from how your script's written than really a problem with the info itself. The video assumes that everyone gets what (for example) "a low-dimensional embedding of the gamestate" is and why it's needed here. I sort of can figure that out, but when it's thrown at me in between other sentences that are also dense with technical terms, delivered at that speed? Well I really didn't feel like I understood all the pieces of the puzzles after that slide.
That's fair, and thank you for the feedback! I've put a lot of thought into balancing between explaining things and assuming people know things, and I don't think I've got the balance quite right yet. I'd like to make a series that starts all the way at algebra and works up to state-of-the-art ML, but the amount of time and effort that would take with animations is enormous. Maybe this will become my life's work :)
@@2ToTheNthPower You certainly have a talent for making videos that explain stuff. So to keep making videos, (while not biting off so much that it feels overwhelming to continue), is likely the best (only?) way forward to get better at it, and to see if there's a career in it for you in the long run. From what I've heard other science communicator youtubers say, having people who *can* ask the stupid or at least less informed questions is important. For example, @Numberphile works so well in part because Brady is *not* a mathematician, so he's constantly asking the obvious next question that other non-mathematicians might have. One way to get those questions early enough in the process so that you can still answer those questions in a video is to have some sort of small group or community that read your draft for scripts that can give feedback on if something is unclear, and to pretty much after each paragraph ask them "Are there any obvious questions that this paragraph raises that maybe should be answered before we move on?". The tricky part is to find people who are both interested enough in the subject to want to read/listen to those early drafts and also give feedback, but who *don't* already know most of what's covered in the script. Once your channel grows it's possible to crowdsource this, but at first, having some friends who are willing to do it might be easier; mainly because community building is important and all, but it also takes a lot of time and effort that, early on, might be better spent on making more videos instead.
I don't know why people in AI don't admit that this field requires technical excellence in high-performance computing to make trainings even feasible. It's important to get the results within a reasonable timespan unlike Hitchhiker's Guide 😂 I've spent the past 3 months accelerating a training from a month to a few hours. Let's make better use of our hardware instead of throwing money at the problem. Modern PC games can squeeze insane amounts of compute out of our machines. Let's facilitate this in our trainings as well 😉
Only one complaint. Too much emptyness. Even if you are talking about atoms in the universe - insert some pictures over your voice, so we don't stare at black screen for this long. For the first 5 seconds of the video I thought my RU-vid app froze and was just outputting sound without video. But contentwise this is top notch, composition is great, and topic is good. Keep up the good work, there's not enough channels that have this quality talking about AI, you will be famous in no time :)
I used a networkx directed graph data structure. Some sort of tree could have worked too, though theoretically I think it's more appropriate to describe it as a graph. Could be wrong tho
@aarondavis5609 I have some stuff I'd like to say, it's a really cool video, and I'd definitely use it to introduce people to the concept (thanks!), but some stuff was a little confusing: like the highlighting of different terms in ucb. Also, P(s) is the policy, and shouldn't be there when we're talking about the stuff before the nn... But great video! Come join, we use transformers now as well
Yeah, that makes sense. P(s) probably should've brought in after the network was introduced. Ooh! Transformers are cool. I'm thinking about making a video on visual intuition for the attention mechanism, at least for NLP. We'll see how long that takes!
I started to look into implementing alphaZero for a diffrent game and this is such a great overview. There are some details that don't quite seem to align with what i got from the paper and its materials, but they are so insignifikant that they're not worth mentioning (besides I'm not even sure I got it right). I commend you for getting things to run, its a project that sounds easier then it is (or I'm just dumb). Right now I'm struggling with the NN part especially since I cursed myself by deciding to write my implementation in rust. Do you plan on publishing your source code?
This is definitely not an easy project, and it sounds like you're making it a bit more difficult by trying to code NNs from scratch, so give yourself lots of credit on that front. I definitely left a good chunk of information out... I don't really discuss self-play here, or adding dirichlett noise to the prior, or compressing arrays and making them hashable so I can put them in a graph without destroying my computer! There's also some ambiguity in my mind as to whether playouts are used at all after training is complete, and that seems pretty important, too. I have my code here ( github.com/2ToTheNthPower/Pente-AI ), and I recommend looking up "Accelerating Self-Play Learning in Go" on arXiv. I think it's a better resource than the original paper was. Hope that helps!
@@2ToTheNthPower It definitely will. Yeah with the neutral network i definitely shot my self in the foot because tensorflow in Rust is pretty bare bones. I'll also look into that paper but i have to say what helped the most was not the actual paper but rather the pseudo code they provided it just sadly skimps out on the neural network front
@@conando025 I gotcha. From what I've read about the neural network, they primarily used ResNets, since residual connections allow very deep networks to train and getter better results than shallow networks. I don't know if the KataGo paper has any pseudocode, so it may not be helpful on that front.
This is the sad state of machine learning technology, it has such great potential to do good but the ones that have the power to utilise it the most will only use it for personal gain, which has been the case with how humanity works for centuries upon centuries.
You made it right into the youtube algorithm, if you are able to produce a new video in the next days, your channel will grow extremely fast. Quality animations, interesting topic and great commentary. Good job :)
