A Nash equilibrium sounds like what happens on roads where traffic evens itself out amongst all the roads towards some destination. When a new road is built, nothing really changes because the traffic just redistributes itself to an new equilibrium.
Shouldn't there also be a reward present in TD error at 42:30 and 50:25 ? edit: ok, it's explained a bit more in the 2015 lecure that this version assumes no intermediate reward
I would love to see how deepmind would build a city on its own in Cityskyline. See how its optimization would create the best and most efficient layout in real time. Maybe we could learn alot from that.
@@Sigmav0 these slides are from an older UCLxDeepMind lecture series lead primarily by David Silver. They do not include content on the newer AlphaZero models. Do you by any chance know if these updated slides are available online
Despite the success of A 0 nets in several games, I feel that is better starting point playing (random number) games with humans. Only then, when it has grasped some basic basics (by itself, not forcibly inserted by hand), let it play against itself. This way it could accomplish in thousands of self-play games what from scratch it´d take millions of self-play games, due to the total randomness and clueless of the first games. It´s not the absolute zero approach, but it has no "artificial" parameters handcrafted either. It learns from its own games all the way.
Playing with humans takes considerably more time than running simulations - so actually, playing millions of games by itself is still faster than playing 100 games from playing humans. Knowing that a game of go takes around 1h, you'd have finished 3 games with a human in the time that it took AlphaZero to reach human level play. Same for chess, when you realise it took Alpha Zero 4 hours reach a level higher than Stockfish... It should be clear from these examples that one of the particularities of AlphaZero is the speed at which it learns. Playing humans here both defeats the purpose of self-learning and actually wastes time.
