A POMDP is not an MDP. The infostate is not a sufficient state of the history. The whole trick is to transform the infostate formulation into a sufficient one.That’s why the early strategy of the opponent needs to be considered and hence the added complexity and the computation. This could be clarified in the first place.
Cool paper. IMHO the r-p-s example is a bit of underwhelming. There is a fundamental difference between games like chess and rock-paper-scissors. in chess the moves in a sequence are way more dependent on each other whereas in r-p-s it's somewhat independent just like 50 throws of a coin are independent of each other. Please correct me if I am wrong. Assuming the results of every game independent of each other (a game consists of one move from each player), picking rock is more potent for p1 (outcomes: 0, -1, +2). Then upon observing p1 picking rock often p2 can up the probability of picking paper more and so on. Eventually, though it should come to the same (04, 0.4, 0.2) equilibrium, starting with somewhat like (1.0, 0.0, 0.0) instead of the authors' suggestion of something like (0.0, 0.0, 1.0).
To understand the first part of this video it is useful to look at the definition of Nash equilibrium and Pareto Optimal strategy. The coursera Game Theory course (1st and 2nd week) is great to understand those concepts.
Pedantic man responding. That is a 3 outcome dice with six sides. Actual three sided dice are seemingly very rare (I have never seen them after a lifetime of playing games) but a quick google search reveals they do exist - it looks like they have to make use of unused rounded areas to work.
deeplearn.org/ is a super website which tracks new releases etc. Besides that I subscribed to a few Telegram channels such as t.me/ai_machinelearning_big_data
it is the strategy whose actions are always a best response. an action is a best response if u(a* | state) >= u(a | state) for all a in action space. note that best response may not be exist for all states of the game (there may not be one clear action that weakly dominates others) however you can acquire a mixture of actions (a probability distribution of actions) that weakly dominates all.
@@RamRachum this is why it is called *best response*. for simplicity assume that actions are discrete and assume there is only one move. players decide what they play simultaneously (without observing other's action). then there are A1xA2 possible outcomes for the game (A1 is the action space of player 1 and A2 is of player 2). search for if any a1 inside A1 is better than all other actions *whatever player 2 plays*. it is the best response for player 1. do the same thing for player 2. and the intersection of best response set of player 1 and 2 is the nash equilibrium. (if there is no pure strategy meaning that no single a1 that dominates all other actions regardless of what player 2 plays, then there may be a mixture strategy (combination of actions with a probability distribution)). for further information you can check the terms best response, mixture strategies and nash equilibrium.
A three-sided dice could exist, but not with geometric faces. Think of an intersection between 3 spheres, kind of in a Venn diagram / Nuclear hazard symbol configuration.
There's other ways to make them too, you can take a cube and fillet three edges in a way that leaves you with a three sided die (each side will no longer be a plane though, and I don't think that a 3 sided die can be made with only planar faces)
