MENACE: the pile of matchboxes which can learn 

9:20 That suggests to build Menace A and Menace B - and have them both learn by only playing against each other

So, when I lose a game I can honestly say "I am dumber than a box of matches"

"Can a Match Box?" "No, but it can learn."

I remember that Martin Gardner article (I believe he published it in Scientific American) and I built this and played it as a teenager in the 60s. This was one of the first steps I took toward becoming a Computer Scientist. That was fun!

"This must be what procreating feels like." UM. Okay, Matt...

Matt 'chbox' Parker

This is secretly one of the best and simplest videos explaining machine learning

"I never thought i'd have a sense of pride over a sentient pile of matchboxes, but here we are." This line was great enough by itself, but he really perfected it by saying "This must be what procreating feels like".

well, i know what im coding tonight

MENACE, for when the machine goes first, and DENNIS, for when the human goes first

Menace doesn't die, it just learns that the only way to win is not to play :D

Corner is by the way the best opening move against humans because it's an unusual move. It's still a drawn game if played right, but people who aren't familiar has a greater chance of doing the wrong move.

Obligatory quote "The only winning move is not to play"

I'd say, start with like 4 of each color in each box, so it's harder to kill off routes early in development. It should learn a bit slower, therefore keeping it more fun at the convention, and it should end up knowing *all* Paths to Victory.

As a dad, I can tell you that procreation carries a wide range of emotions, with pride being a small part. Fear and frustration are much more common.

8:33 @Matt, that's kinda what programming feels like too! The satisfaction of your watching your theory autonomously running, and correctly... Bliss!

No, no, no... Use Tic Tac boxes containing differently-colored toes!

I remember in elementary school, thinking myself pretty good at the tic-tac-toe. But then a friend beat me with a corner starting move. I was quite amazed and have played with a corner starting move ever since. I'm surprised at the disparity between greens and blues in the starting box. Corner move is pretty awesome...

This is absolutely amazing. I love the cross-over of high and low tech and this is the perfect synergy.

Parker sentient beings.

You could also teach matchboxes to play Dr. Nim.

Back in the 1960s Reader's Digest had a "Book of Adventures" that had stories, puzzles, games and activities, all in hard bound. One of the activities was building a "computer" that would play "Hex-a-pawn." This was a game that used the nine square board (3x3) and three pawns on each side. The paws moved as traditionally and the object was to get your color in your opponent's home row. Like this experiment, you had matchboxes with the various board configurations on them and inside were colored beads to indicate the move. I came across this book in the 1970s (computers were becoming more of a reality by then) and spent a snowed-in weekend building the "machine" and playing the game. It was a lot of fun and taught me how programmes worked (basic anyway) and how a computer CAN make a mistake.

I love that it can die out. The way to win is not to play at all.

I love this! When I was 9 or 10, I got a copy of Martin Gardner's "Mathematical Carnival", which contains his piece about matchbox computers, and I was absolutely fascinated by it, though I never tried to build one. Forty-something years on, it still sticks in my memory -- I know exactly where I was (in a dinner queue at school) when I read it! It's great to see it in action. (Actually, I've been mourning for that book, unable to find it for years, and it's been out of print. Happily, a couple of years ago, an ex-colleague from my first job met my ex-partner, and returned it -- apparently I lent it to him sometime in the early 90s -- and I've very happily re-read it quite recently 🙂 )

8:32 "This must be what procreating feel like." Lol.

Ages ago I found a description of a similar learning pile of matchboxes from an old Soviet-time puzzle book. That game was different (a breakthrough of pawns on a 3x3 cheassboard), but it inspired me to make a tic-tac-toe version. I took rotations and reflections into account and didn't need that many boxes (only about 20, don't remember how many exactly); I also used a simpler algorithm where nothing was added, only in case of loss the last move indicator was removed (and if this emptied a box then the used move indicator from the previous box et c.). The simpler algorithm was, of course, worse, because it didn't distinguish between wins and draws (this feature was carried over from the original pawn game where draws were not possible), so in the end my fully trained machine mindlessly cruised into draw even in winning position. I think I grew bored before coming up with the idea of rewarding wins by adding indicators. A slight problem with this algorithm seems to be that it quickly becomes a fan of lines that have brought success. I don't think that corner opening is any worse than centre opening; one might say it is better (because it only leaves the opponent one non-losing move, while the centre opening leaves four in a way), but MENACE apparantly happened to score its first win or two with centre opening and this filled the opening matchbox with green beads, after which it, of course, started to open with centre move and kept scoring its wins with that, and so it snowballed.

How many matchboxes would be needed to learn Global Thermonuclear War?

Inspired by this and a previous video, in a fit on boredom, i programmed a bot to play Nim and let it go second 300 times against a perfect opponent, and the only reason it wasn't infallable is because i wouldn't let the probability of any move drop to 0. But with only 11 possible board states, it made for a very easy introduction into learning programs vs. trying to teach it 300-some board states and how to recognize reflections and rotations.

I might have missed this in the video, but I think an important thing to mention is that the initial state of the boxes _isn't_ one bead of every possible colour, but instead 8 each in the first box, 4 for the second moves, 2 for the third and one each in the rest (something which isn't even covered in the blogpost in the description…). The way Matt explained the setup would have a high likelihood very quickly dying out…

I feel like this is the machine that RU-vid uses for there adbot.

The one dislike is from the person who lost to MENACE.

ONE OF My MOST FAVOURITE VIDEOS ON RU-vid

“I am now joined by the guy who's fault it is!” This is the reason I follow you. Well, that and computers made of matchboxes.

If this really proves anything, it is that anything can "learn" how to do anything as long as it gets some kind of feedback from its environment. This, more than anything, is simple, concrete proof that intelligence and understanding of abstract things can arise from simple physical items and processes. ... We all are just a pile of matchboxes.

"It's learned to resign on the first move." So basically, all it's learned in that case is that it's bad at noughts and crosses.

I made one a lot of time ago. This is amazing as it demonstrates the very basics of what we call "Artificial Intelligence" or Machine Learning.

Now that Matt Scroggs "has" this contraption... He can't be blocked except by two or more creatures.

Great fun, I did this at school at the end of the 1970's also inspired by the brilliant Martin Gardner :)

Oh my god I was at the museum last week! I practically could’ve run into you!

Over 40 years ago, following instructions from a popular science magazine, I build a similar machine out of matchboxes. It was for a different game with less states than tic-tac-toe (so I didn't require that many boxes). But the learning strategy was different: wins were never rewarded. For a loss, you'd remove the bead of the last move where the machine still had a choice left (more than one bead in the box). This, IMO, is a superior strategy for several reasons: 1) You don't need an large supply of beads, and ever expanding boxes. 2) The machine will "die" if and only if the start position is a losing position. (And not "about 10%" as it is for Menace). 3) The opponents cannot cheat. With the learning strategy of Menace, you can manipulate it in making a bad first move by first, on purpose, losing a bunch of games. Once it has a fondness of a bad first move, you can exploit that. And since the rewards for "wins" (3 more beads for every move) are much greater than for losses (lose a random bead), for such a machine, it's much harder to unlearn bad moves. (It doesn't apply that much for tic-tac-toe where the machine goes first, as there's no bad move -- but you can exploit that strategy if you'd use Menace to learn second-player tic-tac-toe).

How many match boxes would it need to learn how to play Mario?

0:25 Yay! Katie Steckles from the Puzzle Hunters on Only Connect!

Don’t beat yourself up about it. Tic Tac boxes are transparent

* the best move if you are going first is corners (in which you can actually win most times playing optimally), and if you are going second it is the edge (in which you will draw versing an optimal player)

Thanks for the vid

1:16 "... the box that matches". I see a connection.

From now on Ill be counting things in "metric universes" xD

I saw this a long time ago in a french science magazine. Thank you very much to bring back this memory. I always thought it was from Von Neumann

It seems like a little part of you died when you called it "Tic Tac Toe"

Now I want to get back into my attempt at programming a neural network into a MUD engine... I mean, thinking in terms of my favorite (PennMUSH), I have rooms (containers that players and items can occupy) and exits which connect rooms. All object types have programmable attributes....and those could be weighted values. Such as "likelihood that this exit is used by a wandering object when it picks one at random" (a rat, maybe). But tic-tac-toe would be a much easier to start with...and the immersive quality of a MUD could make for some fun roleplay effects. A rat maze, however, would be way more inline with the dungeon crawler intention of a MUD.

The irony of it honing in on center moves is that if you know what you're doing, corners are much better, but I doubt this thing is capable of thinking ahead, since a poorly-played corner game is much more likely to lose than any center game.

It's interesting that MENACE learned risk aversion and tended towards the safer "draw" as opposed to the riskier attempt to win outright.

8:32 "This must be what procreating feels like" Oh Matt! I pity your better half.

Wow, this is, in a way, machine learning brought outside of the machine!! I am currently doing a project on Neural Networks for school and this fits so perfectly well with that project! It basically is machine learning! Love it, never thought it would be possible with matchboxes tho...

Is there a reasoning behind the rewarding distribution being +3 win, +1 draw and -1 loss?

Having watched this video years ago I was properly tickled when it came on QI this week, also presented by a math nerd called Matt 🤣

the only problem is every game of tic tac toe is a draw, unless one person is an idiot.

people knowing how to play and it being a solved game definitely makes it tougher. it would be interesting to see this sytem vs only children (people who almost never have strategy) or a version of itself that does the other side

if the first box runs out surely the solution is the put one of each bead back in and keep going?

The way I remember the learning algorithm from long ago was to remove the loosing move from the last box. Only when that box is empty remove the loosing move from the previous box recursively. That way it only prunes loosing strategies and the only way it could die is if there was a strategy for the second player to always win. Regarding using different flavors of tic-tacs the winner could get to keep the tic-tacs as a reward.

I wouldn't call those matchboxes sentient. The matchboxes simply store the learned information, the one doing the learning here is actually the human using the matchboxes.

As a future maths teacher with experience in museum design, this makes me itch to go get a big pile of matchboxes and build one of these stateside....

10:11 "Metric universes"

It seems like it would teach us a lot more about how the machine learns to program Menace B so that humans can move first. A deep analysis of the data generated by Menace A's centre-first strategy, compared to what Menace B does when the centre is left open by a human first move, might reveal some really interesting patterns. I think it could also learn very differently depending on how often it samples its results. Having its number of beads updated after _every_ player would respond in a different way than playing fifty games at a time and updating all those results simultaneously from the same start position. Or updating after every hundred games, two hundred, etc. Starting each box with multiples of each bead could also help smooth the numbers. A pre-learning state of having every option represented in triplicate might bring some insights. For example, those few anomalous blue beads in the opening-move box represented corner moves, and presumably led to some clever corner-based strategies that force mistakes. The machine might learn to give those strategies more weight if it has more opportunities to start with them.

HOW ABOUT A NICE GAME OF CHESS?

This is brilliant, both the principle and its use at a science festival!

tic tac would have been more interesting esp cuz you can reward the winner with a tic tac

I'm a security engineer but my major was in AI. I have done a lot of thinking about this project and how it works. I've tried to come up with ways of improving it and while there are several there is one I think would be the most helpful with both the software and matchbox version. It also gives clues as to how to write the same software for a more complicated game like chess. That is don't start with all the boxes, start with one. Many of the boxes and paths will never be used. There are lots of outcomes which will just not happen in any game. Start with one box, all blank. When the first move is made create a new box, each move after that will create new boxes. Now you have a smaller set to work with. For both versions this will give you fewer boxes to sort through making the game much faster. This will be absolutly critical when creating the same functions for a game like chess. Programing this for chess however is a different beast, you will want to add some AI functions in addition to the machine learning. Though with the above change you could do it with only machine learning. I am considering writing my own software version of this and seeing how few lines of code I can make it happen with. I may be able to do it in less than 20.... Also, maybe using something silly, like bash.

Not a Parker Pile of matchboxes then?

I believe Scientific American published an article on this around 60 years ago. But they suggested that you use a lot fewer match boxes by allowing for reflections and rotations. I believe only two or three dozen boxes were needed. And they suggested that when the boxes lost, the last bead selected in the last box should be removed. Nothing was added for a win. After loosing several dozen games to me, the match boxes became totally unbeatable if they had the first move. A lot simpler than what you are doing here. And it worked to perfection.

"It's 10 to the 27 metric universes across?" What about the old imperial universes?

favorite channel. keep up the good work. really love the mix of humour and information!

What happens if Menace plays Menace? Edit: Also, extremely sensitive to initial conditions!

I believe the phd student Matt was far underestimating the volume the match boxes needed to make a chess-menace. This is because a lot more options exist at each move, here you have at most 9 options per move chess gas sixteen pieces each side and each piece can typically go to several different squares, thus you would need larger match boxes!

but, if you start with 1-1-1 in each box, doesn't that completely erase an option upon losing? instead of just lowering the odds? Also, Menace going 2nd should result in more interesting results, since the opening move is a variable (humans don't always start center), so the countermove will have more variety and as such the result may vary more.

Love the music in this one!

"This is what procreation must feel like." 😂😂😂 Wow, this is one of the saddest sentences I've ever heard!

I remember doing this when I was a kid based on a Martin Gardiner article. Thanks for the memories.

"This must be what procreating feels like" -- Oh Matt...

I'm pretty sure that Martin Gardner did this in the 1950s, but I am glad that Matt Parker is keeping the tradition alive.

So you wanted to make matchboxes learn to win noughts and crosses but it only learnt to draw? That's a real Parker Square of a machine learning routine...

I love the derision with which he says "or Tic-Tac-Toe".

"If the first box runs out, it has learnt to resign on the first move, and that is Bad......" BUT Wargames taught us that is the correct move! ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-6DGNZnfKYnU.html

I can remember, in 1964, we had a publication called "L’Album des jeunes" from the Readers Digest Selection. There was a similar game. It was an "Hexapion". It was very easy to play even for a kid (like me) and a few (not a lot) of matchboxes. The principle was the same and it was very frustating (for me) to failed to win against a couple of matchboxes. At this time I was thinking that the way the "machine" was punished was not a real punition because this punition was helping the machine to win... And on my side I had nothing to help...

I thought, that the best way is to start with a corner.

I'm glad a video about machine learning was finally able to tell me how it is programmed to learn, at least at a basic level. (I know I could have googled it but I couldn't be bothered most of the time that it came up)

tic tacs learning tic tac toe tactics, has science gone too far?

Hey Matt I wrote a programm in c# which simulates your matchbox MENACE. Its mostly a replica of the matchboxes but I made some adjustments like a lower bound on how many different beed from each color stay in the boxes so that it cant die. I also added an auto-learn function where MENACE playes against himself and learns that way.

really disappointed neither katie nor matt said "link in the dooblydoo"

I feel quite strange that how much I find this video very entertaining. Excellent work!..

SOOOOOOOOOO, ....... at the end of the match,the inanimate match wins the match!!?!?!?!?!?!?! That's MATCHLESS!!!!!!!!!!!!!!!!!!!!! (& menace says ; "YOU'VE MET YOUR MATCH!!!!!!!") hahaahaaaaa

It is an interesting that they did it in a museum and maybe some kids and adults alike would learn a bit of how machine learning works. A simple search algorithm would be much more efficient for this particular game though

To be honest this made me truly grasp neural networks. Thanks

I always going in the corner was the best move, since unless they go in the centre you will be able to force a victory. Also, when they do go in the centre going in the opposite corner forces them into a non-corner move (otherwise you can also force a victory). It is these subtleties that allows one to beat a human in tic-tac-toe, so I think calling the blue fellas trivial is a mistake. Otherwise great job, very enlightening!!

Matt Parker for Doctor Who anyone?

Corner is better than center to start. Corner yields 7 wins and 1 possible draw on opponent's first turn. Center yields 4 won games and 4 possible drawn games on opponent's first move.

Why are Matt and Katie always together?

Re: the many people in the comments correctly observing that corner is actually optimal starting move - it is the optimal first player starting move IF you continue to play optimally, but with MENACE's initially random play I suspect center is more likely than corner to at least blunder in to at least a draw (since it blocks off the most possible opponent lines of any move), and since MENACE is going to be losing a lot of games before it starts reliably winning (seen in the initial precipitous drop in the chart) those early phases are going to select for the start that's least likely to result in a loss...

10^27 "metric" universes hahaha

It's the first time I've listened to drum 'n bass in 5 years. Thanks Stand-up Maths. I needed that.