I can't find an extension in the webstore that I think you're using. I think it's private, so it would be helpful if you could make it public. Is it possible?
It's a Google Chrome extension. Not public though. There are tutorials on how to make chrome extensions available from Google. With some Javascript experience it's not too difficult!
Hey! Am in love with your work, you seriously are doing something phenomenal and am here to watch it! I couldn't help but ask, and sorry if you have already addressed this before but have you thought about taking this outside the game? As making it a website that provides a photo-to-location service? It'd be quite revolutionary and will be a hit with no doubt!
I have an open source model and script released, I will be retraining it soon once I have access to more GPU memory. What are the average world scores of all of these models?
@@TraversedTV thanks and very impressive stuff, it's hard to gauge the performance from the small sample size in the youtube videos (have just discovered your videos on twitch)
Maybe we can have more livestreams of AI playing and learning? From time to time, because there was nothing recently, it always nice seeing some activity.
This would GREATLY benefit from attention layers. If you think about how RAINBOLT thinks, he only really cares about certain very specific features, the AI must do the same. I also recommend using a openAI's UNET architecture rather than the basic CNN architecture.
Do you share the source code on github or something similar? Did you also scraped the lng/lat data from the network request when downloading the corresponding panorama to the server or did you do it inside the chrome-extension and how?
Any chance you'd be able to open source some of your code for this? I'm most interested in the frameworky bits for talking to Geoguessr programmatically so I can play with the model part myself. I spent a little bit of time playing with it tonight, took a different approach on the data collection and am trying to use Selenium rather than a browser plugin. I am struggling with getting the camera point of view to pan around so I can try to capture several points of view as you did. I have something working by having it press the arrow key a bunch of times, but it's slow. I suspect you might have been clicking on the compass needle, but in the newer geoguessr UI the compass at the top can't be interacted with. If that's the case, have you found a way to do this with the new UI? Either way, awesome project, thanks for sharing!
Is this project open source? As a geoguessr fan and ML researcher, I would like to try improving upon this as a hobbyist project. Pretty sure that using semi-supervised learning would instantly yield significantly improved results given the currently very limited dataset.
Is the current network only based on the CNN? I'm always a big fan of combining the strengths of AI and classic methods. Can you also make a feature extraction and use that as input? This would be something like: we're driving on the left, so it can only be one of those countries. Sure, with enough data all of that can be learned, but maybe a few well selected features could cut down the amount of needed training data significantly. Nice job btw. :). This is my favorite GeoGuesser AI so far!
You should apply a GAN (generative adversarial network) to your CNN to make it more consistent and robust. This helps it generalize better, and may improve consistency.
Also I recommend increasing the width of your layers, even if you need to remove a layer or two at the end. A small layer can bottleneck the whole model.
I love this implementation - have you considered using the browser extension's JavaScript code to monitor/scrape the network traffic and download the panoramic image files when they are fetched by the browser? This could allow you to acquire the training data faster rather than by interacting with the screen and screenshotting? May also provide you with a cleaner, non-overlapping image. I also like the idea of classifying countries first as another commenter suggested. Great project, really enjoy seeing the improvement
@@supremezzzzzzz presumably by overloading the XHR request and whenever it's downloading an image that you need, save it locally either with the FileSystem API or by posting it to a local web server
That's a good idea but in the context of the game the idea is that both parties get to see "limited information" wouldn't that be considered unfair advantage when playing against a human if the ai gets to see the whole paranomic view but a human can only see by scrolling around? Would be cool to have both options available maybe
This is amazing. The first time I played Geoguessr about 2 years ago I thought "I wonder if someone has or will create AI for this and I wonder how". Then of course your AI play in the tournaments, etc. I actually first thought you used some brute force method of taking your program through every Google covered street in the world, reducing it for speed and then comparing the game image to your massive database. But that's because I'm no good at AI. I'm glad it works more like a human player works. Anyway, I really enjoyed seeing how you did this. I'm wondering what uses this might have outside the game. I first thought about geolocating for news networks (you know how CNN and others check if images sent to them were actually taken in the claimed location etc), then about military uses, then about police in crime investigations. Of course I understand pictures taken by phone will often not match those taken by Google's street view camera, but perhaps some version could move between sources of image. Anyway, I'm sure some other use can be found. Thank you for sharing your methods!
pro tip: take another image looking downward. may need some more coding but i predict it will greatly and quickly increase accuracy. its thr most important clue in the game.
What about training from a pretrained model such as efficient net? finetune it on this data, then run it 5 times with square images and make an average on these predictions. Also, will you kindly share the dataset please?
Very scary. A government could take a photo from your photo and instantly know your location within cell phone tower range... Thus even with your location service and data turned off, you could be found by police or military...
I'd be very interested to see a heat map of what parts of the image the AI extracts important features from. Maybe you could build it by blocking out areas of the image and see how it affects the prediction. (There's probably a better way to do that I haven't studied this since college) it would be dope if the AI discovered new meta that humans never found
Maybe there's some complication I'm not seeing, but would it not be easier to make it a regression problem in the coordinates rather than dividing the map into sections?
Being a big fan of regression, I'm thinking about that too, but I would assume that unless the model is really well trainined and already nails the region, the regression would be much less accurate. Take an example in which the model thinks of a place in the US and one in Europe, the regression model would probably end up somewhere in Iceland. Maybe a regression on the cluster of most certain guesses I'm the heat map might make sense, but it could already be doing that.
but there aren’t enough locations on adw so the ai would just know all the locations. i guess you could train it on 1 map and then play it on another map though, to see how it translates over
It would be really interesting to do some interpretability analysis of the network to see which features it is focusing on to make its guess. Could reveal tricks for human players to use!
Totally agree! I tried that but the results where very inconclusive... It's a bit tricky due to the relatively high image resolution. Will try again when I got time!
Does the plugin follow geoguessrs terms and services? Im asking because each time you create a new game an API call is made which costs money for the game.
