Please like and subscribe if you want more videos like this! Project's website: belay.ai Regine's Instagram: regine_storaa Shot by Markus Skaane, edited by Sam Tuck. Magnus Midtbø tries Australia's national comp boulders (crazy hard)
This guy should team up with Moonboard, with set routes and an option to film yourself on the route, it could end up being a personalised coach in each gym
Although this won't be setting or giving real advice on advanced climbs, it certainly could do very well on something static and predictable like a Moonboard, and with many hours of video to train the AI with. Other than that it will take years and tons of compute to even attempt to understand bespoke sets.
Ah that's a good idea. Exactly the type of application that this could work with. I wonder if you could even add pressure sensors to the holds to get any other useful data. Maybe, maybe not. This is kinda crazy either way spray
@@Cyrribrae With how good the AI's ability to understand the visual data, combined with a basic understanding of gravity, angles, and folcrum, I'd say it could reasonably accurately predict how much pressure is being applied.
Hello, computer vision engineer here! Looks like this program combines an object detection model, pose detection model, and LLM together into a pseudo-coach. The model is likely trained in a fully supervised manner meaning the dev manually curated an annotated dataset of bounding boxes and poses from likely many RU-vid videos. As a coaching tool, unfortunately this will be hard to extend. The application does not appear to provide any added value than superimposing two videos and the LLM providing advice does not know about physics, hold quality, or body types which a real coach would factor into their feedback. Best to rely on human coaches for now.
Well, but this tool can give you extremely accurate movement distributions for every limb. Things like that is something no human can give with such precision. I think a tool like this can be expanded in many ways to give great insights into climbs. For one the ability to compare numerous results and automatically compile the best techniques for each move, like creating a compilation from the best moves out of many runs. This would get you closer to the perfect technique for your body specificaly with every attempt. I think the body type is factored in since the length of every limb is visible to the programm. I also think physics can be applied into such a tool, since it has movement data over time. There's tons of physical information in video, the software can in theory identify how fast things move and how limbs bend depending on speed on impact. There's a lot to this and the developers don't have to stick to the current models they are using.
Hello, also CV engineer here, think there is a lot of value to this system not directly as a coach but as a tool to objectively observe climbing and compare different approaches. I often have it while climbing that a friend struggles with one problem that is easy for me or the other way around and having an objective observer to tell you that you have a higher success chance if you move your hip differently or you need to shift your weight differently at this spot on the boulder would be great to understand what makes the difference between a successful and a failed attempt. How often is the only advice you can give while climbing to "just do it" because it looks exactly the same between climbers, this system can see the difference and give feedback. I won't have a coach while bouldering as it is a casual hobby, but just setting up my phone next the the boulder and getting some feedback would be great from time to time. The LLM part of the demonstration is the flashy bit, but in my opinion the least useful, as you said, the context of the actual boulder are missing.
Fair point. But that's the way to use AI in most cases for now. I work in as an IT Engineer in the healthcare world. We have a lot of ongoing AI projects when it comes to research. It's very hard to have an AI that can come up with conclusion from all factors. AI for us is mostly researched as a tool for doctors to use and ease workload. I think there will be a time when AI can fully replace humans in some cases, but i believe it's gonna be a while until then. I believe this tool on combination with a coach could provide great success.
I think it can be an interesting tool to study the kinematics of the movement and output some statistics, nothing groundbreaking however. Also, there's plenty of literature on human detection and skeleton tracking, one can just use a pre-trained model or other approaches not based on supervised learning. The LLM part can be useful to generate some considerations starting from the data, but nothing compared to a real coach, that, as you said, can consider a lot of other factors such as the dynamics of the movement. These are just speculations however since we do not know in details the functionalities of the software.
@@generic13372 Like he said tho, you can do that by filming the two climbs and superimposing the videos. You'd be able to see the difference on your own that way.
I can't wait for an AI to tell me how I can improve my sit-up technique when getting out of bed for the morning, so that I maximize the energy I have for the day.
Too bad its missing data, but if it records many climbers on known route difficulties it could also potentionally set new routes and also predict the correct difficulty for a newly set route, you could also track progression in your climbing with this so measure how good you are becoming, also would be fun to compare performance, so if you have pro climbers then you can see how you stack up against pros and what you need to improve in your climbing. I think this guy is onto something, but there needs to be real app development and LOTS of data filmed from different gyms and routes etc for this to be interesting. Its very early stage, but the potential to build fun stuff ontop of it is endless, I really hope he has the drive or at least gets a team together that work on a climbing app with one or some of these features.
The website has career postings for multiple positions and a less than 2 min video that showcases many more functionalites compared to Magnus' video. Once the app is released it could possibly make grading a lot more streamlined with users flocking in and feeding data. Interesting how it'll play out.
No, it can't set new routes because it is only dealing with the 2D physics of body position, not the angles of the wall; the size, shape and texture of the holds, or all of the other necessary factors for being able to set a route that matters. This is a parlor trick good for generalized positional advice...IF it is trained on that specific set with lots of video of people doing it. To have it set a new route that actually works it would have to understand the 3D shape of holds, wall, and body. Then it would have to understand friction factors of a particular hold, etc., etc. In other words, a long way to go without any data other than 2D body positions to work with.
@@FibberFinn Disagree completely. The only thing this AI will be good for is a cool app that helps beginners on known climbs, and generalized basic advice at best on climbs that aren't, so that leaves gym climbing out where sets are changed frequently. Not saying there isn't a use-case for some app to use AI to help with basics, that will be pretty cool in itself, but don't expect an AI like this to be able to understand much more than that. Self-driving vehicles are only now just starting to get enough data to be useful and they have way less sensor input than the human body has and uses on a v7 climb, say. The AI is only as good as the data it has to work with, and even though recent AI's can figure out basic physics on it's own, unprompted, there simply isn't any way of getting this to be more than generally accurate on a climb that doesn't have data of someone already climbing it.
Magnus, as a photographer/videomaker myself, I really apreciate the quality of your videos (filming, editing, sound... ). It would be cool to see a behind the scenes here in this channel!
Your gym looks ideal to me, Magnus. Huge features with dynamic, gymnastic, and technical climbing without a ton of small crimps that make fingers feel like they're going to explode. Sometimes, it seems like setters only increase difficulty by decreasing size of the holds. Hope your gym's setting style becomes more popular! Also, cool use of narrow AI.
It’s a cool piece of software, but calling it “AI” is just laughable; it is a data analysis tool that pipes its results into an LLM (sounds like ChatGPT) and feeds that into a TTS program. The bare minimum for something like this to be an “AI” would be having it predict the route or work off a single climb attempt.
I feel like that’s sorta just semantics. It’s like saying someone who made graphic art in Blender didn’t do it themselves because there’s a whole team who worked for years on Blender.
@@jamessmith4172 The right analogy would to take a finished blender picture and to add a few filters and claim "I made this picture". All the training and architecture is finished in open pose.
I think the concept of filming many attempts, gathering data from them and then using AI to analyze that data to point out the differences and similarities is cool and probably quite useful, but I'd be really careful when it comes to letting the AI draw conclusions from that and tell you what to do differently. It can sound really convincing and smart when it tells you to "move your right shoulder a smidge to the left" or whatever, like a coach would, but you have to remember that literally everything it "knows" is just the data you've given it through the video footage, and then the generative AI just fills in everything else based on probabilities.
It will tell you what you are doing wrong if you give it enough examples of moves done right and wrong... but as a climber you'll know better at that point anyways.
AI can analyze you as a climber, know your limits, strenghts and weakneses. Taken that in consideration witht the fact that it will self-develop more and more by watching more climbing videos, He can really know what would be the best possible move for you personally. Now even gym trainers will loose their jobs 😅
just like with anything else, it is more of a guidance rather than an absolute "truth" that must be followed. At all times, exercise good judgment and common sense.
@@Smixi35 Which should tell you why this isn't very useful for anything but Moonboard or beginner climbing. AI's are only as good as the data fed to it and this happens to be through 2D video right now which doesn't even account for the fingers, or friction, or a host of other necessary factors to be useful on anything bespoke and not set permanently, and trained watching climbers on that specific climb.
Awesome! As a person that is interested in motor learning researsch, I can say that the "angle and body position" data is not really valuable information to progress despite being interesting. On the other hand, the conclusions/summary that the AI takes can be really really useful if appropriate language is used! (not focusing too much on body parts but rather on movement's result) love this 😎
@@777Mikos This might be true for the basic beta of a route/boulder (hand/feet placement, sequence) but once you get to more technical stuff, demonstration tends to be useless because it's about adjustment or body tension you just can't "see" For example when technical moves require to shift your weight at precise locations/moments or when you need to engage your ankles to create body tension, verbal explanations is key However I dont see how this can transfer to AI because I'm not sure it can infer this kind of technical advices from visual data only
Do more hard climbing videos. I love this and honestly learn so much. Keep these super awesome climbing focused videos coming. Love your channel. You are the reason I started climbing.
Sick video dude, would love to see a some outdoor climbing vids on this channel! A France trip where you try Rainbow Rocket would be cool, proper Magnus style boulder
Having her there makes Magnus look even more superhuman. She's SO talented, but still can't send it like Magnus. Just puts into perspective how useless we'd all be in those same situations lmao
I had the privilege of not only visiting Norway last year, but also this gym. There are multiple floors you can Boulder on and a vast amount of top rope and lead climbs. The cafe had beautiful desserts and all the staff were super friendly. A wonderful gym that I know the community loves to have as a beacon for all climbers.
Forgive my earlier dumb comment. I deleted it, honestly didn’t think you would notice, and I’m touched that you responded. The second channel is great so far and I, among many others I’m sure, have been enjoying all of it so far. Please keep uploading (I didn’t actually sell my kidney btw)
i've had a similar idea but i am not a climber so i never really pursued it. cool to see someone is! my idea was to use 3d scans of the wall and a 3d actor to attempt moves and calculate the forces someone might endure to complete a move so root setters could have more insights into exactly what type of climber will struggle where
a bit of a waste , putting her on strenght things she does not have. didnt get to start or finish anything . her gigles started to anoy me.. i actualy wonder why she was in the video .. she must be a girlfriend of some guy who was there..
It would be interesting to see the ai applied to setting routes as well. Like giving to the ai the holds, set the grade and style like overhang or slab and the tool will return the route… I guess even easier for the moonboard and alike
i think this might be almost more useful for other sports, as a huge part of climbing is in the fingers and hands while the ai seemed to focus much more on arm body and leg movements. So thinking about olympic sports this might be incredible for certain disciplines
I've been wanting to do something like this for a long time, the only thing is convincing a gym to invest in and buy cameras for speculative tech. Best of luck to the dev
But every body is different in architecture (proportions, weight distribution, strengts and weaknesses) . You can't just compare limb positions and joint angles directly 💟🌌☮️
Indeed. It's a shame this wasn't addressed, because unless proportions and relative strengths are taken into account the numerical comparisons are quite limited in usefulness. That said, it's early days for people applying pose detection to climbing, and the relative numbers for one person over multiple attempts can still be valuable as a starting point (e.g. "this attempt your heel was lower and your left hand was higher on the piece, which worked better" kind of thing). Over time this will hopefully allow scanning in physical geometries (or at least providing a set of limb lengths), and ideally will eventually make use of stereo / lidar-augmented footage to treat things as properly 3D (including the wall and hold geometries and poses), at which point it can get much more powerful, and provide more meaningful individual and comparative insights.
have you considered integrating lidar data so that you can scan outdoor climbing environments as well. the algorithm would have better access to the three dimensional space that climbers are working with, and really taking it to the next level, understanding the physics the climbers are overcoming and integrating to send their climbs :3
This was fascinating, a really interesting use of AI. The potential efficiency gains alone would be valuable for competition climbers. That voice would annoy the hell out of me though, I think it should be John Cleese!
Magnus, you should talk to the company that tested peoples deadlifts last year at the Shaw Classic strongman competition. They had me do a relatively light deadlift on camera and predicted my max, the hardest point in the lift, etc.
This is very exciting to see in action! As a data scientist and an avid climber, this is a great example of how AI can help! Will definitely check out the website to learn more on how it works etc. Can see future use cases to help train people, almost like a pocket coach! Which would be really cool! Nice work
Possibly. I think it would all come down to how it’s trained. Is it solely looking at my performance on one boulder, is it trained from other attempts on the same boulder from other climbers or does the algorithm find similar climbs and give tips based off those moves? I would certainly love to try it and see how it works in real time!
Great video, but too short. I miss old Magnus, when an hour of entertainment was not uncommon. Don’t listen to those marketing fools, Magnus. Follow your heart, make longer videos. Peace ✌️
Would be cool to pair this with something like Kilter Board with repeatable boulders more people can access. Could even do a “climb like Magnus” type marketing.
Gotta be careful with the difference in climber sizes. Trying to climb like Magnus could be a good idea if you have a similar height and wingspan (granted, that's a lot of people), but even then, you won't have his strength. Maybe you need to do a figure 4 in a move where Magnus would "simply" do a one-arm pull-up. I guess it would be interesting to find a strong climber who matches your own size and style.
@@andeolevain that's a good point. I'm just starting out, and I'm an older guy (46), so if I even tried to do something like him, I'd probably rip a tendon right off the bone. This video was interesting though because my career has been boring indoors programming and high performance compute cluster architecture. So blending in computing analytics with climbing is super interesting! Especially so if we can learn more about safety and risk management.
Nice video as always Magnus! Do you know why they removed the normal ratings at your gym? I was so sad to see they removed grades and changed to colors😢.
yea, i think they should put a tag aswell that specifies the grade. Feels more satisfying than to just do a color. Especially considiring he is the owner, he should be able to decide @@boogaloo4640
Loving the climbing content but could the editor put the boulder grades on the screen please? Like Magnus used to. I like to know what I'm looking at but I'm even worse at 'guess the grade' than Branden
I wonder if the ai could potentially work for other things too. I was thinking like wrestling. It could allow you to see how your angle and is for something like a shot in wrestling.
at 12:55 there's a weird sort of optical illusion. When I first saw it I thought the camera was panning to the left making the left side of the wall come closer which made the undercling look inverted and my brain felt like it broke. It took me like five replays just to figure out what was happening.
we absolutely need more videos like this, bouldering, maybe with some additions to them, but just your climbing sessions help to progress on my level by spectating)
Honest question, does it make sense to talk about angles when comparing climbers with different heights? Aren't angles changing based on length of arms and legs?
