NVIDIA’s New Tech: Master of Illusions!

Подписаться 1,6 млн

Просмотров 147 тыс.

50% 1

❤️ Check out Microsoft Azure AI and try it out for free:
azure.microsoft.com/en-us/sol...
📝 The paper "ViCMA: Visual Control of Multibody Animations" is available here:
research.nvidia.com/labs/prl/...
📝 The paper "Inverse-Foley Animation: Synchronizing rigid-body motions to sound" is available here:
www.cs.cornell.edu/projects/S...
📝 My paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Gaston Ingaramo, Gordon Child, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.at/~zsolnai/
Twitter: / twominutepapers
#nvidia

Наука

Опубликовано:

11 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 307

@waedi_ 2 месяца назад

Could be a lot of applications in video games, with controllable simulations and all.

@ET_AYY_LMAO 2 месяца назад

Not really, this doesn't work on simulations but pre baked animations.

@monawoka97 2 месяца назад

Bit difficult given the requirement for a fixed viewpoint. I reckon it's more useful in film, EG a realistic looking building collapse that just happens to avoid any debri landing on the protag.

@ET_AYY_LMAO 2 месяца назад

@@monawoka97 Not even that, it effectively doesn't change the animation, it just swaps around objects / materials at a convenient time where objects are overlapping or colliding and thus can be used for a slight of hand. Its a cool technique, but not exactly useful or groundbreaking. I think I need to read the paper to seriously understand the excitement, maybe there is some cool tricks regarding occlusion and parallelism of the problem solving..

@begobolehsjwjangan2359 2 месяца назад

no more ragdoll dead body in GTA

@Virsconte 2 месяца назад

Yeah, unfortunately it feels like the most useful application there would be manipulating player psychology, for instance with loot box mechanics.

@ThreeChe 2 месяца назад

The reverse engineered physical simulation to match a sound sample is the real magic.

@recube_games 2 месяца назад

I forgot that this is a papers channel and not an AI channel, great to see things outside that space once in a while.

@thedofflin 2 месяца назад

It's good to see a complete and reliable new technology, which can't be said for a lot of AI stuff which gets a bit exhausting

@sean748 2 месяца назад

Give me some rendering and light transport content 😭

@ChristopheLeyder Месяц назад

This the content that made the channel popular to begin with, the AI stuff become quite tiresome honestly.

@lbgstzockt8493 2 месяца назад

I honestly love that this isnt AI, sure AI is cool and impressive, but there is something artisanal about just using "normal" math to achieve this effect.

@animation-recapped 2 месяца назад

AI is normal Math

@Vorexia 2 месяца назад

@@animation-recapped You know exactly what they meant lol, no need to get pedantic over this

@jorgezombie78 2 месяца назад

How do you think a.i. is made, chemistry? It's pure "normal" math too. Why is everyone afraid of a tool?

@AlanMitchellAustralia 2 месяца назад

@@jorgezombie78Not afraid, but it's much easier to reverse engineer and debug "normal" code to understand why a decision is made (with neural networks, it is almost impossible). This implied certainty of "artisinal" systems gives them significant weight over AI in this respect

@animation-recapped 2 месяца назад

@@Vorexia to be quite frank I don’t know what he means. He is completely wrong for assuming any type of AI development doesn’t have basic fundamental math. Would you mind explaining to me what he means because I’m lost lol

@thedofflin 2 месяца назад

This will honestly be great for animators who want realistic physics but also want control over certain keyframes

@AurrenTV 2 месяца назад

Except not really because the technique is entirely smoke and mirrors. It uses camera occlusion to replace things with other things. In the case of the balls it's just using the motion blur to do it and is really obvious if you look frame by frame.

@jamesc5801 2 месяца назад

If the illusion works, it wouldn't matter

@catsnorkel 2 месяца назад

@@AurrenTV and why do you think that wouldn't be useful to animators? A scaled down version of this has been used by animators for decades.

@kawag2780 Месяц назад

@@AurrenTV For the non-animator audience who won't be frame peaking it is good enough.

@jimmy9607 Месяц назад

If it’s not realtime, you just bake the physics and color the balls on the last keyframe. That’s a very old trick. I was confused until he got to the point.

@Pyromma 2 месяца назад

~"All of these techniques are completely handcrafted. No AI anywhere. This is powered by pure human ingenuity. WHAT A TIME TO BE ALIVE!" Thats a sentence I never thought I would hear on this channel about two years ago. How times have changed

@MajorGunner 2 месяца назад

7:36

@theneonbop 2 месяца назад

I remember him saying this a few years ago.

@kwullums Месяц назад

i felt that "What a time to be alive"! So nice to sub to a channel excited about this stuff

@thelelanatorlol3978 Месяц назад

You guys realize he feels the same regardless of whether it's AI or handcrafted? It's all cool tech to get excited over.

@sollybunn 2 месяца назад

When I first saw the simulations I thought that the physics was being slightly altered in an unnoticable way, but upon hearing that it's just changing the apperance of objects I was amazed. For an algorithm which just changes appearances when it's hard to see, I'm not suprised it runs almost in real time without much effort.

@dairin0d 2 месяца назад

I actually came across a paper some years ago which was doing exactly that -- I think they were subtly tweaking the forces over the entire physics simulation to get to the desired state, though, naturally, their method wasn't quite as fast as this (it performed some sort of search over the possible evolutions of the simulation). I can't remember the title of that paper, unfortunately.

@IceMetalPunk 2 месяца назад

It's the appearance *and* the paths.

@sollybunn 2 месяца назад

@@IceMetalPunk ", instead of using motion control to estimate plausible motions for objects with fixed appearances, it can be more efficient to estimate plausible appearances for objects with fixed motions. Consequently, our method, ViCMA (Visual Control of Multibody Animations), is simple and practical. It carries out no complicated optimization or sampling, and, in fact, it only requires the animation trajectory to be generated once. The cost of applying the remainder of the algorithm is negligible." - research.nvidia.com/labs/prl/vicma/2023ViCMA.pdf

@sollybunn 2 месяца назад

@@IceMetalPunk "instead of using motion control to estimate plausible motions for objects with fixed appearances, it can be more efficient to estimate plausible appearances for objects with fixed motions." - the paper. Also the paper name ViCMA (*Visual* Control of Multibody Animations)

@qaesarx 2 месяца назад

Casino owners are like: WRITE THAT DOWN, WRITE THAT DOWN! 😀🤣🤣🤣🤣🤣🤣🤣🤣

@hians8734 Месяц назад

Gacha game: Yes YEs YEs

@crabbyboi9127 2 месяца назад

Nice to see a non AI episode ... for a change...

@zestyorangez 2 месяца назад

brings me back to what the channel was like 4-5 years ago

@Puppybarkk 2 месяца назад

0:40 me explaining how my algorithm works

@Rybz 2 месяца назад

😭😭😭

@davidoffberlin 2 месяца назад

Just 5 more years and Iwill buy a Graphics card lolol

@RandomGuy-nm6bm 2 месяца назад

relatable

@st20332 2 месяца назад

Come on software optimisation phase of tech, let's make the best of what we have through ingenious tricks and whatnots pls, not just simply brute forcing performance with hardware optimisation and "more and smaller", but I suppose software updates aren't as profitable, so who knows what the speed will go at

@filipezappe9312 2 месяца назад

The Hitchhiker's Infinite Improbability Drive😄. And it would be even more crazy if the pink balls ended at the top.

@wrOngplan3t 2 месяца назад

😄🤣

@M4rt1nX 2 месяца назад

You sir had me reading papers. It is really inspiring seeing these implementations. I saw the other day a video of the guy who created ipAdapter by implementing someone's else papers. What a time to be alive!!!!!

@and_I_am_Life_the_fixer_of_all 2 месяца назад

This is probably one of the most fun things I get to do, read papers and see what can I create with them

@MatiEP09 2 месяца назад

two years ago your content inspired me to go to computer Science. Thank you!

@noob19087 2 месяца назад

Glad to see some non AI content.

@bzikarius 2 месяца назад

I am glad, that NV knows, that artist need not just beauty, but artistic control over it. I know how to do such trick end-to-end, but here is a new way, i suppose. The trick is: 1. drop cards randomly. 2. swtich sides for end positions to form text 3. switch sides for start positions to form text 4. if end state and start state do not match, switch state in the middle. So now i will look what NV made.

@bzikarius 2 месяца назад

Okay, so nv did not invent anything new as idea. But the mechanic is tailored well.

@EAGLEBLACKInquisido 2 месяца назад

At 2:11 one can see the magic at work as around 28 balls (from right to left) a pink one on the third row up changes from pink to lavender.

@dynastyst 2 месяца назад

This is super interesting with the obvious limitation that it forces a preferred visual frame with occlusion rendering.

@besknighter 2 месяца назад

I went through the same process as when dumbfounded by a magic trick. First is awe ("Oh, coool!"), then confusion ("wait a minute... How?"), then anger ("How THE HECK is this doneee???"), then enlightenment ("Ohhhh, ok, that makes a lot of sense") to finally back to awe. It's such a simple concept but with really interesting implications and applications.

@adriankelly_edinburgh 2 месяца назад

Those dice were all loaded - opposite faces should add up to 7!🤔😀

@PranayVaidya007 2 месяца назад

Good to see you nack with papers related to 3d Graphics, between the mayhem of AI Muckery

@Clickadee 2 месяца назад

Does it bug anyone else that the dice pips aren't on the right sides?

@yfaitube 2 месяца назад

The minute I watched this I remember I did this exact same trick in After effects years ago. That was inspired by when I was at school doing calculus. Whenever I was stuck I would work backwards from the answer and just stitched to where I was stuck and hope they middle jump wasn’t noticeable:)

@Marco_zlb 2 месяца назад

Man I love how they figured out how the sound looked like. I am quite blind, because I can never see sound sadly

@kairu_b 2 месяца назад

Thank you for introducing these interesting researches as always, sir!

@Aestareth_ 2 месяца назад

felt like something was off in the movement, couldn't pinpoint exactly what it was though. it's like ai interpolation when you don't slow it down

@franciscomagalhaes7457 2 месяца назад

I knew I'd seen you talk about these simulations before. I just didn't remember it had been such a long time ago...

@masoudmaani 2 месяца назад

@1:40 YES! XD Thank you Dr. Amazing as usual!

@davidad660 Месяц назад

If you slow down the video to 0.25 speed at 1:06 you can see that a few of the balls change colour to match the placement. The ball in the centre right turns from blue to pink.

@teeoemm 2 месяца назад

I'm working on a game with a forced camera angle, could make for some amazing results!

@aphaileeja 2 месяца назад

Holy crap!! This plus time travel can change everything!! Perfection awaits!

@Neuer_Alias_erstellen 2 месяца назад

there is a verry easy way to achieve this: just by baking the simulation and apply the texture after

@3sc4p1sm 2 месяца назад

The small music video made me LOL

@Juttutin 2 месяца назад

Our simulations have been so successful we've lapped reality and come back out the other side.

@Jergling 2 месяца назад

Was I the only one who immediately saw the pachinko balls changing color mid-bounce? I know that the visibility function is never favorable in a sim where everything is visible all the time, but the color changes happening during motion blur turned them into multicolor streaks!

@piotrmorag2597 2 месяца назад

I saw some people on yt doing stuff like that in their simulations. YT short from Pezzza's Work channel is in my head when i look at this 😊

@yukimurabrify 2 месяца назад

That would be awesome if implemented on simulations created in After Effects, Cinema 4D and similar special effects softwares. As this would mean having much more control over the outcome of some of the simulated effects.

@WikiSnapper 2 месяца назад

This technique has been seen by many more than 100 people now! Thank you!

@sylfynfotho5522 2 месяца назад

Controlable laplace's demon, damn.

@michaelleue7594 2 месяца назад

I'd love to see an expansion on this where they figure out how to measure the degree of visibility or alteration which is necessary for certain tricks to be pulled off. This is impressive but imagine how much more impressive it could get if we approach the limit of believability instead of hugely overcompensating.

@federicocaputo9966 Месяц назад

If using a camera, figuring out if something is visible has been solved for quite a while. Most computing programs do not draw certain elements to save performance. For the balls, I think they just calculate the points of highest velocity, acceleration and or jerk to choose when to perform the swap

@MustacheMerlin Месяц назад

You should do a video on the RandNLA/RandBLAS paper. It's new algorithms that can do fundamental linear algebra on huge matrices 1000x faster on paper, (closer to 20x in practice but 20x faster is still insane). Things like least squares, for example. The basic idea is that for certain computations you can get an arbitrarily close approximation to the real answer by doing the math on a much smaller matrix made of a random subset of elements from the original matrix, instead of computing the whole thing. So they trade exactness for speed, and notably the tradeoff is configurable so you can tune it to be "good enough" for your specific application.

@mako9673 2 месяца назад

I was able to spot the changes in colord in the pachinko ball simulation. I am sure others did as well. It is still a good trick, but not imperceptible

@anto_fire8534 2 месяца назад

yeah i also did. went back, put it in 0.25 and just saw balls changing colors if they were too far away to were they should have been.

@johnprashanth 2 месяца назад

didn't realise that poker involved dice. what a time to be alive!

@asdf30111 2 месяца назад

621 balls and 621 bunnies? Oh my what a stimulating simulation indeed.

@ideallyyours 2 месяца назад

9 * 69, yes.

@notthatrkelly 2 месяца назад

There's a flaw in the logic of the system. Specifically, when the dice fall (at 2:05, for example), the system is seemingly told to take advantage of occlusion to change the top face of the die to a six and make sure that no non-top face displays a six. The flaw is that the face opposite the six on a die is a one. So we shouldn't be able to see any ones if we can see the six on the same die...but we do.

@ayatokzorro Месяц назад

I was really hoping that this was trying to exploit chaotic system. Like, there is a near infinite many ways cards could fall and arrange themselves on the ground, and so one of those possibilities is likely close to our desired image. So I was hoping that instead of simulating physics, it would "tweak" physics so that the desired outcome was guaranteed to happen, while remaining as visually consistent as possible to a person. That would be insane.

@scooterboom4157 2 месяца назад

I love youre videos man!

@TwoMinutePapers 2 месяца назад

You are too kind, thank you so much! 🙌📜

@ZombieRPGee 2 месяца назад

Watching this and I thought about special effect companies being able to finally control a simulation without just having to hope that the next generation looks better

@S1nwar 2 месяца назад

this is basically simulating Poincare recurrence time^^

@Viperzka 2 месяца назад

It's good to see that classic simulation papers haven't been entirely eaten by AI.

@harry3life 2 месяца назад

I never would have guess they literally cheat during simulation. I would have expected the simply do some reverse physics or something. Setup the final condition and find the initial condition to reach it.

@Bzuhl 2 месяца назад

Ok now we need a new round of trickshot context from Corridor Crew (An animator try to simulate a trickshot faster than another guy doing it for real)

@27Jaak 2 месяца назад

Is this similar tech to the one used by nvidia based on the earth clone model to predict the weather? Would this mean the closer our model to reality, the closer we are to predicting the future?

@intyrnet 2 месяца назад

This is how online gambling sites work

@anteshell 2 месяца назад

It's not. They don't cheat with visible visualisations. They cheat in the background where it cannot be detected in any way. This kind of animations can be pixel peeped rather easily to detect the cheating, which is not something they'd want.

@getsideways7257 2 месяца назад

@@anteshell That's obvious. Still, you can't help but think of those right away...

@getsideways7257 2 месяца назад

Also elections

@anteshell 2 месяца назад

@@getsideways7257 Also not in elections. Votes are not counted based on computer simulations, much less on fixed animations such as seen in this video.

@voidkid420 Месяц назад

Really doesn't and anteshell is also wrong. Online gambling is massively regulated with every single game being subject to audit etc If we say there's a 95.01% return and after millions of games it turns out to be just 95% we would get fined ... so no one's giving up billions in profit to steal your $1

@Metarig 2 месяца назад

You don't need AI for this; you can simply write some code that changes the object's color, texture, or look in some way as soon as it touches the ground to achieve the same effect.

@hiramhill1305 2 месяца назад

When he said he watched specific balls closely, I thought he was saying there were no replacement shenanigans. And the video of the simulated vehicle also made me think it could help the simulated vehicle go from point A to point B. But now I'm disappointed. I thought this was going to be a crazy physics simulations that took the beginning and end state and generated in-between states and worked backwards and forwards deciding chaotic variables like bounce direction and wind as is went, something no human would be able to tell is changing. And the technique was going to be some powerful algorithm that did this trial and error more efficiently than past methods. Like the next best thing after an infinite improbability drive. Now that would be amazing.

@mr.hi_vevo414 2 месяца назад

I seriously thought the trick was taking a start and end, then generating plausible frames in-between them until complete, sort of like how stable diffusion works with noise.

@rachelgilyard3430 2 месяца назад

Really cool stuff!

@TiagoTiagoT 2 месяца назад

Oh, wow, I was expecting this to just be some improvement over that, uh, what was it called? Differentialble Simulations? That one from a while ago where the manipulated the rounding errrors and stuff to subtly steer a simulation towards a desired result, stirring two liquids into an image or doing trick-shots with bouncing objects and stuff like that.

@zestyorangez 2 месяца назад

what an incredible video!

@emrys6895 2 месяца назад

Wait wait, the one where they input sounds and it outputs a video is NOT AI?

@blacksages 2 месяца назад

I think I have seen the ball switching colors in an ad

@Dimencia 2 месяца назад

I don't get it, though... there's already a simple way to do this, run the simulation once with the balls, then when they're at the end state, set their colors to look like whatever you want the end to look like. Then 'rewind' and run the simulation again (with the 'new' colors in place) and it occurs without any magic, a valid simulation that will reliably produce the end result I guess it'd get slightly more complicated with something like the dice, but doesn't seem worth making all this to solve it...

@planktonfun1 Месяц назад

I can do that too but with deterministic math, no AI or recoloring needed

@pandoraeeris7860 2 месяца назад

What a time to be AI!

@ET_AYY_LMAO 2 месяца назад

Honestly it almost seems like this video is totally AI generated. The script is way to excited considering what the whitepaper actually contains in terms of knowledge and useful systems. Basically just a handful of expressions for identifying suitable places to swap around the balls, and sure as soon as you have pre-baked this "score" for your pre-baked animation it is very easy to change the end result without doing a lot of calculations... To me it seems like both the script and the voice is AI generated on this channel these days.

@ramonessix 2 месяца назад

it's crazy how this paper got such few views

@Zifusax 2 месяца назад

2:07 Why is the 1 next to the 6 on the dice :D They should be on opposite sides.

@DougLJames Месяц назад

C'mon Dave!! 😂

@dannyarcher6370 2 месяца назад

6:46 - Holy shit!

@tschichpich 26 дней назад

I really love the handcrafted papers. AI is all great but we also need new algorithms we can really control not black boxes that do something

@rokasmisingevicius6770 2 месяца назад

what if this is how quantum particles work in a unit to create reality

@bengt-goranpersson5125 Месяц назад

Love your videos.

@user-ut2tk2yu6x 2 месяца назад

2:10 watched on 25% speed and the purple balls that get left befind turn blue at some point.. so yeah, they don't make sudden "moves" but they do gradually colorshift.. magic!

@RoniStudio 2 месяца назад

Casino: The trick I've been using is why it was revealed.

@drxyd 2 месяца назад

Very impressive and even better that this doesn't use AI. +1 for human ingenuity.

@Siderite 2 месяца назад

Sounds familiar. Didn't someone already did a simulation where things can go in any direction if they are not observed?

@DataJuggler 2 месяца назад

5:09 That's my favorite job interview discussion. When asked by a big company, how many programmers can we throw at this project, to get it done faster. My answer, 'It doesn't take 9 women 1 month to have a baby. It takes one woman nine months.'

@begobolehsjwjangan2359 2 месяца назад

it takes 9 men to get a woman pregnant faster. more men more probability.

@catsnorkel 2 месяца назад

just try explaining that to MBA bros

@wihatmi5510 2 месяца назад

I thought this worked without cheating. Just considering every possible outcome in a random experiment and computing exactly the possibility that leads to the outcome we want.

@SaumonDuLundi 2 месяца назад

What a time to be alive!

@randomarsh9817 Месяц назад

Peppers? WE DONT HAVE ANY PEPPERS!

@msmith2961 2 месяца назад

Cool.. but what is the use case? In what application would this be useful?

@epiicSpooky 2 месяца назад

Motion graphics

@mattgenaro 2 месяца назад

Master of entropy!

@l.halawani 2 месяца назад

captivating

@MaxHarden Месяц назад

Now scientists can get the outcome right of every experiment!

@Batmancontingencyplans 2 месяца назад

Someone just invented the final destination for AI agents 🤣

@carameljax222 2 месяца назад

Fated simulations

@DelIsInYourMind 2 месяца назад

I can see a new online casinos with "controlable simulations" not rigged at all.

@user-yp2ps3gn3x Месяц назад

Except that the dots on the die don't match reality. Opposite die faces, added up, equal seven. 6-1, 5-2, 4-3, etc. Opposite a six, the face carries a one. But I saw a one on a face ninety degrees from a six, so I at least know they aren't faithfully accurate dice.

@MrSongib 2 месяца назад

Actually good stuff.

@harrytsang1501 2 месяца назад

Hiding in occlusion and chaos, so if a tree falls where nobody's around, it indeed didn't make a sound and perhaps even never happened

@TewaAya 2 месяца назад

So fantasy/sci-fi media/games/shows could finally fix the giant cgi turtle textures in rising of shield hero. The cgi degraded by two decades when the turtle was in the scene.