Can AI kill the greenscreen?

Подписаться 12 млн

Просмотров 584 тыс.

50% 1

Can a color really beat the AI revolution? For now, it looks like it can.
Subscribe and turn on notifications 🔔 so you don't miss any videos: goo.gl/0bsAjO
The greenscreen is a staple of visual effects - and it may stick around even in the age of AI “magic.” The video above explains why.
It turns out that greenscreens, while imperfect, provide certain background separation benefits that are tough for AI to replicate due to the way it’s been trained and the limitations of available data. Preparation can help improve results, but this video shows why, ultimately, AI tools will remain one in a suite of options rather than a greenscreen killer.
Further reading
www.cs.unc.edu...
You can find more of Soumyadip (Roni) Sengupta’s papers here, including links to his various greenscreen work.
segment-anythi...
If you want to try the latest breakthrough in image segmentation, Meta’s demo lets you upload your own images.
runwayml.com/
research.runwa...
Runway ML has a variety of tools that sit on top of AI infrastructure, allowing you to play with all sorts of applications. They’ve documented a bit of their process at the link above.
openaccess.the...
If you want to really learn more, you can read this paper that explains image matting on an important dataset.
Make sure you never miss behind-the-scenes content in the Vox Video newsletter, sign up here: vox.com/video-n...
Vox is an explanatory newsroom on a mission to help everyone understand our weird, wonderful, complicated world, so that we can all help shape it. Part of that mission is keeping our work free. You can help us do that by making a gift: www.vox.com/con...
Watch our full video catalog: goo.gl/IZONyE
Follow Vox on TikTok: / voxdotcom
Check out our articles: www.vox.com/
Listen to our podcasts: www.vox.com/po...

Опубликовано:

27 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 401

@Vox Год назад

Thanks for watching! Join us every Tuesday in April when Phil takes a look at what’s going on with AI. To watch more of our robot revolution coverage, check out our AI playlist here: ru-vid.com/group/PLJ8cMiYb3G5ek1Ux66aJ_qWf6CfBaAkGG

@blaircox1589 Год назад

@DavidLimReportyeah, as in next week. Commenting on what 'AI' cannot do is just about today, in the moment discussed. That's it. The public and media hasn't yet grasped the evolution rate of AI now. And still call it a pattern machine. They are talking about what the system was generations ago, when it started.

@seraaron Год назад

I think greenscreened footage will always be possible to spot until the technology can also match the lighting

@MatthewHoworko Год назад

AI relighting is in the works! I think Two Minute Papers has a video about it.

@SAM.MP4. Год назад

Clipdrop has an AI relight that’s pretty decent tbf

@Victor-kh5rh Год назад

Ray tracing is doing just that

@jeromealday614 Год назад

That's why they created The Volume

@maxpro751 Год назад

@@MatthewHoworko Hold on to your papers.

@Corloi Год назад

Why did they stop asking people to step out of the frame at the beginning of the call, like iChat used to do back in the day? Easy way to tell the AI what the background is

@doujinflip Год назад

UX geeks thought users want filtering to happen instantly and this extra step was too disruptive?

@Cimlite Год назад

That kind of approach doesn't work if someone moves their phone or laptop, either on purpose or accident. Then everything breaks down.

@wobblysauce Год назад

The lighting is the things that make it stand out to me

@izikavazo Год назад

The Filmmakers who are choosing to still replace backgrounds in The Volume and other LED rooms are part of the Hollywood VFX problem. They aren't making enough of their decisions before filming, instead throwing more and more of the responsibility on VFX teams to solve their issues.

@tessiepinkman Год назад

Yup, and they give the VFX team zero time to do it. They just scream "Fix it in post!" without having any idea what the words *really* mean. If all directors, producers and cinematographers tried to do what the VFX houses do for a month _(or six months would be perfect),_ I think movies would get better again instead of this decline of quality we've seen this last decade or so.

@angelgjr1999 Год назад

VFX are very overworked. It’s a very stressful job sitting in front of an editing screen for hours everyday.

@stalekoffee Год назад

@@angelgjr1999 Honestly as a VFX worker, sitting in front of the screen is the easiest part. The hard part is being able to turn around something high quality in short time frames. Some productions are good and allot enough time for VFX, but others are nuts and often exploitative or don't have necessary assets prepared. VFX artists can do anything 2d and 3d as long as they have enough time and money.

@ThePiggiestOne Год назад

Exactly! Change hair, replace hand, replace character with digital double, add weapon, etc. All requests coming up in the editing bay, en masse.

@timonelson6499 Год назад

just spent a day filming with a ton of rear projection... like Hitchcock would do in the 40s. But it looked amazing BECAUSE the lighting was accurate and the background image was meticulously crafted by an artist. My director made a huge point to make as many decisions before filming as possible and because of that the frames turned out so well. Greenscreen, AI, or even simple rear projection is a creative choice and it must be made in coordination with the story, the themes, the lighting, the sound, the cinematography in order to work well as an element of media art.

@FrankHuynh Год назад

Who else here is amazed/shocked how fast AI is coming together in our world 😲

@livia2495 Год назад

it's been developed for a long time

@mikebauer6917 Год назад

The real unappreciated change is the ability to store, share and evaluate millions, even billions, of gigabytes of data. RU-vid alone is something like 2 500 000 000 gigabytes of data.

@BarnabyTheEpicDoggo Год назад

Its always been there, the youtube algorithm, the insta feed, the advertisements on Facebook, it's just more visible

@Ottophil Год назад

@@BarnabyTheEpicDoggo you think those are AI?

@RegisteredHuman-pk3ed Год назад

I’m terrified

@Chemson1989 Год назад

Why AI hasn't replaced GoT S8 already?

@Vlow52 Год назад

There is a key difference that makes the comparison insufficient: traditional green screen has a permanent but static potential, while ai could be constantly trained until the perfect result will be achieved, but then you can apply even more features green screen can’t produce like changing the painted out parts to different things.

@MrKeenun Год назад

I recently discovered AI background removal in Filmora and I couldn’t believe the results, helped me with a poorly lit background that was obvious during jump cuts and I was able to replace it with a still image

@innerlocus Год назад

When I started in commercial art in 1976, we literally did 'cut and paste' photographic text.

@triniteemoore43 Год назад

wow you’re ancient

@innerlocus Год назад

@@triniteemoore43 Ancient, yet still learning as 10 years ago I went to school to learn Adobe software including PS. Vote BLUE 💙

@innerlocus Год назад

@@TheKing-qu8cm University of Minnesota in 1980, let me see what school could be like when teachers have earned their respect.

@RATSKETCHES Год назад

Mind blown 🤯

@travissobeck4939 Год назад

The Sheila definitely knows what it did!

@PogieJoe Год назад

Phil is such a great host. Always love his segments!

@lindenPRZ Год назад

Sheila Jr. will be missed.

@superafins Год назад

that very last sentence about artists having a lot of time in their hands has gotta be wishful thinking. as any VFX artist will tell you, more often than not the deadlines they're given are borderline unfeasible.

@omidfilms Год назад

Funny how the arts were once the riskiest fields to get into and now they’re the safest

@JeffPeppers Год назад

*if you're a specialist **and know how to readily and frequently get work :\

@JQMusic5 Год назад

Shiela junior deserves no sympathy. She knows what she did

@JaykPuten Год назад

I missed this episode of vox until now Phil's line "we can get rid of Shelia Jr, she knows what she did" made me laugh so hard I woke my wife from her sleep thinking I was having some kind of deadly problem I'm so glad nurses run on love of their job and *mostly* caffeine, because that joke was worth buying her an extra energy drink in 3 hours

@Estepario_Hesse Год назад

There is another detail to consider, and that is that the color palette used in the cuts must match the background lighting.

@RBFilmsja Год назад

The GOD box, unreal, and led walls are nice. But there is a difference in picture quality of items recorded, vs the quality of the walls image. They seem to be what’s in and the future.

@TheRealistMus Год назад

The AI field is moving way too fast. It needs to slow down a bit, until we can somewhat adjust to the consequences.

@Cris_the_coder Год назад

LOOL nonsense like if we can even control that. inevitable like computers getting better

@Soooooooooooonicable Год назад

Why do people keep saying this. My life hasn't changed one bit 🤣🤣

@WillyJunior Год назад

@@SoooooooooooonicableYet. Your life will probably look very different in 5-10 years due to AI.

@syndic3d Год назад

OMG! The Sobeys & Tim Horton’s at County Fair Mall in New Minas, Nova Scotia! Go Nova Scotia! 🎉

@ejoshcoron Год назад

Loving this series!

@5H4D0WOfficial Год назад

AI has grown so much, it's hard to keep track of them.

@StefanCreates Год назад

and it's gonna grow even more and faster

@CereaIkilla Год назад

AI is getting so smart so rapidly it's hard to keep up

@ShowmetheAItools Год назад

You can keep up with me. Don't be late!

@techcafe0 Год назад

don't be fooled, AI is not AGI, not even close.

@SreenikethanI Год назад

@elfrjz doesn't AGI have the capability to ... rewire itself to become ASI soon?

@angelgjr1999 Год назад

@Zaydan Alfariz I’ll give it a decade before most jobs start disappearing, I wonder what will replace capitalism?

@thePronto Год назад

It is smart like a bench full of tools is smart. Is it the tools, or is it the craftsman?

@RamiresHelena Год назад

I need to know what Sheila Jr. did 😂

@x4virom Год назад

I think killing the boring aspects of a process kills the process itself

@napoleonibonaparte7198 Год назад

Low budget films can become top tier with the right amount of effort at this rate.

@Clark-Mills Год назад

Rotoscoping with the rear-screen for tweaking is still a win as you will be less likely to have "fringes", the composition is in the can, reflections, speculation and lighting are a done deal.

@BakeBakePi Год назад

Great video! I cant wait to see how much these ai technologies improve over the next few years

@recurse Год назад

Sweet Jesus, that background with the Sobeys is in New Minas, Nova Scotia 😳. How on earth did you come to select that!?

@ryanortega1511 Год назад

Address?

@Borrowed_Rowboat Год назад

As long as profit and growth are the the primary motives in an enterprise, technology that helps a worker - an artist, in this case - do their work more efficiently and with less tedium will NOT free them up for more leisure time, but rather raise the standards by which their productivity is judged.

@JuanThaSilva Год назад

0:25 Masterpices. Mhm, horrifying.

@ALeprechaunn Год назад

On Dune they used sand colored screens instead of green screens because they knew they’d have to roto everything anyway and that way there wouldn’t be green spill in the shot.

@Sinnamonie Год назад

But wouldn’t that be harder to detect what is skin tone and what is “green”screen?

@sohanarahaman8070 Год назад

Man, this is really interesting to see honestly about this whole thing in the whole process of the green screen effect in a different AI green screen. The whole differences in that.

@JesiErin Год назад

You should look up the Lytro cinema camera that was abandoned when their employees went to work for Google. They made light field cameras that could calculate an object’s distance from the lens and remove background by thresholding distance. No green screen needed. I hope google does something with this technology but it seems to have disappeared.

@engchoontan8483 Год назад

The same algorithm to get the matte applied to enlarged portions and limited by clarity-resolution DPI of target video-output (not master) for real time performance. Focal-length is a clue Zoom and movements of previous frames are vectors Stereoscopic "view" from other LCD-panel technology as a quick analog guide

@engchoontan8483 Год назад

After what people experienced from news and actors, they will be even more sceptical and cynical about the usual-suspects and will demand proof for every sentence. The pathological-liars are toast

@ThisSteveGuy Год назад

They should try using switchable polarized lenses with those LCD screen walls.

@brettcameratraveler Год назад

Working on a hybrid green screen/projection technique. The flexibility in post of a green screen with the lighting and look of an LED wall. 1/100th the cost. Oh, and it's 3D.

@EditingPodcast Год назад

Loved this video. ❤

@ДжейДжей-и4з Год назад

I see a possible solution to the problem in the possibility of integrating the metahuman technology from unreal engine 5 into a real person in the frame. For example, to cut the outline of a person at a certain distance and replace this outline with the outline from the 3D model of metahuman, created by the person in the video. The same goes for hair. Replace the edges of the head with 3D hair that rubs against the movement of the person's head in the video. And the most interesting thing is the question of lighting. If you recognize the light from the backspace and illuminate the 3D model of the person, you can take just a layer of light and overlay it on the 2d person from the video to make it appear as if they are so illuminated. In short, if you create a 3D layer of the person, you can solve the green background issue perfectly.

@lukerinderknecht2982 Год назад

SHEILA jr™HAD IT COMING!!!

@shankpuppet Год назад

Amazing editing and explanatory powers!

@llgla Год назад

I remember the green screen tech in the early days had a similar problem where you'll see a green or white outline around the subject. I don't know what had improved to make the outline disappear.

@derAtze Год назад

Back then, in the early ages of digital cameras, the colour information was sampled in less pixels than luminance, basically it was a sharp black and white image overlaying a slightly blurry colour image. But since the greenscreen effect depended on the blurry colour information, not the sharp luminance values, so it was less precise. Modern (cinema) cameras sample as much colour as luminance, so that problem is gone. A good/bad example of these greenscreen fringes is seen in the StarWars prequels, since they were shot with some of the first professional digital cinema cameras

@unspecialist Год назад

@elfrjzanti aliasing isn’t a property of real footage, sometimes it’s better to have a basic grasp of the tech before trying to educate others

@unspecialist Год назад

@elfrjz I’m a high end VFX supervisor with 15 years of experience, I’ve used many softwares and I’m well known in the industry. Don’t be stubborn and get your terms right. AA has nothing to do with alpha channels. It’s a reformat filter.

@lindadawson902 Год назад

Timestamps & key points by Tammy AI 🔥🔥🔥Enjoy! 0:51: 🤖 AI's ability to separate foreground from background is improving through segmentation and learning from manually created datasets. 3:46: 🎥 AI green screens have limitations and are optimized for speed rather than Hollywood-level quality, but can be improved with the use of masks. 7:56: 🎥 Visual effects artists often replace LED walls with meticulously traced backgrounds for better quality.

@tomaccino Год назад

Why do you need timestamps on a fairly short video?

@lindadawson902 Год назад

@@tomaccino playing with new technology. 😄😄

@jwcas318 Год назад

@@lindadawson902 Thanks for sharing! Tammy AI is very useful!

@Drocks0 Год назад

Lost it at the robot shooting the kid with lasers

@jeromeyeo6110 Год назад

I am curious about the shadow and reflection effect of a green screen. I recalled seeing a reflection of the people on the floor in a talk show while actually, the background studio is all green including the floor. How did they do it?

@DanielRieger Год назад

If the floor is reflecting the green, it can be replaced just like the actual green screen

@jeromeyeo6110 Год назад

@@DanielRieger I see. Thanks 😁

@benhutchinson9054 Год назад

What did Sheila jr do?

@hannuback Год назад

Wait, wait, before me move on, I think we need to know what Sheila Junior did...

@HjRobins Год назад

I feel like the best method would be to have an LED wall with a really fast refresh rate and a global shutter camera recording at high speed, so then you can have some frames be the subject in silhoutte. Of course this will require interpolating the frames to perfectly line up with the non silhoutte images, but ideally it'll be so quick that it's only visible to the camera so that epilepsy is ruled out.

@HjRobins Год назад

I know they've done similar things with lighting on the Matrix and recently on Thor 4 but to be able to dynamically keyframe changes in lighting in post.

@DanielKaspo Год назад

Sheila Jr. catching strays 💀

@omkar_sawant Год назад

Awesome video! Why is a Verge video on Vox though?

@lvedder99 Год назад

Saw this on my school board and it looked cool so I’m gonna watch it later

@Lawliet734 Год назад

AI can't figure out where a long-haired cat ends, and neither can I!

@davadh Год назад

If you worked in VFX, then you know... you'll still need to rotoscope everything 🤣

@darkfent Год назад

That last part, even tokusatsu had started using it these days, most noticeably in KingOhger

@Sanny987 Год назад

I disagree with the final conclusion of this video. AI does not mean less time working; it means fewer people will be given more work with higher output expectations.

@chingwu8366 Год назад

After watching a tiktoker showing how green screen people are doing, I believe there's a long way to go before AI can really replace artist.

@KizulEmeraldfire Год назад

5:43 - "… This is Cris Valenzuela. He is the CEO of this very popular A.I.-focused editor called 'Runway'. "Their green screen tool is *one* way of solving this A.I. green screen problem …" No, no: it's RUNWAY of solving this problem! :D I'm sorry. ._.

@connecticutaggie Год назад

On a more general note, one other thing that I was just thinking about was the movie Avatar: The Way of Water. 100% computer generated so no post production, right. NO! My nephew is a VFX Artist and he was really busy just prior to the release of the film "fixing things". There are some things that only the human eye notices and only a person with a really good eye for that (and artist) can fix.

@henri-julien Год назад

uhh what do you mean no post-production lol it would basically be all post-production

@connecticutaggie Год назад

@@henri-julien I guess the main questions is "what is production for an animated movie". O think of the production as the producing of the main video content - which, for an animated movie is the animation.

@tecpaocelotl Год назад

I remember a ps3 game called movie scene or something like that that allowed it to remove the background without a green screen.

@silverhawkscape2677 Год назад

Such a Beautiful Picture of a Wife and her sister. ❤

@RiaRua Год назад

Give it a decade... as long as there's money to make, these things will get exponentially better.

@alvinthearchitect Год назад

“She knows what she did” 💀😂

@CaptainHoers Год назад

Oh, Vox. Let's be real. It's not going to mean more time with your family, it's going to mean being made to produce twice as much in the same amount of time. That's how capitalism works.

@Fugitive292 Год назад

Real smart video, detective Gordon.

@voideric Год назад

Just a correction here, it's "less time removing green backgrounds, and more time editing other stuff", or maybe "less time squinting at screens, and more time unemployed"

@israelimc14 Год назад

Exactly as you mentioned the future for roto/alpha depends on learning. The process is soft key, hard key.. These are what can preform on industry Ai based already on nuke, silhouette.. Which have tons of data already. What surely need to keep in mind some finaling always need to happen

@allisonfernandes4958 Год назад

Integrate them all together why not, just so green screens along with the other top two (including artist)

@Ohio_Greg Год назад

Great video!

@thomasalford7678 Год назад

Lollll “she knows what she did” 💯 😂

@jensenraylight8011 Год назад

this is the only time where it's okay for AI to help human doing tedious stuff, we should prioritize making AI that help not replace

@Nindota Год назад

You can normally tell when a green screen is used by looking closely at the hair as it's one of the hardest parts to get right.

@SuburbanPhilosophy Год назад

Davinci Resolve already did it in v18. also, instead green screen you can use green lights and a white wall :)

@Ziqver Год назад

Ai should not be compared to photography, filming, painting, creating musik, writing or anything an artist do. Ai is not filming, painting or composting. So no, it can't paint a masterpiece. It can only generate and imitate from an information that already is there. Therefore, AI should not compete with humans artwork. Just wanted to clear out the confusion most people have.

@professerutonium Год назад

there is NO COOLER background than the New Minas Sobeys

@shaikhaali7447 Год назад

Loving all the AI videos, Vox!

@concretely Год назад

What a great example to look at in order to learn about how machine learning works!

@angrymp40 Год назад

Sheila hits 😂😂😂

@kepipek6598 Год назад

Makes sense.

@savagepro9060 Год назад

Green Screen: AI sits on my matte at my doorstep

@ethyrice Год назад

the term is “Etheric Static”

@DC-cm3tg Год назад

You know whats gonna happen is not more family time but more work

@bj124u14 Год назад

what about combining the two

@brainwithani5693 Год назад

Been wondering this myself

@digitalzealot7026 Год назад

I love all the recent videos 🤗💛

@MyBroSux24 Год назад

So where does things like Google Pictures "Magic Eraser" feature fall on that graph? Does it work well?

@thokozanimanqoba9797 Год назад

The lighting gave it out.... Bad lighting that doesn't correspond with the background will always give it out

@mikebauer6917 Год назад

Solution; baldness or hats for all!

@SourishBasu Год назад

It already has, runway ml

@DodaGarcia Год назад

Does any of the TikTok green screen filters allow for a difference matte, as in, it learns the background before the subject steps into the frame? That solves the laptop issue.