Hand Tracking with Unity XR Interaction Toolkit

Подписаться 35 тыс.

Просмотров 69 тыс.

50% 1

I was waiting for this for a long time and here it is ! Hand Tracking is now possible with Unity XR Toolkit ! In this video we are going to learn how to setup a new project using this feature and be able to use the interaction system with it !
❤️ Support on Patreon : / valemvr
🔔 Subscribe for more Unity Tutorials : www.youtube.com/@ValemTutoria...
🌍 Discord : / discord
🐦Twitter : valemvr?lang=en
👍 Main Channel : / @valemvr
🔥 Tiktok : / valemxr
····················································································
📝Best VR Assets ( These links are Affiliate links which means it costs the same to you and I get a small commission. Thanks for your support!)
VR Interaction Framework
assetstore.unity.com/packages...
Hexabody (Physics VR Player)
assetstore.unity.com/packages...
Mirror and Reflection for VR
assetstore.unity.com/packages...
Auto Hand (automatic hand grab pose)
assetstore.unity.com/packages...
Hurricane VR (vr physics interaction)
assetstore.unity.com/packages...
3d Hand Model for VR
assetstore.unity.com/packages...
····················································································
Full Series on How to make a VR game • How to Make a VR Game ...
····················································································
If you enjoyed this video, here are some other really good channel you should appreciate :
⌨️ Game Dev
Brackeys : / @brackeys
Dani : / @danidev
Code Monkey : / @codemonkeyunity
👨‍🎓 Vr Dev
Justin P Barnett : / @justinpbarnett
Vr with Andrew : / @vrwithandrew
····················································································
00:00 Intro
00:56 Setup
06:48 Hand Visualizer
11:31 Hand Demo Scene
13:49 Testing
15:48 Outro
#vr #vrdev #madewithunity

Опубликовано:

27 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 159

@MuddyWolf Год назад

FINALLY! Thanks for sharing Valem! This is so cool!

@flym0 10 месяцев назад

You bloody star mate. Working perfectly.

@lalaith_music 3 месяца назад

You are saving my thesis

@1Chitus Год назад

Great explanation and clear thank you!

@CactusVRstudios Год назад

Love your tutorials!

@karolledzinski710 Год назад

So Gooood update on XR TOOLKIT. Thanks Valem for tutorial 🤘😁 btw nice hairstyle 😁

@ValemTutorials Год назад

Aha thank you :D

@axs203 Год назад

Thanks Valem!

@ricardojaviergarnicapena6749 Год назад

Excellent video. Thanks

@Beno951 Год назад

Finally! Now XR toolkit actually feels complete.

@A_walk_in_the_park Год назад

It would be nice if they would enable full body with hand tracking. I don't know if you can do that now, since body only follows controllers.

@vinigamedev4158 11 месяцев назад

Hands don't appear in Windows PC builds.. so.. no

@Balakademi Год назад

Thank you. Just in time

@_virtualbeings_ Год назад

Fantastic tutorial Sir! Not only did you help me make handtracking work, but by following everything I also got Oculus Airlink to work again. The trick was to switch to OpenXR. Goodbye Oculus plugin, you've become obsolete and we won't miss you. (FYI, I used Unity 2022.2.17f (latest version), and it worked identically to the tutorial.)

@vroomescape3417 5 месяцев назад

We won't miss it, but would be great to know to do hand pose grabing and physic hands with the OpenXR! 🙏

@adityapatel6438 Год назад

very cool tutorial bro

@rklaus7732 4 месяца назад

Awesome!

@oliverhinox Год назад

I love Valem Tutorials😀😀😀😀

@oliverhinox Год назад

your tutorials are super easy to follow. your chanel is the best way for my vr games to go to the next level. please keep posting videos.😄😄😄

@musikb.s5106 10 месяцев назад

thank you very much!!!

@hiimkarl Год назад

now this is cool

@dkfactotum Год назад

Finally!!

@lordzepticon Год назад

I added the hand tracking stuff to the "XR-Interaction-Toolkit-Examples" project. The materials at the end was already set to standard shader, bit still pink. I had to set them to "Universal Render Pipeline/Lit".

@SingularityXRS Год назад

LETS GOOOO

@nitsnbolts1 Год назад

Great tutorial as always :) would be nice to know how to detect different hand gestures through code

@soya8983 8 месяцев назад

I also want to know how to do that.

@GamerCreacher Год назад

JUST YESTEDAY I WAS WONDERING IF I COULD MAKE HAND TRACKING

@bradfordfermin4688 Год назад

Thank you so much for the tutorial! Is there any chance of you doing a tutorial on enabling passthrough with this openXR hands setup? I can't seem to get it to work no matter what I try

@jundakong4515 10 месяцев назад

Nice tutorial! Thx a lot! One question, how did you test it through Unity Editor PlayMode, was that from QuestLink?

@lucutes2936 3 месяца назад

12:01 i love that energy

@SingularityXRS Год назад

I literally have been waiting for this!!!, I really don't like oculus SDK

@karolledzinski710 Год назад

... We just need full IK body tracking add to this and nothing else is needed so far. 🤘🤘🤘

@tuongngo1749 Год назад

Can you create some tutorial about hand poser with this stuff? Or we can use the exact logic with your previous video about the custom grab hand pose?

@Bayonora Год назад

You need a tutorial about the Buttons in the Hand controllers ( B, A, Y, X)

@saumya3470 Год назад

hi, can you make the next video about making a custom scene like this? also did you have a chance to look at TactGloves by bHaptics?

@ebenknot Год назад

👏👏👏👏👏

@SeanBannister Год назад

For anyone unable to get the hands to display when you place your controllers on the table, make sure that the Oculus app is set as the default OpenXR runtime and not Steam. This caught me up for hours. You can change this by opening the Oculus app going to Settings and changing "OpenXR Runtime".

@Kitchwamon Год назад

Thank You So Much!!! I was going bonkers looking for this

@A_walk_in_the_park Год назад

How would you go about setting colliders for these hands? I don't want them to go through objects...

@emiliotube01 8 месяцев назад

such wow

@pastuh Год назад

I was in AI world for some time.. time to implement this future :D

@halbzwilling Год назад

Hey Valem! Thanks a ton for your help! I have a problem where I my hands don't show up in the Build! (I updated an older project and might have some other issues) Is it mandatory to setup the android settings or was that just to show how it works on both platforms?

@halbzwilling Год назад

Update: i enabled developer options in the oculus settings and it started working in the build 🙌

@ashtonpan7618 Год назад

@@halbzwilling Do you mean work in the .exe file build ? Cound you tell me the version of your Editor ? I am stuck on the handtracking in PC build

@Syertim Год назад

this is awesome! Anyway is someone else experiencing a strong input lag when using the hand tracking? Is it normal?

@Syertim Год назад

hey Valem, would be cool to see like a "virtual desktop" tutorial with XRI 2.3 where you can interact with your desktop with hands tracking!

@fgd190722 9 месяцев назад

What do you suggest to have laptops for doing development for VR

@jbjbjb_ Год назад

Hey Valem Tutorials, thank you so much for all the tutorials! I do have a question, though. If I want to drag an object (possibly a very large box or a model), how can I do so? I think this is a bit different from grab, or can similar mechanisms be applied to this situation as well?

@jbjbjb_ Год назад

Grabbing just hold the object on top of my hands 😅

@MarkRiverbank Год назад

@@jbjbjb_ XRGrabInteractor should work, just set it to use a dynamic attachment point.

@unclefeather22 9 месяцев назад

in case you get an error when importing the xr hands, specifying to import versions 1.1.0 instead of 1.3 of xr hands might fix it

@SoundGuy Год назад

Is there a way to have the hand model display on the controllers when holding the controllers?

@shinzor Год назад

How would you make a convincing rollercoaster like in Switchback VR?

@Draxi_1 Год назад

OpenXR is finally usable for something. It just lacks controller models.

@flym0 10 месяцев назад

@10:24, is there a way of turning off the controllers' mesh and have just the hands?

@pamford45 Год назад

Is there any way to detect a grab pose in order to grab and throw objects, like a basketball?

@devinstewart1743 Год назад

Is there a way to use Meta passthrough along with this solution? I tried using the new Oculus Integration Package along with this to enable passthrough but it seems to break switching between hands and controllers with the Complete XR Origin Hands Set up prefab. Any suggestions would be great!

@ERICA-eu5xv Год назад

Hi, Is there a difference between using the xr toolkit and using the Oculus integrated package?

@dumbfoxmedia 8 месяцев назад

Any idea if this works for Quest 3? I've followed the general setup from this video and it doesn't appear to be working. I don't have a Quest 2 on hand, so I'm trying to determine if it's the versions of the packages I used or if it's simply not supported.

@isabelleus2881 Год назад

I have issues with the Openxr plugin said to not be selected as plugin provider when I press play, while it was selected in the project settings for both desktop and Android. I tried to make sure I had all three packages (up to date): Input System; XR; XR core utilities (I didn't have this one); and ofc XR Plugin management. You can find this info under the hands package dependencies. It didn't work. Valem, do you maybe know what is going on?

@TheLaterPixel Год назад

Hey Valem! thanks for the tutorial, it was very informative :) I noticed an error occurs when i switch from hand tracking to the controllers where it says "could not find active control after binding resolution" any tips on fixing this error?

@yorukaze1234 Год назад

Seems that this error only appear in Unity Play Editor Mode. When I build my game in Android, side loading it into my Oculus Quest 2, and open it in Quest. Everything works fine and I can easily switch between controller and hand.

@ashtonpan7618 Год назад

Same issue with you, and my editor is paused when poping up this message ... 2021.3.20f1 XRTK2.3.1 XR Hands 1.1.0, OpenXR Plugin 1.7.0, Oculus XR Plugin 3.2.3, XR Plugin Management 4.2.1

@Miquel_Angel_PB Год назад

I have the same error

@vinigamedev4158 11 месяцев назад

same here

@nrv_maksim5964 Год назад

Hi Valem, can you tell me something about UltimateXR?))

@lonelanrete Год назад

can we use teleport locomotion with hand tracking?

@iSilver69 Год назад

Can you use Auto Hand with the XR Device Simulator in the XR Interaction Toolkit? I can't get it to work.

@travist1169 Год назад

A good idea for a new video would be to show how to do a full hand grab!

@wanglinda2212 7 месяцев назад

I understand XR Interaction2.3(Complete XR Origin Hands Set Up) physical Hand as XR Hand,same thing, right?

@user-pi5wb4yg1z Год назад

Cool! But there is some deviation when my right index finger touches my left hand. How can I fix it?

@EvzieBoii Год назад

Hi Valem, is there any chance you could show how to use these new features along with Passthrough from Oculus integration?

@danieldubinko9561 Год назад

Agreeeeed

@empressvr Год назад

I'm currently attempting to put the two together, let me know if you're interested in being kept up to date on what I find!

@blitzkreig4887 Год назад

So it can read input. Then why exactly do we need headset plugin support ?

@erasmobellumat3973 Год назад

Valem, i need help. How to detect if a finger is closed? I'm working on a game that will uses a virtual guitar hero guitar inside VR!

@AnythingSoupVODS Год назад

Hello, I am currently running the openxr 1.7.0 and it won't load the OpenXR tab below the XR Manager. I keeps spitting out an error about the Meta Quest Pro Controllers, I've been stuck on it for a while and can't figure out what to do. Any advice would be great! I'm using Unity 2021.3.23 at the moment Edit** I found that going back to an earlier version fixed my problem, great tutorial btw

@TheSebi0707 Год назад

Are hand poses already usable?

@pietro8648 Год назад

Hi, I'm making a VR project for my university. My first idea was to do something with the controllers but then I saw your videos about hand tracking. Do you think hand tracking is capable of different actions/interactions like a controller? Or is it limited? What would be some good practises to adapt a VR game to a fully hand tracking VR game? Like menus, exiting the game, movement, adapting buttons, recognizing hand gestures/poses?

@NanashiNokaze Год назад

it can be very good if you use pose tracking to trigger events. simple gestures can be tracked and used to trigger functions through unity's events/Actions system. it's not as accurate as I'd like but it's enough to move virtual controls when you pinch or close your fist etc.

@soya8983 8 месяцев назад

@@NanashiNokaze How can I achieve that? I want to use hand movements to achieve continuous movement and teleportation.

@NanashiNokaze 8 месяцев назад

@@soya8983 basically I used the meta hand tracking package and tracked the bones, and compared the detected positions to an array of stored positions over time. It the bones matched a gesture I had saved, it fired a method that read classes to define what to do.

@NanashiNokaze 8 месяцев назад

@@soya8983 if you'd be interested in hiring help I do contract programming work in unity with c#.

@aureliopuertamartin3631 Год назад

Can we detect hand poses with this package? For example grabbing an object with the fist closed instead of just using a pinch gesture?

@louisviel2915 9 месяцев назад

Hello, Don’t know if you still care or have found a solution to this But from what i’ve seen, it oNly recognise pinching wich your fingers by default. You will have to implement yourself a logic to recognise gesture, or go into an asset store plugin

@1979zeesh 7 месяцев назад

i can not see Origin property in Hand Visualzier script...i think because of this when i put down my controllers, hands dont appear

@najwamohammed3961 Год назад

Dont forget to turn on floor on the xr origin if ur controller doesn't move

@evFPV Год назад

Hey Valem! Great tutorial! In my case with the "Complete XR Origin Set Up" from the "XR Interaction Toolkit" doesn´t show my hands, I see the controllers all the time :( But when I did the first step, using the hand visualizer script, I could see them. Any idea what is happening? Thank you!

@philippdaube6047 7 месяцев назад

Did you activate the auto switch from Controller to Handtracking in your Quest Settings?

@trainingdesign9786 Год назад

I have thw the following error "Hand Tracking Subsystem not found, can't subscribe to hand tracking status. Enable that feature in the OpenXR project settings and ensure OpenXR is enabled as the plug-in provider." I have the subsystem enabled as it should be and I dont see any hands

@user-xd9gt6pc6d 11 месяцев назад

Hi, when I use hand tracking and close my left eye, my hands display the wrong position. I found the problem. OpenXR: Render Mode: Multipass

@ItsBixbyBoo Год назад

Will it support index?

@user-ej2cs4nf2g 7 месяцев назад

I have only headset of Oculus Quest 2. Is it impossible to use handtracking using Only Headset? Should I have to have controllers?

@karandhir2411 Год назад

Does this work for Pico or only oculus?

@user-mi1vv1ih6b Год назад

Can Pico4 or Pico3 do this things?

@haroldvicknair1989 Месяц назад

I have tried everything, but I cannot get this to work, I am using the XR hand 1.3 and there is no slot for XR Origin. I have my Oculus app OpenXR Runtime set to MetaQuest. I have tried a new blank project; I have tried the VR Core template. I cannot get this to work no matter what I have tried. Has anyone else got this to work with the newer stuff on a Quest 2?

@FlockofSmeagles 8 месяцев назад

Wait. So we don't need leap motion anymore?

@iliyasmarat7610 10 месяцев назад

help me please everything works fine just doesn't work for pc versions

@yorukaze1234 Год назад

Anyone knows how to solve if error : "Could not find active control after binding resolution" UnityEngine.InputSystem.InputManager.OnNativeDeviceDiscovered (int,string) appears after trying to switch from hand tracking to controller in run time?

@TheLaterPixel Год назад

running into the same error ye

@travist1169 Год назад

Same here

@vinigamedev4158 11 месяцев назад

I couldn't make it work in a Windows .exe Build. Any thoughts?

@nwy_0186 Год назад

can this be use in pico 4?

@karolledzinski710 11 месяцев назад

Open XR version 1.8 is not compatiabile with XR Hands it shows error "Hand tracking subsystem not found or not running. " This XR Hands still cause a lot of trouble to set up... Planty informations missing in this tutorial like Left Hands Stabilized... XR Input Modality component doesn't work at all... Generaly all prefabs given in XR Interaction toolkit 2.4 are full of bugs. Xr orgin sopouse to be ready to go complete set up but is not ready to go of course is not working.... Ehhh how to create games when basic stuff are mess up.

@judahhanes 9 месяцев назад

how do u get physics with it?

@chetnanayak4716 2 месяца назад

how do you switch from controller to hand

@triple3204 11 месяцев назад

it lets me use my hands but it doesn't track my fingers, any fixes?

@user-eo1kn3qn7e 11 месяцев назад

is there a way to use hand tracking and controller tracking at the same time?

@yuzhengchen7780 10 месяцев назад

Did you find any solutions?

@Nika-ss7dv Год назад

When I press any button on my controllers they both fall out of the world and I cannot get them back unless I restart the game mode. Also the hands are not showing. How can I fix this?

@PawelZielinski_ Месяц назад

The hand visualizer component is not popping up for me

@danieldubinko9561 Год назад

How can i make my hands collide so they dont go through eachother

@nife6902 Год назад

Can I use it with HTC Vive?

@connorherrod7723 Год назад

My HandVisualizer class wont load, anyone else have this issue? I’m using the same version of unity and have all the current packages and samples. Totally empty project otherwise.

@CustomGPTSolutions Год назад

Hey Valem I just tried it. I feel I followed every single step you mentioned. I don't have any errors. But when I put my oculus controllers down and wait for a few seconds, I don't see my hands. I have no clue why my hands don't show :(

@soya8983 Год назад

I also have the same problem

@TheArmatt Год назад

Check whether automatic switching to hand tracking is on in the settings of the quest itself. I had this problem too.

@carlosabiamer Год назад

@@TheArmatt this fixed it for me, thanks!

@MarkRiverbank Год назад

I have the same issue, plus the "could not find active control..." errors. I guess it's a preview package for a reason. I backed it out for now and may try again in a new project. The hand-tracking in Meta's own toolkit worked great, and had a lot more functionality with poses and pose recognition, as well as other Oculus features like voice commands. I like Unity's toolkit, but if you know you're developing for Quest, then Meta's toolkit has some advantages [but unfortunately, crappy documentation].

@CustomGPTSolutions Год назад

@@TheArmatt thanks for the suggestion. I tried that too. And then the VR scene doesn’t even load for some reason.

@barboradarmova1368 Год назад

Has anyone else the problem of handtracking not working in build, only in editor?

@RiverExplorerGames 10 месяцев назад

They seems to have updated things. When I get to the 10:00 point and hit PLAY, nothing shows up. No controllers, and no hands. And as they have changed some of the names and serialized fields (one seems to be deleted). It could be a I just made the wrong guesses.

@flym0 10 месяцев назад

I've been caught too. Which versions of the relevant XR packages do you have in your project?

@anonymoussloth6687 Год назад

How do I run the unity app in quest when i enter play mode?

@ozbae Год назад

Same issue

@anon3020 4 месяца назад

i am having a issue where i install the interaction toolkit and the complete rig isnt there did they remove it in a newer verison?

@ValemVR 4 месяца назад

The name has changed its something like Hand Interaction Rig or something

@anon3020 4 месяца назад

would it still be in xr interaction toolkit because if so I don't see that either@@ValemVR

@xialah2882 Месяц назад

@@anon3020 It is XR Origin Hands (XR Rig) inside Samples > XR Interaction Toolkit > 3.0.3 > Hands Interaction Demo > Prefabs

@vinothm8834 11 месяцев назад

rotation is not working for me

@lucutes2936 3 месяца назад

1 year passed

@user-yu2oe6sx8m Год назад

Can't even enter the VR scene with this error:DllNotFoundException: UnityOpenXRHands assembly: type: member:(null)

@script4724 Год назад

same here :/ if you find a fix let me know please

@user-yu2oe6sx8m Год назад

@@script4724 with Quest Pro problem solved itself

@user-zb3iz2hi9n Год назад

Is there anyone who succeed working in PICO4?

@raphael3024 Год назад

But but but where 2 handed grab ? 😥

@ValemTutorials Год назад

There is an example of two hand grab in the scene i show at the end but I need to make a tutorial about it too :D

@raphael3024 Год назад

@@ValemTutorials Yes pls misteur valem

@gorillaxvr. Год назад

@iliyasmarat7610 10 месяцев назад

For the PC version, it only sees the controllers, not the hands.

@flym0 10 месяцев назад

Mine is working fine - can you add any more info for your problem?

@yuzhengchen7780 10 месяцев назад

@@flym0 Could you let me know what Unity version and device you're using? I've got a Meta Quest Pro running on Unity 2022 LTS. I created a new Unity project and followed the video step by step. I was able to get my controllers working just fine, but the hand tracking isn't functioning properly. I can see the hand mesh sticking with the controllers, but I can't control the hand mesh after I put the controllers down...

@yuzhengchen7780 10 месяцев назад

@@flym0 Thanks and I figured out the problem! I was using SteamVR as my Play mode OpenXR Runtime in the project settings. Once I switched to Oculus as my OpenXR Runtime, the hand tracking started working perfectly.

@flym0 10 месяцев назад

@@yuzhengchen7780 Smashing, great news.