How to get Face Tracking in VRChat

Подписаться 1,3 тыс.

Просмотров 108 тыс.

50% 1

*All links and downloads below* also note this video is fairly outdated now
Sranipal 1.3.1.1:
hub.vive.com/en-US/download/V...
Mirror:
drive.google.com/file/d/1Wt9M...
VRCFaceTracking GitHub:
github.com/benaclejames/VRCFa...
VRCFaceTracking Wiki:
docs.vrcft.io/docs/intro/gett...
3d printable face tracker mount for the Index (you'll also NEED a USB 3.1 adapter or else it will not work):
www.thingiverse.com/thing:533...
smile.amazon.com/gp/product/B...
Face Tracking Template (Just toss your animations in here!):
github.com/Adjerry91/VRCFaceT...
Universal Eyeball Tracking Controller (From : github.com/regzo2 )
drive.google.com/file/d/18B90...
VRCFT avatar setup tutorial:
• Fast Face Tracking Tut...
(Binary parameter tutorial is now outdated, but heres the github):
github.com/regzo2/BinaryParam...
OSCMooth for emulating ik sync!:
github.com/regzo2/OSCmooth
VRCFT Discord for more questions/answers and templates, everythings changing fast so you definitely want to look here:
/ discord
*You should not need to pause in between switching avatars any more
If you want a better microphone for the Vive Pro Eye, you can use a USB hub and sound card. MOST USB HUBS WILL NOT WORK! I have tried several USB hubs and none under $40 have worked with the facial tracker. This is the cheapest hub that works (and the slimmest) that almost everyone uses (or at least a knockoff of this model):
smile.amazon.com/dp/B07X8ZLYLR
Here is a good lapel mic to replace the terrible Vive mic:
www.amazon.com/dp/B016C4ZG74
Here's a sound card to plug it into:
smile.amazon.com/dp/B07RV6VBNR
Photo of Index with 3d printed mount from NamBo# 1337
Song: Phantom Sage - MIKO [NCS Release]
Music provided by NoCopyrightSounds
Free Download/Stream: ncs.io/miko
Watch: • Phantom Sage - MIKO | ...

Опубликовано:

2 мар 2022

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 198

@SonitoShori 2 года назад

you literally included everything I couldve needed for the facial tracker, even another 3d printed mount that will actually work! thank you so much!

@greighson3766 2 года назад

oh my gosh its you sonito!!!!!

@SonitoShori 2 года назад

@@greighson3766 wtf how on earth did you find this 😭😭

@greighson3766 2 года назад

@@SonitoShori i saw face tracking on my recommended so i looked it up

@JuanigD13 Год назад

The fact i watched the whole video even without owning one of those

@rxnoinvr8099 Год назад

Same

@doodlemess_ Год назад

SAME😭

@ChaseRedFox 11 месяцев назад

You need money lots of it😂, full vr kit is almost 4000$

@Drud 11 месяцев назад

@@ChaseRedFox nyoooooo 😭

@Drud 11 месяцев назад

I don’t even have VR chat nor a stable computer. All I have is a laptop. 😭

@pokegeaks2 2 года назад

Thank you so much for posting, just got a facial tracker and printing my mounts for my quest2, been looking for some form of tutorial on it for the past few days

@linuskarlsson6208 2 года назад

Me seeing eye lazers: hey that's cool I would love to have an avatar with that. Me actually thinking about it: nevermind I do not want people to see exactly were I'm looking

@musicdemon3453 2 года назад

the amount of detail in your video blows my mind. and you are a fur too makes it even better 😊

@reval2685 2 года назад

Pretty neat showcase! Just finished up adding face tracking blendshapes to my avatar base. Will give it some more testing and that'll be one more base out there with official face tracking support.

@toffygulf8989 2 года назад

you got me at the end, i loved it!

@illDiology 2 года назад

It's absolutely nuts the level of technology we have access to these days.

@Whitepawprint 2 года назад

I'm just amazed watching the avatar talk!

@tuvelout Год назад

Me 2

@20nik 2 года назад

Love the video! Keep it up dude ^^

@arilemons1059 2 года назад

This is very informative :D Thank you

@gabrielfl481 2 года назад

That end. I must subscribe

@Jr837_ 2 года назад

I dont even play Vrchat, but this video is oddly entertaining

@NafrytiNosferatu 2 года назад

I love the paper cup texture of your Rex!

@garrisonwalter Год назад

Best Tutorial Ending Ever!!! XD

@xCHILLGHOSTx Год назад

This is Actually Really Helpful Thx ^^ boop

@sparkle5164 Год назад

I got the Vive Pro Eye a couple of years ago because the regular one was out of stock and my dad was like "well now you can have eye tracking, so just get that one" XD I finally decided to look into adding it on my VRChat avatars so I can use it :)

@Betonoszlop Год назад

Fantastic, Disney moments IRL!

@wisgarus 2 года назад

IIRC, some relatively newly announced headsets are starting to slowly include eye and/or entire face tracking as well as some more experimental ones even promising inside-out full body tracking. Will be sick if in a couple of years we start seeing even the more affordable VR headsets like what Quest 2 is today release with all these tracking methods so that immersion can go to a whole new level in VR games and especially VR social apps. Though ofc there's also the matter of rigging and adjusting the models themselves for face tracking, but I'm pretty sure there's a few that already do that, so if everyone gets face tracking, the demand for such avatars will go way up, and more people will make them.

@haroshadow 2 года назад

right now a lot of the vtuber and hobbyist discord friendstreamers enjoy VroidStudio weeby models with iphone + iFacialMoCap for extremely well-done facial tracking. like there is nothing else out there better. lots of the bigger vtubers who have a real 3d model have been going the iphone route. I just hope one day VR can meet this same standard, because people have expectations now.... Like not even the Live 2D models from NijiSanji or HoloLive have good facial tracking and people just kinda overlook that.

@wisgarus 2 года назад

@@haroshadow I find it kinda weird that there's some vtubers with exceptionally good visuals on their models like Girldm but then all the super extra popular main-stream ones just use well drawn, but otherwise basic 2D rigged sprites that don't look nearly as good or expressive in the end

@haroshadow 2 года назад

@@wisgarus i agree but it is mainly because a lot of the "lowest common denominator" type of basic bitch weeb fan actually have admitted lots of times that a 3d model is ugly or uncanny valley. they want 2d only. who knows why.

@cappsami 2 года назад

Thank you for the simple and informative material! However, I'm having issues having the ears move with the eye and face tracking. I'm unsure if there's something set up wrong on my end where it's locked into the controller controls?

@StatekTi 2 года назад

You can make a new animation for the gesture layer using the ear bones (instead of blendshapes) and tie them to the Left/RightEyeLidExpanded parameters, at least that's what I did.

@butterfish4987 2 года назад

Great video

@StatekTi 2 года назад

Upvote the OSC issues on the Canny and VRChat's GitHub: feedback.vrchat.com/feature-requests/p/increase-the-update-rate-for-osc-parameters As of 4-18-22, you should no longer need to pause between avatars!

@andyswoodyrubnips 2 года назад

thanks mann for the info sounds like i should just wait for vrchat to release it. i dont want mee face being a slide showww to others.lmao

@trvdwaffenstoff9748 Год назад

This is really awesome, then you see the price of the headset and you're like 💀

@kyuoe 6 месяцев назад

I would watch it all again just for the ending

@the_realbwkool 2 года назад

I'm gonna be getting an Index so it sucks that I won't be able to yet eye tracking but maybe they'll eventually make eye tracking for the index

@MikeCore 2 года назад

right, love your voice

@Slycamer 2 года назад

@statek - I talked with Varjo support and they are planning to put in eyelid tracking but they didn't give a delivery date

Месяц назад

How much of this is still relevant? I got a Winterpaw canine and followed her tut' but only have eye tracking with basic blink (no winking or pupil tracking) and no face tracking. I thought it might ne my headset (Vive XRE) but when i use the free Wolf Link rex AV, the tracking is superb. Any thoughts on where the disconnect could be?

@Parzeval Год назад

Hiya Statek, Do you know if the face tracking only shows locally (just for you) or are others able to see your tracked face on the avatar?

@StatekTi Год назад

Everyone with your animations shown can

@axyllpurplefurr1781 2 года назад

I liked.💜🌺🤗

@ConalFlood 11 месяцев назад

thanks man

@user-uu9ru3de1y Год назад

weird question, but think this could be done using the right joycon's ir camera?

@starshipshari Год назад

I just ordered my vive pro eye, I can't wait to set it up. Thank you so much. Anything I should know at this current time?

@Will_Is_Coming Год назад

you should get ready for trying to get the sranipal to work.. it propably wont

@berryvr7184 9 месяцев назад

This has helped allot

@StatekTi 9 месяцев назад

It should be fairly straightforward using one of the face tracking templates, you just have to make animations for each of the mouth movements you want and then tie them to the right parameter

@MunchPremium Год назад

Do you know anything about the omnicept edition bc im thinking about buying it (good deal used)

@StatekTi Год назад

Not worth it, I returned mine

@KClove. 2 года назад

Is there anyway to do this in blender? Full and smooth mocap sounds good

@Negative-3000 Год назад

I'm going to be getting a meta quest pro and I'm wondering if it's built-in full face tracking would work with VR chat?

@StatekTi Год назад

We're working on it right now. It's barely usable in its current state, but getting better

@Scampcam Год назад

is the vive eye the only one that does eye tracking? not index?

@omsmada Год назад

any change you'd know any way of getting a 3d printable adapter for the droolon f1 eye tracker to fit onto an index?

@StatekTi Год назад

There's DIY eye tracking for the index that already work better than the droolon (though that isn't saying much)

@MZittylp Год назад

great tutorial but does anyone know a fix for VIVE_SRanipal not detecting my facialtracker? sometimes a reinstall of the VIVE_SRanipal helps but now it doesnt help anymore

@najtmer_ 2 года назад

I see more activity generally Nice movie

@DoctorPhileasFragg 2 года назад

Oh, charming!

@Kav_Games Год назад

I really wish there was some kind of mount for a quest 2 for eye tracking. I've made so many ridiculous faces in vrc just for nobody to be able to see them.

@StatekTi Год назад

You can check out EyeTrackVR

@Kav_Games Год назад

@@StatekTi Thankyou I had no idea this existed I'll look into it

@ninjarider3289 9 месяцев назад

Is there a way to use eye tracking with vive cosmo elite?

@Blockistium Год назад

What is the USB hub listed in the description meant for? Is it supposed to be plugged into the headset's USB-C port?

@StatekTi Год назад

If you want to add an external microphone and use the face tracker at the same time, you'll need it specifically

@NesVirtualStudios 2 года назад

Awesome video! And I'm assuming the facial tracker does not work with the original vive?

@StatekTi 2 года назад

I think it's technically possible but not easy

@Teknickel_ftw Год назад

Dose this work with a Tobii HTC VIVE Devkit? I end up picking one up for it's base stations, It would be neat to try out.

@StatekTi Год назад

I know a few people in the face tracking discord have it but I'm not sure how well it works, if at all

@zeoxiro 3 месяца назад

Hi, can someone explain how to enable face and eye tracking without PCVR?

@omsmada Год назад

know any places selling the vive pro eye new cheap anymore? It's seemingly only available >$1000 anymore, probably because of this stuff...

@romanlegionary6376 Год назад

Im so lost on 2nd software because it shows nothing but files and completely lost

@myooralsystem6102 Год назад

what kind of pc do you need for stuff like this? (Coming from someone who just wants a prebuilt cause doing myself isnt an option)

@StatekTi Год назад

If your CPU maxes out under 70% in VRChat, you can probably run it without issue

@cyanidexeros6368 2 года назад

Is it possible to have mouth tracking on the OG HTC vive? I 100% cannot afford the vive pro eye since I'm borrowing my vive from a relative.

@StatekTi 2 года назад

You'd need a USB 3.0 extension cable going from your tracker to your computer, but its doable

@w_rcake9119 Год назад

so the normal vive pro doesn't work with eye tracking? damn

@superjare23 Год назад

Does EAC on VRchat prevent Facial Tracking from working?

@StatekTi Год назад

Nope!

@Misteurpro Месяц назад

WAIT DID YOU SAY LEGALY POSSIBLE? Like was it illegal before?

@KlearChristal Год назад

When I try to install it says the installer was interrupted

@codywolf6833 2 года назад

Do you know if the G2 Omnicept Eye tracking works as well?

@StatekTi 2 года назад

Theoretically it should be able to work nearly as well as the VPE, but not enough people have one (and the people that do have one prefer other headsets for multiple reasons) so theres no current eye implementation for VRC and no mouth implementation for anything.

@NekhayInc 5 месяцев назад

And this will work in standalone mode (that is, without a PC ) on meta quest pro?

@StatekTi 5 месяцев назад

Nope, there's no support for standalone

@ArcticArua 5 месяцев назад

Oof, gonna have to get the quest pro and deal with the mixed playspaces

@RyneVera 2 года назад

now I just need models that support lip tracking

@crodojoe Год назад

ohhh i remember you now

@x-Rubin 2 года назад

Hey, so how exactly would one make a custom avatar face tracking ready?

@StatekTi 2 года назад

If the blendshapes are already there, just map them to the face tracking parameters found on the github. I just added an example Unity project to the description.

@helpshizi2324 Год назад

Will any of this work for a oculus 😅

@alwaysyouramanda Год назад

The texture of the paper cups at the dentist in the 90s 😻🥲

@ivancarrasco5358 Год назад

I think they discontinued the vice pro eye now there’s this new attachment I assume can mount to any of the vice pros?

@StatekTi Год назад

You can 3d print a mount for it to fit on anything

@reyluna0 2 года назад

Hahahaha the fucking end 🤣💀

@skyrfelkin Год назад

What’s the eye laser asset called and where can I get it???

@StatekTi Год назад

It's in #file-share in the VRCFT discord

@Yutaru 2 года назад

I'll wait for the camera to work with face tracking like the full body that is available with a single camera and doing some work instead of buying that because too expensive

@aiyonara9870 Год назад

Considering buying one but I have a question. Will the tracking work with any avatar or does it require creating an avatar to make it work?

@StatekTi Год назад

You have to use an avatar configured for face tracking

@aiyonara9870 Год назад

@@StatekTi oh boy time to learn blender and unity 😂

@Justin-op8us Год назад

I guess Quest Pro is going to solve all of our face and eye tracking issues. That is, assuming we can afford to drop $1500 (or whatever it'll cost).

@Freezie1988 Год назад

UPDATE! the varjo aero now DOES track eyelids!

@Freezie1988 Год назад

the only trouble i have is OSCmooth or what this thing is called.. When i put this in my fx layer (yea i did it correct with help from others ) i crash the entire lobby in vrchat when i calibrate my eyes with my varjo :D

@Snexzyy Год назад

I clicked here for the Lazer eyes in the thumbnail

@WerkonX Год назад

Hi i was planning to get droolon p1 module for pimax and i was wondering what other things than pimax software and VRCFaceTracking i would need to get it to work.

@StatekTi Год назад

I believe you toss the dll in appdata/roaming/vrcfacetracking/customlibs from github.com/benaclejames/VRCFTPimaxModule/releases/tag/1.1 and then unblock it in the file properties, but I can't find anything concrete on that. You may want to ask in the VRCFT server

@WerkonX Год назад

@@StatekTi thank you very much everything works

@chill.huskyy2930 Год назад

so i got this working, but im a little annoyed with how the mouth still reacts to mic audio on top of my tracking. which makes the tracking look dumb. is there anyway to fix this? or is it just avatar specific?

@StatekTi Год назад

Face tracking-specific visemes (that are less extreme), particularly limiting the aa one. There is a tool for it by Sacred in the VRCFT discord I believe

@chill.huskyy2930 Год назад

@@StatekTi thanks

@whatsevenhappening8295 2 года назад

Hey Statek do you know any Avatar worlds where i could find furry avatars with eye and mouth tracking?

@StatekTi 2 года назад

Jerry's Mod has a few, xMetalWolfx also has some older ones I think

@SunG34r Год назад

That old tracking software thing still hold true?

@StatekTi Год назад

It's been updated a few times but yeah everything still works

@doubleatheman Год назад

Any guides or solutions for using the quest pro?

@StatekTi Год назад

You can join the VRCFT discord for guides on how to set it up. There should be a new official release with support for it in the coming days/weeks that I've been waiting for to make an update video on

@FireStillPlays 2 года назад

Hi! Do you know if this will work with the HP Reverb G2 Omnicept Eye/Face Tracking?

@StatekTi 2 года назад

There are some people working on it but as of right now it will not work. The mouth camera outputs a raw video feed so you'd need to train your own NN to get anything useful out of it, and I don't believe the eye cameras support floats for eyelid tracking (which is extremely important for social VR).

@FireStillPlays 2 года назад

@@StatekTiThat's a huge bummer. I'm going to see if I can use the camera feed and run it through htc's lip tracking software to get some results. It doesn't seem there's much I can do about the eyes though, so damn. I guess i'll just be patient. Thanks for the quick response!

@StatekTi 2 года назад

@@FireStillPlays Unfortunately you won't be able to get any good data from it even if you were able to pass it through their software. The camera feeds are from extremely different angles (and with the Vives in stereo), and even tilting the Vive tracker up/down a few degrees will start to ruin your data.

@FireStillPlays 2 года назад

@@StatekTi Well damn, guess patience is my only option then :P Thanks again for responding so quickly! P.S your avatar kicks ass!

@pklpklpkl 2 года назад

@@StatekTi Hey, I wanted to mention that the Omnicept headset in fact does have data output that includes "[float] Openness" for blinking as listed in developer documentation(along with several other interesting features and values). The mouth tracking does appear to provide just a 400x400 grayscale image, but there are several methods of grabbing that to be explored later. I am willing to work with someone on this implementation, starting with the eye tracking, and working with mouth image data after. I have already contacted HP techs about possibly providing a form of active mouth tracking via an alternate SDK to achieve something similar to HTC's. Wanna see where this could go or have anyone willing to collaborate? (if it isn't obvious, I have this headset)

@TriggerTail Год назад

What VR headset are you using? Just because I'm looking to buy VR, but I don't know any headsets that are good, so, which one could you recommend?

@StatekTi Год назад

Vive Pro Eye. The only other real option is the Quest Pro right now

@TriggerTail Год назад

@@StatekTi Thanks, needed help about this, but couldn't find an answer, this might very help.

@Justguy363 Год назад

@@StatekTi what about pico 4 enterprise? Because i can't find any vrchat gameplay with pico 4 enterprise with face tracking.

@StatekTi Год назад

@@Justguy363 They only just recently released the ability to grab face data but it's being worked on

@Justguy363 Год назад

@@StatekTi oh ok, i wonder how much time would take to implement

@Rum0r Год назад

The Sranipal mirror isn't working :(

@StatekTi Год назад

Should be now

@maarmerman3889 7 месяцев назад

So the best solution in 2023 are still Vive Pro Eyes or Meta Quest Pro, right?

@StatekTi 7 месяцев назад

Yeah

@horizonchain Год назад

Hey could i use my phone instead im really curious

@StatekTi Год назад

Yep, there's a module for Meowface

@AnimeGamerCreater 2 года назад

no idea wat this is but cool

@FurryNonsense 2 месяца назад

Didn't work

@Lynn1226ddd Год назад

Wow is there a tutorial for quest pro too?

@StatekTi Год назад

Ninka_ has one up right now ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-uZh7cY170rY.html

@HikikomoriDev 3 месяца назад

Wouldn't all this work optically with a webcam of sorts and some middleware?

@StatekTi 3 месяца назад

When there isn't a big vr headset on your face, yeah

@strattonschaller2937 2 года назад

Does anyone know what the color scheme of his avatar is called? I have seen it all over the place but have not found a name for it

@sizzrp 2 года назад

It’s from retro drink cup designs

@StatekTi 2 года назад

Solo Jazz

@vhgiv 2 года назад

I am saddened the Aero Has No eyelid Tracking

@pandamochiyum8854 2 года назад

Me too, that literally killed my wanting to own one haha

@joey_outdoors 2 года назад

You mean I gotta take my headset off, push P on my keyboard and go back in VR to change my avatar, take my headset off again, push P to unpause it then go back into headset before I want to use it, each time I change my avatar? Sounds like a PITA.

@StatekTi 2 года назад

Not anymore (see pinned comment)

@Fatez Год назад

Me watching this without a vr headset

@tuvelout Год назад

Never needed one. Phone or webcam.

@SteadyShadow 10 месяцев назад

Any updates?

@StatekTi 8 месяцев назад

Yes but I've been dead

@akimbofurry2179 Год назад

On google, you who they send for people asking about face tracking.

@robertwyatt3912 2 года назад

Holy shit

@mtnlion_857 7 месяцев назад

I want these, but I dont trust myself with them on while partying

@KQCNT Год назад

what about quest pro?

@StatekTi Год назад

It'll work as well with the newest VRCFT software

@naughtykid000 2 года назад

all I heard was "money money money money, money mon mon, money mon"

@ItsSlusher Год назад

Dose this still work with the easy anti cheat update?

@StatekTi Год назад

Yep

@RayTieRom Год назад

Is this possible with the quest pro yet?

@StatekTi Год назад

It has been for a while yeah

@RallyMango11 Год назад

Been using Quest Pro for about 6 weeks now and absolutely love it! Only drawback in models is, those done for Vive don't track the eyelids correctly with Quest Pro. They only start closing when eyes are almost completely closed. But there are few public avatars made with Quest Pro toggle that fixes it. Gotta do my own Nardo to be completely satisfied