I think the lighting on her face doesn't look good and it looks like she is lit by an artificial light source try tweaking it. I think it would make a huge difference.
I think a good idea would be to add fake specular highlights to the eyes that are independent of the lighting system. I believe they do that at Disney to make eyes more lively and allow for stylization in cutscenes
The difference between the before and after of the opening shots is amazing! The game is looking so amazing! Can't wait to play! Signed up for the patron!!
With the understanding that there will always be limitations, the one issue I think that's really holding it back is the speed at which her expression changes and even blinks: It's too slow. It reminds me of old Playstation games or bad animated cartoons where everything feels awkward because there's too much time between each line of dialogue, and the characters' expressions change so slowly that it's like you can see the animator trying to find the facial change button, or manually moving a slider. Blinking should be extremely fast. Also, her eyebrows and some of the skin of her socket, are moving whenever she blinks, but blinking should move nothing but the lids. When she smiles or frowns, it obviously shouldn't be as quick as blinking, but it feels awkwardly plodding how it is now. In Sifu, the facial expressions oftentimes just switch from one to the other within one frame, and although it's noticeable, it also doesn't look _awkward,_ oddly enough. So I think with the face, faster is better, and too fast is better than too slow.
really appreciate the feedback, thanks! I tend to agree but when I go faster it starts to feel a bit "sketchy" if that makes any sense, there's probably a middleground but I agree I haven't quite found it yet. still room for improvement :)
A slight correction on your comment about the blink moving nothing but the eyelids. This is not accurate. I can see what you mean within this video, the brows especially do move too much but facial animation is the art of subtle movements. When you blink your eyebrows do move down a touch as the muscles that contribute to a blink move beyond just the eye. You can get a sense of this by placing a finger on the center of your eyebrow and then blinking. There is a noticeable pulling down of the eyebrows. The top of your cheeks also move up and inwards slightly. Once again if you place a finger on your cheek while blinking you can feel it. Now saying all that the characters shading is quite flat, a lot of that movement detail would probably not actually end up showing too well, especially in moment to moment gameplay so I do actually agree in the end that just the eyelids with VERY subtle eyebrow movement down would be more than enough. You are also totally correct on the speed of the transitions between expressions, I would definitely double that transition speed as a bare minimum. Saying that what is shown is a very solid foundation to work from and with it being code driven is more tweakable rather than a bunch of precanned facial animations that need a lot of back and forth at an animation asset level!
Nice, it's looking much better at the end! The shot in the car from the side with the low lighting looks pretty dang good, although the more straight on shots still feel a bit stiff to me. A few observations: The eye behaviour is incorrect. We only move our eyes in a continuous motion when we're tracking a moving object. When we're changing our visual target we usually move in saccades, which are nearly instantaneous jumps. At 5:10, we see the eyes move smoothly to a new target before the head catches up. That should be an instant jump to the new target. Actually I think my eyes often move in a rapid burst of jumps - I jump to the area of the target, then I jump again to exactly focus on it when I've found it. I also feel like she turns her head way too slowly - that motion feels like it takes me about 250ms. Actually all of her movement feels a little slow. It also feels weird how rigid her face is. Our expressions change, we breath, swallow, slightly wet our lips, etc. Faces have idle variation. Her face looks pretty good when we see it shadowed, but quite uncanny when well lit. Her skin texture blows out very flat and we lose all of the detail in the texture. A bit more texture and shadowing/AO would probably help here. This game has come a long way!
Awesome feedback, thank you! The tricky thing is, and I didn't go into this in the video just for length, but I'm testing on these straight-on shots but optimising for the side/moving/distant shots if that makes sense. So sometimes the movement needs to be exaggerated a bit to even see it from those shots. I think perfect world I'd create two sets of each, one more realistic for the close-ups and one more exaggerated for the distant shots. But there's a ton of room for improvement, I agree. I'll probably be tweaking all of these right up to release haha.
The facial animations are looking really good! I'm certainly no expert when it comes to animating, but one thing I noticed is that Kaylee's face looks a bit rigid. I think part of the problem is that humans very rarely are completely still. There's always some shift in expression, breathing, or subtle eye movement. But great Devlog as usual, I'm looking forward to the next one!
Fun fact: Audio Spectrum data can be used to extract vowel phonemes in real time by using the magnitudes for individual frequencies to find the 0, 1, and 2 formants and then comparing the 1 and 2 formant frequencies to each other which maps to each vowel. Consonants are identifiable based on how they change over time but can vary from speaker to speaker, they're also still somewhat detectable by Audio Spectrum data too "Acoustic Phonetics: Formants - University of Manitoba" has some stuff on this
Great job! Facial animations are definitely a daunting task. One thing that might help is the speed of her eye movements when she looks at something else. People's eyes tend to jump from target to target rather than moving smoothly (unless they're tracking something that's moving). If you try yourself, it's actually really difficult to move your eyes gradually from one point to another. We also tend to blink when we change what we're looking at. Not always, but most of the time.
Sir always being your biggest fan, I really want u to read this, I live in a country where internet is a very scarce thing and I am glad to get the privilege to get access the internet and i can only watch few videos a month before my internet is over but... watching ur videos I get motivated to study and leave this hell and work in a place like u, U set my bar of what I can be... please never stop grinding..
The before and after shot is insane! Although the effects are subtle, they've made a dramatic change imo. It all really came together when you added the facial animations and I definitely see more life in her now.
I think you did a great job with the eyes, here is a piece of info that might help it go to another level. Eyes doesn't move in a smooth way, they flicker and move snappy. You can try to record your eyes where you pan with them from left to right and you will see it, our brain just interpretate it as smooth. So to make it less uncanny, you can make it much more rapid. Another neat trick is that our eyes rarely move without blinking, we often blink when moving our eyes. Yes I'm an animator :) Amazing progress though, I love the effort you put in.
Great Devlog! I really loved the part where you show code and highlight what you're talking about! I think the end result is good and you should definitely be proud! I smiled myself when I saw that she smiled while in the car!
It looks way better than it was previously. but I think you need to have a look in to anatomy a little bit more as well. For example a blink and closed eyes are two different expressions, usually with a blink, it's fast and has a little bit less pull on the eyebrows than when you intentionally close your eyes or narrow them. Here I think you might have crossed over slightly in to intentionally closing eyes instead of blink, the speed of the blink is also giving that impression. Another issue with the eyes is that they have a very "always 100%" look to them, it's probably down to the character always panning left to right and never looking up and down. When in deep thought, like your test scenario, the "but where's their mother though", you would naturally look down a little bit, or up in optimistic thought. She has the thousand yard stare when she says "must have been separated". When you look down with the eye, the top lid would normally close a bit, of course you might also relax the brow down slightly as well, here, she's pulling the brow down but the eyes remain staring forward. Drop the eyes down, allow the eyelid to close slightly, then in the second statement about separated, lift the eyes back up so the eyelid opens again. This would remove the constant look of a stare that she has, it's mainly down to the eyelid not moving up or down because her eyes always just pan sideways. You could have a glance at the intro I have on any of my videos, in the intro my little cat mascot pops up and presents the video title. The only elements of his face that move at all are his pupils, eyelids and ears, but you can see how he can convey emotion just by opening and closing his eyelids. If you have a look at my latest video, he's sitting next to my avatar as I talk, watch how his eyes can look up or down to show him just minding himself and chilling out.
Kyle, you're amazing, I've been loosely following this game since you introduced the petting feature and i loved the demo I think you're doing great work and i certainly do believe in you (and your tricks) Nice one
Looks good! Faces are hard. One think I notice is that her eye shader is really white and flat. Maybe try to fake some shadow on the eye balls? I think that would improve it even more :D
A really easy thing you can do to make the eye movement look better is to couple the blink amount to the gaze direction. When eyes look up and down, the eyelid moves with them, keeping the upper lid just above the pupil. If you don't have this, it makes the eyes look more stare-into-your-soul-y.
Also, make sure any phonemes that make a round mouth shape deform the geometry around the upper lip, nose, and lower cheeks. Likewise, check that your emotion blendshapes engage that area. Having a big dead zone in the middle of the face that functionally never moves is a really common problem with videogame facial animation, but it's fixable!
Fun fact: Céilí dances are still *somewhat* common in Ireland today, especially in the Gaelteacht areas (Irish speaking regions) and during the summer when young students go there for 2-3 weeks to learn Irish or at the Fleadh Cheoil
Love the progress, one feeling I get though when watching the final piece is that she feels a bit too "smooth". Like head movements look smooth and well, uncanny. Like moving maybe a bit too much at times, and too smoothly at others. While we don't "jerk" our heads too quickly to the side it usually isn't a perfectly smooth movement, with some essence of a jerk motion at the beginning especially and speeding up very rapidly in the middle motion and slowing down quickly to a specific point we are looking at.
Every time I see a new video Im like "Is he STILL not finished?" :D Btw a problem of why I think she looks unnatural is because the eyeballs are too white, the iris is too bright and the pupil too black. Just changing the texture/material could make a big difference aswell.
hi kyle!! I'm new to your videos and obviously late to this one, but I still wanted to toss in my two cents :) I'm sure you've been working on a lot of the things I'm going to bring up since it's been a while, but I wanted to touch on them anyway! part of the reason her expression still feels a little lacking is the lack of 'life' in her eyes. adding a highlight that is independent of the lighting system will do a lot to change that. if it's possible I'd recommend a highlight layer that sits 'on top' of the eye texture/iris etc layer, with perhaps two or three different shines in them at some distances, so that it looks dynamic when the eye moves, sort of 'fakes' an environment reflecting in the eyes at all times, and still offers a quite permanent touch of life. so that wherever she's looking, there's at least one light in her eye, but it can change depending on where she's looking. and the shine not moving with the eye will give you that touch of realism, even if it still adds to the feel of liveliness if that makes sense! I also think the eye blink is quite well done! the main thing that stands out to me is that it feels like she's blinking too little, and too regularly. humans blank a hell of a lot, and while it may not look nice in a game to be blinking all the time, adding 'double' or even 'triple' blinks into the rotation of the animation will make it feel a lot more natural. sometimes we blink a couple times at a time after all because of weather conditions or bright lights or just because our eyes need a little extra moisture haha. there's no reason to overdo it of course, games are inherently stylized and can never be a 1:1 to reality, but just adding this kind of variety would do a lot! another thing is that humans often blink before we turn our head. it's a way to counter vertigo. so if we're turning our head to the side, we'll often do it kinda fast, and we'll keep our eyes close for the split second it takes to turn our head. I don't know if that is something to try to add to this game necessarily, because she does look quite good even just looking around, but I think this is another thing that could be fixed with more irregular blinking cycles and on occasion faster blinking. because it'll add to the realism :) I really like how subtle the expressions are overall, it feels realistic and nice. but do consider that if we've got a camera quite a far distance away, we may not be able to read the expressions. perhaps you could alter the intensity of the expressions depending on if it's a cut scene (more subtle) or in world (a bit more exaggerated, to ensure it's well communicated to the player no matter where that player is in the scene). I haven't played the demo yet so I'm not totally sure how scenes with her are handled, but just as a thought! I really love her outfit, she feels relatable and real. and I've thrown your links at a bunch of friends of mine who I know will adore this game just as much as I will when it's ready! it's been incredibly inspiring looking at your devlogs, I'm not a game developer nor do I have an interest in becoming one, but I am an artist and I love getting a peek at other art forms. the visuals of the game are fantastic and I can't wait to run around as the collie. it's really nice to see minority languages represented in small was too! incredibly excited for all this! thank you for sharing your work with us!
yo man, i am also another beginner dev working on my first big project, i was looking for devlogs to watch and keep me motivated and came across yours!! i was actually working with facial animations recently because there are a lot of cut scenes i've been planning out while writing a few of my scenes/scripts and it's gonna really be important, i thought back to one of my favorite franchises, metal gear solid, and how hideo kojima had talked about how he was taking the criticism of the public for MGS V to heart, and was gonna trim down on the dialogue a lot, and the solution he found was that facial expressions, are so important. he emphasized that he was able to trim something like a dialogue scene of a character saying "hey snake, you look down, are you feeling alright? you don't look too well", and instead, have that same character just say: "Snake....?" while showing a concerned facial expression. those two scenarios would equal out the same result, while the latter literally trimmed everything down 110% and simplified it, all thanks to the facial animation lol. that really had an effect on me and how i looked at things. so then lately i've been trying out different tools and options just as you have too, and one of the ones i absolutely LOVED the most WAS the facial capture. unity's facial capture system is actually super easy to set up, i was able to in less than 4 minutes after downloading the asset and app on my iphone, all i did was type in the ip address onto my phone and bam, connected! then all i did was literally hit record, and acted out my scene...then i took those animation keyframes and just applied them to the actual character, ( applying them to the LOD 1 model with all the nice detail of course lol, for use in cutscenes) and man, its just way better than hand animating. the reason is, with any type of hand animation like what you're doing, is that it can easily fall out of line when it comes to accuracy. when you're doing facial capture, or any motion capture, youre literally capturing the exact motion and limitations of the body, realistically, rather than trying to recreate them by hand and analyzing video of the reference.. you'll get those accurate results and it looks so much better at the end of the day, i know you said you wont use motion capture for this but maybe you just needed the right perspective to change your mind lol! like what i said, what youv'e been doing is SO MUCH more complex and complicated than motion capture, with how easy it is to record mocap these days, without the heavy duty pro equipment, all we need is out phone/webcam and we are literally ready for anything, seriously spend one hour looking into it and i guarantee you that you're sold on it haha the fact we can do all that for free with mocap these days are incredible, take advantage of that before things begin going up in price when more people begin using it!!!! it won't take any massive changing of your project, it'll compliment it perfectly, what better thing to use than actual facial motion capture for facial expressions? just record yourself acting out the motions, and then you'll be able to adjust anything afterwards, perfecto! anyways i hope you keep it up, coming across this video has led me to subbing to you to follow along on your journey!! good luck on this and i hope to see an update of you trying it out bro!!!!
If you haven't already done this, try adjusting her eyebrows. When blinking subtly the brows barely move, but doing a hard blink requires more muscles of the face so the muscles around the brows would just slightly follow. 7:16 is exactly what I'm talking about!
If you want to beat up the uncanny, there's one rule that is often over-sighted: Asymmetry. And I'm not writing about extreme variations like "the Rock face", but just a slight variations between the left and right in terms of speed and extend. People aren't noticing it, but everyone move the left and/or right side of the face slightly differently. It's a case of barely a few frames of difference, but never the less it's noticeable in animation. We blink our eyes at different pace and speed when we focus/look at something as to avoid loose as little as possible information about what we look at. People who train both sides of their mouth's muscle equally are ones in hundred of thousands and it affect some of our habits such as chewing or smiling more one side than the other. (If you ever bite the inside of your cheeks/mouth by mistake, have you noticed it's often the same side? That's because even our jaw is moving not perfecting up/down.) So adding a tiny bit of consistent asymmetry help define the character better when animated.
This is already looking so much better dude! As an artist who puts a lot of thought into faces and expressions tho, I think you would really benefit from doing some more work on the eyes. Rigging them so they can look up and down I think will make it a lot easier to avoid “pug eyes,” but if that’s too difficult I think just lowering the eyelids a bit sometimes will go a long way. On most people, it’s not very often that you’ll see the whites of their eyes above their pupils when their face is neutral, if that makes sense. So just lowering that lid a little bit and covering the top of the pupil like maybe 40-60% of the time in idle animations I think will go a long way.
Amazing! Loved the approach to lip syncing, elegant and nicely stylized. And the facial expressions work so well especially with the excellent character model, subtle was a good call! Some nitpicks/suggestions: - Blinks could be a bit faster. - Eyes could've been done with shapekeys or eye bones IK re-targeted at runtime. - When she turns her head it feels like her eyes aren't quite far enough in the corner or something idk - When eye's look up and down our eyelids follow, not having it makes the eyeballs feel more disconnected from the rest of her then they already technically are even if the eyes only look up and down a little bit it's apparent the lids are stiff - I saw in another comment fast expression changes apparently looked a little odd but maybe a touch of overshoot could make them work? - And I completely agree with the other commenter's suggestion of idle face animations.
Really appreciate the feedback, thank you! On the shapes/IKs for eyes, this way the eyes don't actually need to be a separate mesh which is a nice little benefit, but other than that yea I agree you could totally do it that way
The facial animation is looking good. Some adjustments to hand animations to be mildly less stiff could be useful (tho much easier said than done). I wonder if adding some movement to her hair (subtle not like full hair sim) would help the animation of the head and give her a little more life in her actions? Anyways great work! And I love the code bits haha so don’t take those out
All of these improvements are trully great and complex. However I believe a very simple fix is all you needed to make her expression much more natural, by making sure the eyes look like they are in their natural resting pose. She shouldnt look awkward in a still shot, which unfortunately was already a problem before any animation. The upper eyelids are simply too high over the iris. No human being has that much white space on top of the iris, it makes her look like shes constantly holding her eyelid up, which looks unnatural. In fact this is so important, that the eyelid slides down when someone looks down and up when looking up, so preserve the ratio of white to dark in the eye. This specific coloring gives a feeling of calmness, versus the inverted one (which your character displays in most situations where it doesn make sense) is associated with agressiveness, surprise, fear etc. TLDR: change the base shapes upper eyelid to start of lower, to where it is covering half the brown of the iris, above the pupil, and then adjust the lower eyelid as needed. Then skin the eyelids so softly follow the eyes up and down motion (or create more blend shapes or other crazy rigging, you seem very capable a programmer)
Looks great! Have you looked at uses more "snappy" eye movements at all? I often found the smooth motion can contribute to the uncanny feeling. And btw, the Irish Céilí is pronounced same as the way you were saying the Scots 😊
Amazing dev log!! Just an idea, since they're out in the woods for a long period of time (or at least that's what it seems like) it doesn't really fit her having a full-on smokey eye. Maybe a more natural, or no makeup would make more sense?
As always loved the video! I thought the changes were nice and even though they're subtle it makes the difference. I would only say that i am not really satisfied with the texture of the eyes. They feel "blurred" and "faded". maybe switch it up to a more refined and vivid texture for close up scenes?
i notice the lips don't squish together on "oo" or "oh" sounds, that might be another way to make the lip sync feel more lifelike? you know, like when you look in a mirror and make those shapes with your mouth, your lips pucker and push outwards a bit. also, when someone smiles, their lips thin up as the skin stretches over the smile, so maybe that's another thing you could do to improve that. feel free to ignore me though, especially if there's too much work done to touch things up at this point. i'm just addicted to little details LMAO
@@kylebanks Yeah, the upper eyelids are usually maybe 1/3 closed (very rough estimate) when looking straight ahead, and they track the vertical / pitch movement of the pupil relative to the head. So when you're looking down the eyelid will be more closed, when you're lookup up it will be more open, and when you look very far up you even instinctively raise your eyebrows to get them out of the way of your vision.
Character Creator 4 also has lots of tools for solved these character related problems. They have been doing it for over 20 years. Why not save yourself some time and focus on making a good games instead of solving these hard problems industry profrssionals have been solving for decades.
I'm a technical artist for 4 years now, I understand the joy of achievement to reinvent the wheels. But we have to be realistic when we work on our own, time is precious resource for indie game devs. It's practically the only resource you have, time and talent.