If you're curious about the license and terms of use, you can see it in detail here: github.com/ReForge-Mode/Radish_Tools_Phoneme_Extractor This is the longest video I've made so far. The process of research literally started three years ago. So if you want to support me on making these free useful tools, you can do it via Patreon and Ko-FI - www.patreon.com/ReForgeMode - ko-fi.com/reforgemode
Hey, recommend the best way to get vaginal and rectal blendshapes functioning using VROID models? Been working on a new electronics prototype and it's not as fun with only her mouth.
Yes you can! There are two ways to so it. You can either: - Copy paste the keyframes from one clip to the end of another clip. Not recommended because it could easily mess up the timing. - Use Timelines. Timeline is a feature in Unity that allows you to make cutscenes. It is a series of animation played one after another. Think of it like a video editor with multiple tracks for different objects. You would use Timeline for lipsync anyway in conversation, so might as well handle everything in timelines. I've included Timeline examples in the project files when you download this tool from my Ko-Fi page.
Thank you so much for this! I've been able to test your project, and I was wanting to import it into my existing project which is already using UniVRM 1. How hard would that be, or do I need to remove UniVRM and import your project as a package?
Glad I could help! You don't need to remove anything. Though, I would recommend doing the lipsync animation on a separate project and just take the animation files to your real project. Because when you try to build the game, the build will throw out errors. All codes I used in this project only runs in the Editor, not in the built version. You can run this in your project, but don't forget to delete my script or move the script to another folder before you build the exe file.
Glad you like it! Maybe I'll do it somewhere in the future! I'm sure there are some methods I can use. It could certainly be more accurate than this method!
Thank you for this! I had a question regarding loading custom audio files - I am unable to create the animations for them as it throws an ArgumentOutOfRangeException.
@@ReForgeMode It is a red error warning. The audio file is 16-bit .wav file. The error comes up when I am loading my audio.wav folder to Generate the LipSync Animation. To be precise, the error thrown says ArgumentOutOfRangeException : Index should be non-negative and less than the size of the collection. Thank you for your help!!
It seems to be a list issue. Can you post the screenshot on my Discord server? (Link is in the description) If there is any major error, I will need to fix it. Thanks for reporting this!
Hmmm... No, I don't think there is. But the source code is available, so if you want, you can build the executable yourself. Or maybe you can use Virtual Machine that runs Windows to do it. The Phoneme Extractor is not a heavy program by any means. Of course, I have very little experience on doing these, so you might have to do some googling.
Hi, maybe I'm too much of a newbie to Unity, but I can't get it to work in another model after importing into the scene, could you help me with that please?
Sure! Today is the right time to start learning Unity! Let me know what is the problem in details. You can also send screenshots so I can help you better by joining our Discord server! Open up a forum there and we'll help you directly.
Yes, I'm pretty sure this is compatible with any platform, since the end result is just an animation file. The only caveat is that you should not include this tool in your game build. I will update it soon so it will automatically be discarded once the project is built 👍👍
Hi, I got an error at compilation: "Assets\Scripts\CreateAnimationCurse\CreateAnimationCurve.cs(6,26): error CS0234: The type or namespace name 'AnimationUtility' does not exist in the namespace 'UnityEditor' (are you missing an assembly reference?) " Did I miss something?
Huh, it looks like Unity messing up again. Did you open the project in different version of Unity than what I noted on the github page? That could be the problem.
Oh, seems that this project is not intended for build. I found an explanation related to this problem: "UnityEditor is a namespace specific for Unity Editor internal stuff and custom editor scripts ... it is not available in a build so you don't want to use anything from it in a MonoBehaviour "
Oh no. The lipsync tool is not meant to be in built. This is because the function that generate the animation file can only be run in Unity Editor. it can't be run in the finished game. You only want to use the finished lipsync animation file in your final game build. But if you want to include this in your build and not cause errors, you can add: #if UNITY_EDITOR #endif This will make it so that this script won't be added into your final build, so no errors there.
Very cool! Do let me know how it works and when the video is uploaded! Of course, this video is quite outdated. My latest lipsync video is using an open source uLipSync plugin that captures mic sound in real time. But it's not accurate as I would like it to be. I was wondering on how accurate your script will be, and if it allows for live mic lipsync. If you open-source it, I think it will help many people in their gamedev journey. That's always been my goal after all.
@@ReForgeMode Hey, its a rather simple solution I came up with for my own game. It has no "live mic" option, so I'm sorry if it sounded misleading. It works with text based conversations. (Which can be coupled with a voiceover of the same text). I simpley provide a phonetic string, and the script loops over it, and moves the A, I, O, E, U blendshapes. So if the voiceover is "Good Night", i provide a text "gud nait" and it will give me a good enough result, by moving the U, A, I blendshapes, during the voiceover duration. Obviously i can improve the percision of this script, but for now its good enough. Today or tomorrow I will add a Short video to my channel showing how it looks, for a tutorial it will probably take me some time though.
@@Mikes-Code I see! So you're handing the automatic phoneme translation to the user instead. Yes, that would work for most cases. I wonder if it works for different people with different accent... I guess you can just change the phoneme translation. Then again, you won't notice the viseme that closely either. I still want to redo this tutorial sometime in the future. But automating the phoneme translation process is such a pain, because I need to rewrite the whole Radish tools into Unity. That's gonna be a struggle and a half. Looking forward to your video though! Are you sharing the repo publicly?
@@AaronAsherRandall Yes, the example in the intro is recorded from the engine. You are free to try it yourself. It's free unless you choose to donate to me.
Major Fail. I was hoping this system could be used on a custom made character, but your scripts are hardcoded looking for significant words. There is also no ability to link Blendshapes within your scripts
I will update this later. For now, you will need to modify the blendshape names in the script. This is no excuse for me not to make it easier to use. Thanks for notifying me. Could you tell me what you need so it could work in your character? Like the option to specify where the blendshape is located?
@@ReForgeMode Hey sorry for being an asshole. I tried editing the Clipcurve lines of your code but it wont link to the proper mesh -- not sure why, but its most likely a unity bug back from 2018. I bypassed this problem by manually copy pasting the actual keyframes into a new Clip that was created in the editor. Many thanks for the free tool.
Hello, this is great work. I appreciate the time you dedicate to it. I would really like you to help me switch to another model and use several at the same time in one scene. Please tell me how much you would charge me in dollars in a private message. Thank you very much.
Why yall acting like this looks good? Barely any of the phonemes is being reflected in the animation. If you're just post processing audio, you're better off faking it via audio amplitude and detecting specific phonemes like "O" and "U" than going this route.
