Train a VITS Speech Model using Coqui TTS | Updated Script and Audio Processing Tools 

Hi I hope you saw the download link for the hindi model, if not its in the video description. Let me know if it works at all lol The voice quality isn't very good, but the starting samples weren't clear either. You can probably use it as a base for training a clearer sounding voice though, assuming the character set was mostly correct

Yeah please shoot me an email. I probably won't read it for a few days but it'll be there. I'm in poor health so I don't have a lot of time to work on things lately. If you haven't tried deepfilternet for denoise, try poking around its command line application. Easy to use and works really well a lot of the time. Voicefixer is another, but it's a bit of a pain.

@@nanonomad Get well soon!

Thanks! I think that did the trick. Much appreciated

Sorry I missed this. Which bit? You can make them manually if you have to

@@nanonomad I put them in manually now but now tensorboard does not work and this pops up down the line "AssertionError: [!] No training samples found in /content/drive/MyDrive/vits-vctk-narrator-22k-ds/". I'm new to coding and python, sorry for being a bother.

@GZILLA is that when you're trying to run the training cell with trainer.fit() at the end? That suggests that the dataset isn't found or structured correctly. Did you make it using the colab notebook tools? Edit: I just noticed the dataset name and path are the same as I was using. Is yours in a directory on gdrive named vctk-narrator-22k-ds? It doesn't have to be. You can put it wherever, but the cell with the variables at the top has to be changed, and then run after the changes. The variables won't get set until the play button is hit.

In the VITS videos I use the VCTK dataset format. The dataset format is handled by the data loader. You can write your own loader function, I'm just lazy, so I adjust my datasets to follow the vctk format instead. Audio files are in a sub directory wav48_silence_trimmed/speakername Text files in a sub directory txt/speakername Audio files are 22k mono flac format named ending with _mic1.flac

Back up your checkpoints so they dont get overwritten that way you can A/B test if you need to and not lose progress. You can try a restore run from the ckpt and reinitilize the DP, but it generally seems to train well unless you have a strange dataset. Has it been getting steadily worse? It may just be "that thing" that models seem to do where they fluctuate in output quality (in spite of the graphs looking fine). Could just need more training and to be pulled out at just the right spot in the cycle. Also may need to adjust the loss target or lower LR if you're not getting best loss checkpoints

If you need to make a dataset by hand I would recommend using Audacity to split audio, or use a ML voice activity detection model. I guess I broke sox, but it was a crude method anyways. With audacity use the segment on silence tool, then file save multiple and it'll save each segment separately. It will handle overlapping silences to keep sentences whole if you fiddle with the noise floor slider when segmenting

I'll try to look at it today, but if it's just one of the processing sections you can probably skip it. Which segment was broken? Or is it more than one part?

Yourtts is a derivative of vits, with the speaker encoder and some arch changes. But I find yourtts very inconsistent. Probably vits using dvectors is what I would try. But 100 samples could be 2 mins or 9 mins so that's a pretty big range. I'd say if you have around 5-9 mins of great audio with a good distribution of phonemes you may be able to make it work. You'll probably have to adjust the eval split because the loader is going to complain about the small dataset

@@nanonomad I get nonetype has no length at some point during training when trying to continue and it stops. Specifically on rev4 notebook any advice?

12gb comfortably, but iirc you can get it going on 6gb if you lower the batch size, lower the learning rate so it trains properly, lower the max text length

Should be able to. I just did a quick search and found this: github.com/coqui-ai/TTS/discussions/1604 There may be a Japanese language model already trained on the Coqui model hub, but I'm not sure.

Weird. I checked and I have all the filters disabled. Your not the first person that I've heard that from though. A lot of my comments vanish, but sometimes reappear weeks later. Quite frustrating

Hi You may be able to fine-tune an already trained model if there is a Chinese model on the Coqui hub. You will need to research to see if it was trained using phonemes or characters and continue training with that method. If training a new model from the beginning, you will need to decide if you want to train using characters or phonemes. If using phonemes, you need to see if there is support for the language in the phonemizer (espeak, espeak-ng, or gruut). I think espeak has good Chinese support, but you will want to check the documentation to verify that. Phonemes train easier and faster than characters, so if you can, use phonemes. I posted a video about training a Spanish language model a while ago - that notebook uses phonemes. For Chinese, you would need to change the two letter language code in the phoneme_language line, and set the language code for the dataset Check the Coqui github discussion board, though. A few people have posted about training Chinese models. You may be able to work with them or see how they did things