Fine Tune Transformers Model like BERT on Custom Dataset. 

📌 Hey everyone! Enjoying these NLP tutorials? Check out my other project, AI Demos, for quick 1-2 min AI tool demos! 🤖🚀 🔗 RU-vid: www.youtube.com/@aidemos.futuresmart We aim to educate and inform you about AI's incredible possibilities. Don't miss our AI Demos RU-vid channel and website for amazing demos! 🌐 AI Demos Website: www.aidemos.com/ Subscribe to AI Demos and explore the future of AI with us!

I searched lot read lot to solve one simple compony assessment problem but not able to solve...it ...as wont find any fine tunning video. You are gem

Hey Pradip. Your videos are very informative. Just a suggestion, instead of putting chapter numbers can you put a small description so that one can jump straight to the desired timeline

Great video!!! You just solved a proposed RFP at my work. Thanks Pradeep!!!

Thanks for the video, I can understand easily from your explanation.

It's a nice tutorial brother.

New subscriber here. Thanks for this clear explanation. I have watched a couple other videos of your and still watching but i have this question that you did not get to in this example because you had only 1 epoch. If i trained say for 10 epochs while tracking metrics (e.g., validation loss, accuracy or F1 score), if my best model was arrived at at the 6th epoch, how do i specify saving that 6th epoch? Thank you.

GREAT video! solved exactly what I was looking for.. thanks so much!

very nice video and well explained , well done !

Thanks for this video. Really helpful. Can you do a similar video for pretrained NMT model for let’s say Danish language?

Is it natural way to create custom dataset?! Can't believe you have to write custom class for this simple task.

Hi Pradip,whats the purpose of creating Pytorch custom dataset when we already have our own dataset

Brilliant hats off

Hey Pradip, for News Summarisation project can I fine-tune BERT with CNN/Daily dataset ? Will this perform better than the basic BERT model ?

Great explanation and the notebook works! I followed the notebook and fine-tuned a BERT model. I found two ways to use the model: tokenizer = BertTokenizer.from_pretrained('custombert') model = BertForSequenceClassification.from_pretrained('custombert',num_labels=2) ; tokenizer = AutoTokenizer.from_pretrained("custombert") model = AutoModelForSequenceClassification.from_pretrained("custombert"). Either way, I can't load the tokenizer. Is this because I didn't update the vocabulary? And what's the difference between "AutoModelForSequenceClassification" and "BertForSequenceClassification"? Thanks a lot!

Excellent!!

nice explanation dude

Hi @Pradip Nichite , Thanks for the great explanation :) I have a question: I have a machine generated data which is not natural language(Although the sequence of words in the data is important). I do not have any labels in the data, would it be wise to fine tune BERT and generate word embeddings using BERT? The idea is to check if BERT would generate more meaningful embeddings when opposed to word2vec skip gram. Thanks in Advance :)

Wow, thank you so much

Hi Pradip, thank you for this tutorial. I just want to ask you that do you have any tutorial for fine tuning BERT (or BERTology methods) for GENERATIVE question answering task? Hope you can see my comment. Thanks in advance!

Very Helpful

Hi Pradip, thank you for this tutorial. Is it possible to fine tune the BERT model to predict a multiclass output? For example, emotions rather than a binary classification like this example.

Amazing Thanks Man

Hi can we use the same code for distilbert or roberta as well?

Hi Pradip, this is a great video. Thanks for your efforts to create this for us. Could you please give me some advice to tackle the data privacy issues when using these pre-trained model from hugging face? I understood that when we import these pre-trained model and do training, we might be sending the private data that we are training through API? Based on your experience, if we want to secure the data from public but still enjoy the benefits of these pre-trained model, what would you recommend? I know hugging face is promoting their private hub demo. What do you think about that?

HI pardip, I was following your code and got this error Target size (torch.Size([8])) must be the same as input size (torch.Size([8, 2])) can you help me fix it? I was simply running your notebook in google colabb

Followed the same approach but getting this error for trainer.train() method Expected input batch_size (1360) to match target batch_size (16).

Hi Pradip, how can i solve this problem ? InvalidRequestError: The model `curie:ft-wrAQszDv88OVOWOQSjjqLZqe` does not exist

if the number of labels are 3 for example [positive ,negative , neutral] what are the changes of the code

Hi Pradip, thanks first of all for this great content! One question, I reproduced exactly your code from this tutorial and the model seems to work like yours in the video, however, it doesnt predict correctly the toxic label for inputs from the training-data. For example for the comment_text from Line 14 from train_data the label should be toxic = 1 but the model predicts almost 0 for toxic. Can you explain what is wrong? This is the comment_text from Line 14: Hey... what is it.. @ | talk . What is it... an exclusive group of some WP TALIBANS...who are good at destroying, self-appointed purist who GANG UP any one who asks them questions abt their ANTI-SOCIAL and DESTRUCTIVE (non)-contribution at WP? Ask Sityush to clean up his behavior than issue me nonsensical warnings... Is the reason that the model predicts "better" the toxicity than labeled in the train_data or "worse"?

This is BERT Mobile ?

Its not working at cell of #define trainer args=training_arguments please make one more video as soon as possible 🙏🏻

Can you explain the LOSS metrics please

👏👏👏👏👏👏👏👏👏👏👏👏👏👏

Basically. You wish to limit people's ability to express themselves and arbitrarily label them as "toxic". Gotcha.

Hi Pradeep. Can I please get your email id.