Finetuning Gemma 2B (w/ Example Colab Code)

Подписаться 78 тыс.

Просмотров 9 тыс.

50% 1

In this crash course to fine-tune Google Gemma, Adithya SK teaches us the end-to-end process of model fine-tuning.
Steps involved:
1. Setting up the environment
2. Loading the Model & Setting up Chat Template (Prompts)
3. Loading the Dataset and formatting the datset
4. LoRA Config
5. Model Fine-tuning (Training) Gemma and Push to Hugging Face Hub
Finally Testing and Closure!
🔗 Links 🔗
A Beginner’s Guide to Fine-Tuning Gemma Blogpost - / a-beginners-guide-to-f...
Colab used in the code - colab.research.google.com/git...
Follow Adithya:
Twitter - / adithya_s_k
Github (LLM Alchemy) - github.com/adithya-s-k/LLM-Al...
❤️ If you want to support the channel ❤️
Support here:
Patreon - / 1littlecoder
Ko-Fi - ko-fi.com/1littlecoder
🧭 Follow me on 🧭
Twitter - / 1littlecoder
Linkedin - / amrrs

Наука

Опубликовано:

1 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 33

@vamshi3676 5 месяцев назад

Simple and Clear. Thank you.

@GAllium14 5 месяцев назад

Best vid I saw on this topic ❤❤

@IamalwaysOK 4 месяца назад

Thank you for the informative video; it was fantastic! Could you provide guidance on using any medical dataset for similar task?

@intelligenzaartificialeitalia 5 месяцев назад

Love u Sir , your work is insane ❤ can u make a video about fine tuning gemma on best RAG dataset 🙏🙏

@shakeelvohra 5 месяцев назад

once the model got ported to huggingface, could not run inference in hugging point with error saying "The checkpoint you are trying to load has model type `gemma` but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date."

@thesuriya_3 5 месяцев назад

bro i used the same code but after initialise the model and tokenizer thn i run inference its not predicting the answer and aslo im not getting error input shape and if i use gemma 2b not predicting anwer it blank why ?

@johnharryduavis3414 4 месяца назад

Hi bro what if we only have 2 column for our datasets like Question and answer. Can you help us?

@theenriquecarbo 5 месяцев назад

Yesss

@ravishmahajan9314 5 месяцев назад

I have just one question : On functional level i got what you were doing, but how did you learn all these on technical level? I want to learn all these. Can you please provide me some resources from where i can learn all these? I have some experience on Python.

@sk8l8now 5 месяцев назад

Thank you so much. I have a quick question. I'm trying to run this in a Conda JupyterNotebook on Windows 11. But I'm having an issue with BitsandBytes (I think it doesn't allow for int4 on Windows). Do you know if it's possible to finetune gemma in my Jupyter Notebook or if it has to be done on Linux or a collab?

@shakeelvohra 5 месяцев назад

same for me on colab - says pip install accelerate but i already have

@user-bd3uu3fz2d 5 месяцев назад

How to chat with github codebase /local codebase using gemma ?

@jahanzaibasgher1275 5 месяцев назад

How to know the context window size for gemma? What if I train it on longer sequences... that is why I wanna know.

@sk8l8now 5 месяцев назад

It's just under 9k tokens.

@nhanguyenthe3551 4 месяца назад

Thank you for the informative video. I finetuning model Gemma 2B instruct. I searched on the internet and found the prompt formatted as follows: INTRO_BLURB = "Below is an instruction that describes a task. Write a response that appropriately completes the request." INSTRUCTION_KEY = "### Instruction:" INPUT_KEY = "Input:" RESPONSE_KEY = "### Response:" END_KEY = "### End". Does it work? Or both format prompt = """user {prefix_text} {data_point["instruction"]} here are the inputs {data_point["input"]} model{data_point["output"]} """ work.? Thank you?

@jayavibhavnk3688 4 месяца назад

my colab keeps crashing while reloading the model, help pls

@susdoge3767 2 месяца назад

great resource, why didnt we do any kind of padding etc?

@susdoge3767 2 месяца назад

datacollator does it, thanks

@serkandayicik3736 5 месяцев назад

we can fine-tune gemini to other language ?

@1littlecoder 5 месяцев назад

it's got a huge vocabulary so might have some benefit if that language is part of the Gemma base

@vamshi3676 5 месяцев назад

I have a question, why are you saving the model and merging the model with original and then pushing to hub. why not push the model directly to hub after tuning?

@1littlecoder 5 месяцев назад

The fine tuned result would give you only the lora adapters, if you push directly, then you won't get the complete model

@vamshi3676 5 месяцев назад

@@1littlecoder Can you make a video explaining what is merged 4 bit, merged 16 bit, GGUF, CPP, its all confusing. If you have any resources that can help me out on this topic, it would be really helpful.

@vamshi3676 5 месяцев назад

@@1littlecoder Thank you so much.

@zyxwvutsrqponmlkh 5 месяцев назад

Why fine tune the worst model ever released? Why not fine tune an actual useful model instead?

@1littlecoder 5 месяцев назад

do you actually think it's the worst model?

@zyxwvutsrqponmlkh 5 месяцев назад

@@1littlecoder Yes, well IDK specifically about the 2 billion version, but the 7b is by a mile the worst 7b model I have seen in a long time. IDK what they did to cheat on benchmarks but try throwing stuff it it, it's in an unusable state. Frequently it cant even spell right. And that's at 7b.

@zacboyles1396 5 месяцев назад

@@1littlecoderyes, the worst, and it’s considerably racist. Not racist like some users spent lots of time tricking it into saying something offensive or racist, no, Google’s model is excessively racist - with full intention to be racist - right ‘out of box’.