LLM basics #1 with the LLM Science Exam Kaggle Competition - Zero-Shot approaches

Подписаться 4,6 тыс.

Просмотров 3,7 тыс.

50% 1

Talking about ways to use an off-the-shelf language model to solve a multiple-choice task. Covering:
- Intro to the Kaggle competition
- Benchmarking with GPT3.5
- Using the OpenAI function calling API to enforce structure on answers
- Using Llama2 as a classifier by examining the logits (next token predictions)
- Using perplexity to evaluate question-answer pairs
Notebook using the OpenAI API to test GPT3.5: www.kaggle.com/johnowhitaker/...
Llama2 demo notebook: colab.research.google.com/dri... (quickly made for this video, don't trust the calculations, rather start with the below notebook)
Notebook testing different open models with the perplexity approach: www.kaggle.com/code/takamichi... (a good template to start experimenting since it shows how to run as a submission.

Игры

Опубликовано:

6 авг 2023

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 12

@kiriyama0 Год назад

This is pretty amazing and very informative, I am looking forward for the next videos of this series. Thank you!

@numannebuni Год назад

Very interesting, thank you! The notebook also helps me in my journey to have more control over LLMs output.

@uselessrobotics5383 10 месяцев назад

Hello, Just wanted to say thank you. Sometimes you just come accross a hidden gem and that makes your day :). Thank you for taking the time to share with us. The concepts are unusual and intersting.

@GiovanneAfonso 7 месяцев назад

Very informative, thank you for sharing! Very cool how you calculated the final score, it seemed very complicated on kaggle

@bikashpatra119 Месяц назад

Thank you for this nice getting started video. Could learn a lot from it. One question, did you write the function json schema or you used any function to generate the schema.

@utkarshx27 Год назад

As a beginner, it is quite beneficial to me.

@datasciencecastnet Год назад

Jeremy hsa created a very nice notebook going deeper into a GPT3.5 baseline that beats the example shown here thanks to some more thoughtful prompt engineering: www.kaggle.com/code/jhoward/getting-started-with-llms/

@mappu7612 11 месяцев назад

I don't see the difference between prompt calling chat gpt and function calling chat gpt. Both are still in a format we ask the model to respond. So, curious. Whats the advantage?

@datasciencecastnet 11 месяцев назад

Function calling is much more likely to follow your desired structure. If you're just prompting you may get "Sure, I can help you create a question! Let's start:..." instead of just the thing you want.