Understanding the Llama 3 Tokenizer | Llama for Developers 

TLDR - from llama 2 to llama3 they switched from sentencepiece to tiktoken - vocab size 32k -> 128k - ~15% fewer tokens for english, ~50% fewer for "some other languages"

Aston please explain the attention mechanism , Actually I am stuck in the chapter "Attention and transformer" of your book d2l

My instinct is tokenization is underestimate in importance. Usually the hard and boring fundamentals are where the magic happens. Across all fields.

Informative

Is the Llama 3 paper out yet? He mentions it @ 24:02

could someone from the meta LLaMa 3 team please explain how to train my very own tiktoken tokenizer like you guys did for llama 3. there is no opensource steps to recreate this

You develop Tamil language for Tamil users

so this guy is payied to use open sourced tiktoken

ok.

x

Classic example of a provably smart guy not being able to express his thoughts... 5 minutes of pain is all I managed to force myself to watch. A shame.

i don't think this format works unless the intent is to discuss at a high level.