BERT vs GPT 

true

Remember it as INPUT>MODEL >OUTPUT MODEL INPUT (NLU) - txt recognition, vision recognition/image/movie, sound recognition/voice, MODEL OUTPUT (NLG) -txt generation, image/movie generation, sound/voice generation + tool integration MODEL PROCESSING -Basic (classification summeration extraction) advanced (reasoning, planning, orchestration)

What is cross attention

@@Tech_kenya It's when word vectors reference other word vectors as opposed to just referencing themselves.

@@methylphosphatePOET so kinda the opposite of self attention?

Thanks a lot :)

there is also Whiser model, that has similar facebook BART decoder part, but has audio decoder.

Not when it's decoding. No.

Thanks So much