Graph Language Models EXPLAINED in 5 Minutes! [Author explanation 🔴 at ACL 2024]

Просмотров 2,7 тыс.

% 170

How to make powerful LLMs understand graphs and their structure? 🕸️ With Graph Language Models! They take a pre-trained language model and fit it with the ability to process graphs. Watch if you're curious about how this works (hint: choose the right positional embeddings)!
📃 Moritz Plenz and Anette Frank. 2024. Graph Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand, ACL 2024 aclanthology.org/2024.acl-long.245/
💻 Code for Graph Language Models: github.com/Heidelberg-NLP/GraphLanguageModels
Follow Moritz Plenz (first author) on:
➡️ LinkedIn: www.linkedin.com/in/moritz-plenz-72478a2ba
➡️ Twitter / X: x.com/MoritzPlenz
Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael, Sunny Dhiana, Andy Ma
Outline:
00:00 LLM for graphs
01:08 Motivation
02:02 Key idea of Graph LLMs
02:25 Relative Positional Encodings
03:00 Method (Graph LLMs)
04:04 Experiments and Evaluation
04:49 Results
06:07 Outro
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: www.patreon.com/AICoffeeBreak
Ko-fi: ko-fi.com/aicoffeebreak
Join this channel as a Bean Member to get access to perks:
ru-vid.com/show-UCobqgqE4i5Kf7wrxRxhToQAjoin
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
AICoffeeBreakQuiz: ru-vid.comcommunity
X / Twitter: AICoffeeBreak
LinkedIn: www.linkedin.com/in/letitia-parcalabescu/
Threads: www.threads.net/@ai.coffee.break
Bluesky: bsky.app/profile/aicoffeebreak.bsky.social
Reddit: www.reddit.com/r/AICoffeeBreak/
RU-vid: ru-vid.com
Substack: aicoffeebreakwl.substack.com/
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research
Video editing: Nils Trost
Music 🎵 : Space Navigator - Sarah, the Illstrumentalist

Наука

Опубликовано:

6 окт 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 7

@bensimonjoules4402 2 дня назад

Its interesting to see "attention" on graph structures again. I think in the future a more structured knowledge representation may play a role on improving reasoning, as we could leverage logic and rules using engines on them, like compilers aid in code generation.

@jmirodg7094 День назад

Excellent! need to go deeper that could be a game changer for reasoning, as it makes more sense to reason on a graph rather than on the next token.

@bharanij6130 2 дня назад

Thank you for this video Letitia! As always amazing :=) Side note: Loved the silent Good Bye :)))

@quebono100 2 дня назад

It would be interesting how much computation this needs

@AICoffeeBreak 2 дня назад

Do you mean for training or inference? Training is a finetuning setting and you can see performance curves in Figure 4 in the paper. arxiv.org/pdf/2401.07105 Inference costs as much as the base LLM.

@quebono100 2 дня назад

@@AICoffeeBreak both things. overall computation. Thank you, I will take a look

@MoritzPlenz 8 часов назад

Hi, I am Moritz (one of the authors). I don't have much to add to Letitia's reply, but here is another relevant part of the paper, taken from section 4: Being transformers, GLMs have the same computational complexity as their respective LM. For sparse graphs the lGLM could make use of sparse matrix multiplication, making it more efficient than a corresponding LM or gGLM. However, for our experiments this was not necessary.