Transformer Attention for Time Series - Follow-Up with Real World Data

Подписаться 423

50% 1

In a previous video ( • Transformer Attention ... ) I looked at an approach to using Transformer Attention in time series forecasting. The data used to test the model in that video was extremely simple. In this video, the model is tested against more complicated data and some implications of the model are discussed.
Code:
github.com/BrandenKeck/pytorc...
Attention Is All You Need:
arxiv.org/pdf/1706.03762.pdf

Опубликовано:

18 апр 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 7

@Stacker22 Месяц назад

Love the video's and your presentation style!

@lets_learn_transformers Месяц назад

Thank you!

@rariwa День назад

do you have solution for long seq to seq problem? I need to predict seq of vector (15k size) for lets say 100 steps ahead. NN.linear always gives me memory error.

@elmo.juanara Месяц назад

Thank you for your knowledge sharing. Can the code run on the jupyter notebook as well?

@lets_learn_transformers Месяц назад

Thanks @elmojuanara5628! The code should run just fine in a notebook - some additional work may be required based on GPU availability of the notebook, but I believe some services such as Colab handle this very well for CUDA.

@AtousaKalantari-y4w 4 дня назад

Why didn’t you use positional encoding?

@lets_learn_transformers 2 дня назад

Hi @AtousaKalantari-y4w, in later videos I began using positional encoding as implemented in PyTorch. However, in this video and the one prior, I used only vanilla attention. Positional encoding is a general improvement and I believe that it should be used in almost all cases!