[ARCHIVED] ViViT & NaViT papers: How Sora encoded space-time patches | Shawn's ML Notes

Подписаться 913

Просмотров 1,5 тыс.

50% 1

⚠️ An updated version of this video is available at: • [UPDATED] ViViT & NaVi...
Update on April 8th, 2023 (only in the updated video, link above):
- Fixed missing narration on slide 25
- Added explanation for accuracy increase from upsampling (thanks to @ryuku4966 !)
- Amplified audio track
--
Thank you for checking out my video notes on ViViT & NaViT papers: how Sora encoded space-time patches! I would love to share my ML learning journey with you.
Paper information:
- Arnab, Anurag, et al. "Vivit: A video vision transformer." Proceedings of the IEEE/CVF international conference on computer vision. 2021.
- Dehghani, Mostafa, et al. "Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution." Advances in Neural Information Processing Systems 36 (2024).
Please let me know in the comment section regarding any questions, points of discussion, or anything you would like see next. See you in the next video!

Опубликовано:

26 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 16

@moienr4104 8 дней назад

I just started my PhD in ML, and I have to say, the way you explained this paper is elegant and amazing! Kudos, and I subscribed. I hope you go through more SOTA papers :))

@yuxiangwang9624 6 дней назад

Glad it was helpful! :)

@AlejandroAristizabal-wo2zg 5 месяцев назад

Awesome video! I've always wanted to delve into ViT but haven't had the time. This video really did help reinforce my understanding, as well as add some really insightful details into all of these new methods. Thanks!

@yuxiangwang9624 5 месяцев назад

Thanks for appreciating and leaving a comment! :)

@ryuku4966 5 месяцев назад

Nice vid. Could be when you upscale it works better cause then its like the model is looking at smaller patches. An interesting ablation would have been to consider smaller patch size and check

@yuxiangwang9624 5 месяцев назад

Aha that's a good explanation! Makes perfect sense to me. I appreciate the reply & feel happy that I learned more through sharing!