Тёмный

How to deal with late data | Event vs Processing Time | Stream Processing 

Code with Irtiza
Подписаться 13 тыс.
Просмотров 3,6 тыс.
50% 1

Опубликовано:

 

1 окт 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 9   
@anidea8012
@anidea8012 Год назад
I'm currently working on a project in which I'm getting event time > process time; for example, in my database, the process time happened in April and my event time happened in May. How is this possible? Please help me. Or is this any kind of cyberattack?
@irtizahafiz
@irtizahafiz 10 месяцев назад
Can there be errors when recording the event time? Maybe the events were sending the wrong timestamp to begin with? Or vice-versa, maybe the process time is recorded incorrectly.
@JardaniJovonovich192
@JardaniJovonovich192 2 года назад
have you used Kafka streams in production? If yes, what tool have you used to load test such application?
@irtizahafiz
@irtizahafiz Год назад
Hi! I work at a Product team, not an Infra one. So unfortunately I am not really sure what tools are used. They are abstracted away from us.
@kushangowda
@kushangowda 10 месяцев назад
Why are you over engineering Why don’t you use timestamp to identify?
@irtizahafiz
@irtizahafiz 10 месяцев назад
If you are doing window-based aggregations, and your event arrives beyond the window "grace period", you will either ignore the data or need to recompute.
@kavinkumar2633
@kavinkumar2633 7 месяцев назад
For systems like ad aggregator, it makes sense to ignore the late data ,correct?@@irtizahafiz
@bjugdbjk
@bjugdbjk 2 года назад
Good to see u back buddy !!
@irtizahafiz
@irtizahafiz Год назад
I will be back again haha.
Далее
Window Aggregations | Stream Processing
10:33
Просмотров 2,9 тыс.
Лиса🦊 УЖЕ НА ВСЕХ ПЛОЩАДКАХ!
00:24
Bearwolf - GODZILLA Пародия Beatrise
00:33
Просмотров 399 тыс.
Event Time and Watermarks | Apache Flink 101
11:58
Просмотров 12 тыс.
Watermarking your windows
12:43
Просмотров 2 тыс.
Design an Autocomplete System | System Design
19:13
Просмотров 8 тыс.
Stream Processing 101 | Basics
10:17
Просмотров 10 тыс.
Data Streaming, Explained
11:55
Просмотров 33 тыс.