Тёмный

Lambda Error Handling features for Kinesis Data Streams and DynamoDB | Kinesis Error Handling Demo 

Ajay Wadhara
Подписаться 7 тыс.
Просмотров 2,2 тыс.
50% 1

Kinesis Error Handling if using Lambda as a consumer. When using Lambda as a consumer, it becomes very tricky to handle poison messages (or problematic messages) which can cause your Lambda to fail and keep on re-trying the same batch of messages again and again.
In this video, I have covered Lambda Error Handling features and Retries in the case of Kinesis Data Streams.
Do watch the theory part if you are not clear about the tricky issues which you might encounter if you are using Lambda with Kinesis and not using Lambda Error Handling features. The link is below:
Lambda Error Handling with Kinesis (Theory Part): • Lambda Error Handling ...
Kinesis Producer Example (with Java): • Kinesis Data Streams J...
GitHub Repo: github.com/ajaywadhara/Lambda...
Chapters:
00:00 Introduction
01:17 Create Lambda
01:40 Create Kinesis Data Stream
02:07 Add Kinesis Trigger to Lambda
03:29 Send Stream of Messages to Kinesis
05:02 Write Lambda Code
09:45 Test Lambda with Error Handling features
12:26 Add Error Handling options in Lambda
14:19 Test Lambda Again with Split Batch and Retries
16:02 Check SQS (DLQ)
17:40 Complete Flow with Error Handling
#KinesisErrorHandling #LambdaTutorials #AwsTutorials

Опубликовано:

 

19 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 11   
@asupat12
@asupat12 Год назад
Thank you for the video. Such videos are very practical and to the point.
@nitinkulkarni7942
@nitinkulkarni7942 Год назад
Excellent Tutorial.. Very well done. Thank you
@eshwarnag
@eshwarnag Год назад
Excellent Demo! Answered all my questions.
@AjayWadhara
@AjayWadhara Год назад
Glad you liked it
@nitinkulkarni7942
@nitinkulkarni7942 Год назад
I cant believe there are only 434 views. This deserves a lot of views.
@AjayWadhara
@AjayWadhara Год назад
😀 RU-vid’s algorithm is harsh. But i do get motivated by such comments. Thanks 🙏🏻
@nitinkulkarni7942
@nitinkulkarni7942 Год назад
Incase of bisect function on error, lets say we get 100 messages. We process first 49 messages and publish them to SNS topic. Then the 50th message is an issue. In this case the batch will be divided into 2. I am still going to get all the messages that I already published to SNS right? I will end up publishing the messages multiple times until that bad record is isolated. How do we solve this issue?
@AjayWadhara
@AjayWadhara Год назад
Hey Nitin, you have to think about making your system Idempotent. You might have to add another service such as DynamoDb that takes care of this and use SNS with DynamoDb streams. Also think about Event Filtering if that is possible in your use case
@meashish
@meashish Год назад
I wanted to know, how to redrive the messages in DLQ back to Lambda once I fixed the Lambda code.
@meashish
@meashish Год назад
I had to write a script for redriving the message in DLQ. 1. The script first take the shardId and sequenceNumber from the DLQ message. 2. Fetches the record body from Kinesis using shardId and sequenceNumber. 3. Construct the Lambda input. 4. Invoke the Lambda. 5. If lambda invocation is successful then delete the message from DLQ. If it is more than 24 hours and the data retention for Kinesis is 24 hours then you won't be able to get the message from Kinesis later. Default retention period for kinesis is 24 hours.
@AjayWadhara
@AjayWadhara Год назад
Perfect solution. 👍
Далее
КАКУЮ ДВЕРЬ ВЫБРАТЬ?😱
00:43
Просмотров 1,6 млн
Что с Аминой из кампуса?
00:46
Просмотров 1,8 млн
지민 (Jimin) 'Who' Official MV
03:28
Просмотров 10 млн
КАКУЮ ДВЕРЬ ВЫБРАТЬ?😱
00:43
Просмотров 1,6 млн