Тёмный

Spark Kafka Cassandra | End to End Streaming Project 

Data Engineering
Подписаться 174 тыс.
Просмотров 87 тыс.
50% 1

Опубликовано:

 

22 окт 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 51   
@aakashsengupta9026
@aakashsengupta9026 2 года назад
Thank you for the video. It was indeed a detailed explanation. Would request you to create more of End to End Project Explanation videos specially on projects related to Hive and Spark.
@neelamadhabkhaya98
@neelamadhabkhaya98 Год назад
Your voice and pitch is very high to motivate more to go through this video.
@dmraje
@dmraje 2 года назад
Sir this is awesome use case. It will be help full for us if you give the web app code and how to run it . This way we will get clear picture of how producers are integrated into web app
@Shubham-gd3oh
@Shubham-gd3oh Год назад
Hi can you create video on real time streaming platform Kafka and take the input data from json format will be very helpfull
@contentifye
@contentifye Год назад
Please share the code of the web application you created...
@shubhamwasnik5118
@shubhamwasnik5118 2 года назад
great explanation . I'm waiting for another project but please in intellij
@syedmushtaq4416
@syedmushtaq4416 Год назад
Great Project Use Case !!. Also, Can you please make a video on Challenges faced in real time stream processing and how to handle them ? 🙏
@KumarKumar-kx4ct
@KumarKumar-kx4ct 2 года назад
Comprehensive Tutorial! great job and thank you very much, I am stuck at 16:22 when using --jar as I keep getting "bad option --jar" appreciate if you can share exact command with an example
@wennie2939
@wennie2939 Год назад
Nice demo! One question though, in your example, can Cassandra DB be the consumer to read Kafka message without Spark? I'm just not seeing the purpose of Spark Streaming here.
@SathishKumar-mz7fw
@SathishKumar-mz7fw Год назад
yes, synch connectors are available to connect Kafka to Cassandra, but why Spark?
@sachindubey4315
@sachindubey4315 Год назад
@@SathishKumar-mz7fw spark is for ETL (mostly for T = data transformation ) here we didnt see but in real time scenario T are there . like to crean the row data
@jassyoldgoldies
@jassyoldgoldies Год назад
Everything is working fine for me but records are not shown in Cassandra Table when select * query is fired on cassandra table. Please suggest the solution.
@vijiinfo
@vijiinfo Год назад
awesome video. . may i know how do i look for spark straming code?
@kunuturuaravindareddy5580
@kunuturuaravindareddy5580 2 года назад
superb ,for people who always asks for a real time real time project .Thanks for your videos.
@theguruwithin_
@theguruwithin_ Год назад
Hi, after downloading the two jar files for Cassandra Connector and Kafka streaming , where do we paste these jar files???
@priyankavarahagiri3104
@priyankavarahagiri3104 8 месяцев назад
please upload installation of cassandra on aws ec2, or point me to the video link. i am having hard time installing it
@vuluc2408
@vuluc2408 2 года назад
I watched your video, I would like to be guided to create a website like in the video Thank you for reading
@jassyoldgoldies
@jassyoldgoldies 2 года назад
Any new video after this one in which problems faced in storing record into cassandra from spark? Everything is working fine except, record is not inserted in cassandra table. Any help?
@kalyanb6995
@kalyanb6995 10 месяцев назад
Hi I have one doubt what if the same user visits same product but different server did the value count increases in cassandra or it will add as new record???
@AyushGupta-pc8ue
@AyushGupta-pc8ue Год назад
Please tell how did you integrate your web application with Kafka (jps)
@jassyoldgoldies
@jassyoldgoldies 2 года назад
I am stuck at one point. I am using same versions of all applications which are used in your nice video and went nick to nick as per the video. I am not using JSP web application and only typing record in producer terminal and getting the response in Consumer Terminal but, record is not inserted in cassandra table. The Cassandra shell shows following warning scala> 22/10/26 12:02:06 WARN RandomBlockReplicationPolicy: Expecting 1 replicas with only 0 peer/s. 22/10/26 12:02:06 WARN BlockManager: Block input-0-1666765925800 replicated to only 0 peer(s) instead of 1 peers Please guide where I am wrong. With above problem solved will give me lot of confidence. Please help. Thanks
@jagadeeshkumar2278
@jagadeeshkumar2278 2 года назад
Why do we need a Spark in this use case. We can resolve the use case without Spark having a consumer application for Kafka (may be Spring Boot app) and write the same thing to Cassandra?
@dataengineeringvideos
@dataengineeringvideos 2 года назад
Yes we can , but when it involves core data processing with MPP then spark / storm like technologies needed
@vishalrana302
@vishalrana302 2 года назад
Hi can you make video on how did you connect jsp to kafka
@sharathchandra3632
@sharathchandra3632 Год назад
Bro Vera level explanation Great content for streaming in big data
@sitarampal6753
@sitarampal6753 11 месяцев назад
Very nice thank you
@wikiexportsindia3146
@wikiexportsindia3146 Год назад
Fantastic realtime examples
@zahraelhaddi6980
@zahraelhaddi6980 7 месяцев назад
can we follow up with th project in the free tier
@aasrithmadivada5727
@aasrithmadivada5727 Год назад
great explaination for beginners
@dhairyavasavada740
@dhairyavasavada740 Год назад
sir how to download packages for different version of spark and kafka
@mansoorhussain1155
@mansoorhussain1155 2 года назад
Can you pls create more projects pls
@santhanalaxmi1776
@santhanalaxmi1776 2 года назад
Bro please give some idea about batch processing project
@zentamil
@zentamil 2 года назад
Very useful one... Thanks for sharing the valuable one...
@sansha3881
@sansha3881 Год назад
Thanks you sir.
@krishnareddych7607
@krishnareddych7607 2 года назад
Highly appreciated for making this nice video
@lithesvar1920
@lithesvar1920 2 года назад
Finally 🔥.very useful one thanks bro.
@sairamanamara5335
@sairamanamara5335 2 года назад
This video really helps , thanks.
@contentifye
@contentifye Год назад
please provide jsp code...
@neelamadhabkhaya98
@neelamadhabkhaya98 Год назад
Nicely explained
@tejashwersai2464
@tejashwersai2464 Год назад
can you please share kafka producer java code
@zakariaelaaliji7747
@zakariaelaaliji7747 8 месяцев назад
the code please
@hell-o8470
@hell-o8470 2 года назад
Can you please provide the jsp code?
@tusharmore7801
@tusharmore7801 2 года назад
where can i download the project ? good vid.
@dataengineeringvideos
@dataengineeringvideos 2 года назад
Thanks , please find the code link in the description , where I have shared the complete steps and code
@tusharmore7801
@tusharmore7801 2 года назад
@@dataengineeringvideos Ok are all your projects available and downloadable in your github ?
@maheshbhatm9998
@maheshbhatm9998 2 года назад
Thank you it was a good session
@chundrugopalakrishnachowda4228
@chundrugopalakrishnachowda4228 2 года назад
Rajni kanth of Data Engi
@knightganesh
@knightganesh Год назад
Super bro 👍
@feelfree7162
@feelfree7162 2 года назад
I'm getting an error like java.lang.NoClassDefFoundError: scala/Product$class
@dhineshsubramani5959
@dhineshsubramani5959 2 года назад
@Data Engineering Same issued i faced. In the shell terminal it shows like this "java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V"
Далее
Do Data Engineers Need Cloud Computing ?
8:55
Просмотров 6 тыс.
Cool Wrap! My Book is OUT 🥳
00:27
Просмотров 930 тыс.
Real time End to End PySpark Project
25:00
Просмотров 56 тыс.
The ONLY PySpark Tutorial You Will Ever Need.
17:21
Просмотров 138 тыс.
Apache Kafka in 6 minutes
6:48
Просмотров 1 млн
Streaming from Kafka Source
16:56
Просмотров 6 тыс.