No video :(

rdd in spark | Lec-9

Подписаться 21 тыс.

Просмотров 25 тыс.

50% 1

In this video I have talked about rdd in spark in great detail. please follow video entirely and ask doubt in comment section below.
Directly connect with me on:- topmate.io/man...
For more queries reach out to me on my below social media handle.
Follow me on LinkedIn:- / manish-kumar-373b86176
Follow Me On Instagram:- / competitive_gyan1
Follow me on Facebook:- / manish12340
My Second Channel -- / @competitivegyan1
Interview series Playlist:- • Interview Questions an...
My Gear:-
Rode Mic:-- amzn.to/3RekC7a
Boya M1 Mic-- amzn.to/3uW0nnn
Wireless Mic:-- amzn.to/3TqLRhE
Tripod1 -- amzn.to/4avjyF4
Tripod2:-- amzn.to/46Y3QPu
camera1:-- amzn.to/3GIQlsE
camera2:-- amzn.to/46X190P
Pentab (Medium size):-- amzn.to/3RgMszQ (Recommended)
Pentab (Small size):-- amzn.to/3RpmIS0
Mobile:-- amzn.to/47Y8oa4 ( Aapko ye bilkul nahi lena hai)
Laptop -- amzn.to/3Ns5Okj
Mouse+keyboard combo -- amzn.to/3Ro6GYl
21 inch Monitor-- amzn.to/3TvCE7E
27 inch Monitor-- amzn.to/47QzXlA
iPad Pencil:-- amzn.to/4aiJxiG
iPad 9th Generation:-- amzn.to/470I11X
Boom Arm/Swing Arm:-- amzn.to/48eH2we
My PC Components:-
intel i7 Processor:-- amzn.to/47Svdfe
G.Skill RAM:-- amzn.to/47VFffI
Samsung SSD:-- amzn.to/3uVSE8W
WD blue HDD:-- amzn.to/47Y91QY
RTX 3060Ti Graphic card:- amzn.to/3tdLDjn
Gigabyte Motherboard:-- amzn.to/3RFUTGl
O11 Dynamic Cabinet:-- amzn.to/4avkgSK
Liquid cooler:-- amzn.to/472S8mS
Antec Prizm FAN:-- amzn.to/48ey4Pj

Опубликовано:

29 авг 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 63

@rajat1999gupta День назад

Man, "How to and what to" point you mentioned in the video was magical, cleared the whole picture

@biswadeepdatta5218 8 дней назад

Best explanation. You are sent by God.

@laboni8359 11 месяцев назад

best explaination of RDD i found on youtube, and no doubt one of the best teachers !

@vishaljare1284 8 месяцев назад

the way you are teaching is excellent and easy manner.

@sabarnaghosh1658 Год назад

please be consistent,and post video ,,i am depending on you to switch my career to data engineering within 2 months

@sekirrom4169 2 месяца назад

best one sir , you showed us the real process behind the scenes. Thanks

@codingwithanonymous890 10 месяцев назад

Every where rdd had same meaning but no proper explanation thank you sar

@user-zo9nz9uj4i 3 месяца назад

Hi Manish, Your lectures are amazing and full of knowledge. Thanks a lot for uploading such beautiful conceptual videos.

@amarzond5253 4 месяца назад

Love you sir thank you so much......❤❤❤ बोहोत अछा सिखाते हो

@rishav144 Год назад

very well explained

@ig2947 5 месяцев назад

Brilliant..!! Thank you so much for your efforts in teaching

@akumar2575. 4 месяца назад

i started watching lecture from last one week and now from today i am going to complete the lecture within 10 days // target started from today let's support me for this and i know i am a slow learner but i can do it 🙂// counting started from toady date 20/04/24

@manish_kumar_1 4 месяца назад

Isko jaldi complete karne me bahaduri nhi hai. Take your time and try to understand in great depth.

@akumar2575. 4 месяца назад

@@manish_kumar_1 ok and thank you

@user-zr7zb8zn4m 6 месяцев назад

Best explained, the way to teaching is very clear. thanks for providing such content on youtube. I have one request, could you please create a video on difference between RDD, dataframe and Datasets

@sankuM Год назад

Another good one, @MANISH KUMAR!! 🙌 By any chance, do you still work with them at Jio? 😅😅 Also, can you share what type of architecture are you working with along with data volume? Keep continuing this series! I'm preparing for #spark for my next switch with your series!! :) Thanks a lot again on behalf of all DEs!!! 👍👏👏

@nitilpoddar 7 месяцев назад

Hi @sankuM it has been 8 months, can I ask you about your progress and how has it been going with your switch ?

@sankuM 7 месяцев назад

Hey@@nitilpoddar, it is going good only! I switched in May last year only! Knowing about fundamentals and optimizations thereof really help in day-to-day work I feel now!!!

@harshalunhale4369 4 месяца назад

at 12:15, I think it should be age < 18 for RDD1 and then again age < 10 for RDD2 🙂

@ChandanKumar-xj3md Год назад

hard concept but you made a very easy explanation.👍

@HanuamnthReddy 7 месяцев назад

Best yotube channel....for learning 😊

@Abhinavkirtisingh Месяц назад

mast samjhaya bhai maza aa gya!! Thank you

@user-qx6zr8hy1v 2 месяца назад

Thank you sir for best tutorial ❤

@satyamkumarjha4185 2 месяца назад

Hi Manish, when is the RDD originally created? Is it created just after JVM code is generated by spark engine? Thanks for all the work you do.

@siddharthdang1521 4 месяца назад

hi Manish, your explaination is very good. Keep us teaching and inspiring always!!! Do we deal with unstructered data on a frequent basis, if yes then is it through RDD always? If possible can you give some example or make a video on it.

@kartikgupta2299 23 дня назад

Pehele aapne kaha features of RDD me ki optimization or disadvantage me No optimization Mtlb kya hai Or full control on our data chahie to rdd use krna chahie ye point clear kro

@rajun3810 5 месяцев назад

excellent content #Love you Manish Bhai

@shreyanshrathod2007 3 месяца назад

Thanks for the explanation, Manish. One doubt, when we apply multiple transformations and generate new RDD's, are all the previous RDD stored in the memory until an action is called? Thanks in advance

@vijayvhanale8749 5 месяцев назад

really amazing explanation, thanks :)

@ChandanKumar-xj3md Год назад

Q. When you say 500MB/128MB = 4 Partition, is this means that every single partioned data will be stored over each node ?

@manish_kumar_1 Год назад

Yes this partitioned data is distributed over the network

@adityaverma4770 9 месяцев назад

@@manish_kumar_1 each node may have different capacity, right? it is not necessary that each node should accommodate only one partition. say we have each node with capacity of 300 mb so in this case it can accommodate 2 partitions of 128 mb each.please confirm?

@pravinyadav8372 8 месяцев назад

@@adityaverma4770 yes it can execute two partitions as well in a single node but the executor should have multiple cores. If it is only having 1 core then one executor will only execute one partition. Hope this helps

@wayzonic 4 месяца назад

Node and executer both are different. Single node can build multiple executers base on node capacity

@helloanalyst 7 месяцев назад

Thanks for sharing valuable information,👍 जब भी कोई RDD के बेस पर नया RDD बनता है तो वो जो नया RDD बनेगा वो क्या same क्लस्टर मे बेनग़ा या फिर किसी दूसरे मे.

@wayzonic 4 месяца назад

On same cluster

@mayankkandpal1565 9 месяцев назад

nice explanation.

@riyakhandelwal8154 Год назад

In previous videos, it was mentioned by you that a DAG will be created only when it encounters any action(Job--> DAG). In this video you have mentioned a DAG will be created after every transformation. DAG created as an when it scans every line of code ? Or is it created when any action is encountered

@manish_kumar_1 Год назад

DAG is created when an action is encountered.

@08rajdeepsonawane94 7 месяцев назад

Nice explanation sir

@anshukumari6616 Год назад

Waiting for the next video !!

@Vinodkumar_19879 5 месяцев назад

Love you guru ji

@codetechguru1 7 месяцев назад

@manish_kumar_1 bahi can you explain it?. Q1. DataFrames in Spark are built on top of RDDs, but they use a more efficient execution engine called Tungsten. Q2. RDD don't have compile-time checking. In Vise versa Data frame have Compile-time and runtime checking.

@sabarnaghosh1658 Год назад

That means every single dataframe will eventually transformed into RDD right?

@manish_kumar_1 Год назад

Yes

@akumar2575. 4 месяца назад

done👍