Тёмный

transformation and action in spark 

MANISH KUMAR
Подписаться 19 тыс.
Просмотров 29 тыс.
50% 1

In this video I have talked about transformation and action in spark in great details. please follow video entirely and ask doubt in comment section below.
Directly connect with me on:- topmate.io/manish_kumar25
For more queries reach out to me on my below social media handle.
Follow me on LinkedIn:- / manish-kumar-373b86176
Follow Me On Instagram:- / competitive_gyan1
Follow me on Facebook:- / manish12340
My Second Channel -- / @competitivegyan1
Interview series Playlist:- • Interview Questions an...
My Gear:-
Rode Mic:-- amzn.to/3RekC7a
Boya M1 Mic-- amzn.to/3uW0nnn
Wireless Mic:-- amzn.to/3TqLRhE
Tripod1 -- amzn.to/4avjyF4
Tripod2:-- amzn.to/46Y3QPu
camera1:-- amzn.to/3GIQlsE
camera2:-- amzn.to/46X190P
Pentab (Medium size):-- amzn.to/3RgMszQ (Recommended)
Pentab (Small size):-- amzn.to/3RpmIS0
Mobile:-- amzn.to/47Y8oa4 ( Aapko ye bilkul nahi lena hai)
Laptop -- amzn.to/3Ns5Okj
Mouse+keyboard combo -- amzn.to/3Ro6GYl
21 inch Monitor-- amzn.to/3TvCE7E
27 inch Monitor-- amzn.to/47QzXlA
iPad Pencil:-- amzn.to/4aiJxiG
iPad 9th Generation:-- amzn.to/470I11X
Boom Arm/Swing Arm:-- amzn.to/48eH2we
My PC Components:-
intel i7 Processor:-- amzn.to/47Svdfe
G.Skill RAM:-- amzn.to/47VFffI
Samsung SSD:-- amzn.to/3uVSE8W
WD blue HDD:-- amzn.to/47Y91QY
RTX 3060Ti Graphic card:- amzn.to/3tdLDjn
Gigabyte Motherboard:-- amzn.to/3RFUTGl
O11 Dynamic Cabinet:-- amzn.to/4avkgSK
Liquid cooler:-- amzn.to/472S8mS
Antec Prizm FAN:-- amzn.to/48ey4Pj

Опубликовано:

 

13 апр 2023

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 194   
@Shivam-Tiwari11
@Shivam-Tiwari11 5 месяцев назад
Literally i am watching it at 3 am in the morning and my eyes are not blinking. That much interesting ur way of teaching is. I hope you will not stop creating the content because of demotivation of any type let’s say less number of subscribers ( i can feel it’s disheartening sometimes when u put that much efforts) . You just increased my crave for learning. Dil se shukriya bhaii ❤
@tusharkantighosh4641
@tusharkantighosh4641 2 месяца назад
same happens with me bro....these contents are diamonds....
@ARGTalks-gx6xs
@ARGTalks-gx6xs Месяц назад
Please continue this series brother, I just started watching this and found it to be fantastic resource. Keep up the good work.
@khurshidhasankhan4700
@khurshidhasankhan4700 8 месяцев назад
Interview k liye bahut benifit ho raha hai ❤
@vishwajeetkushwaha64
@vishwajeetkushwaha64 4 месяца назад
Most informative @ Thanks Bro
@ranidalvi1064
@ranidalvi1064 5 месяцев назад
Most informative and needful video. thank u so much.
@MrFirstScientist
@MrFirstScientist 10 месяцев назад
Appreciate your hard work.
@RaviKumar-xo5bz
@RaviKumar-xo5bz 4 месяца назад
sach mei aapko bahut ache se aata hai ... one place for all IT courses
@ujjalroy1442
@ujjalroy1442 Год назад
Phenomenal lectures loved it✌️👍
@pritiiBisht
@pritiiBisht 6 месяцев назад
Very Informative.
@pankajsolunke3714
@pankajsolunke3714 Год назад
This series is phenomenal.❤
@muizzrehan1433
@muizzrehan1433 Год назад
Amazing work. Thanks for lectures...👏
@omkarm7865
@omkarm7865 Год назад
Very very helpful content.. Please keep it up
@abhishekkumar-gupta
@abhishekkumar-gupta 9 месяцев назад
Very informative video. Must watch for someone beginner or intermediate in pyspark.
@udittiwari8420
@udittiwari8420 5 месяцев назад
very helpful series sir thank you for giving us your imp time ❣
@aasthajain4814
@aasthajain4814 7 месяцев назад
Appreciate it truly
@abegpatel5300
@abegpatel5300 9 месяцев назад
You are just awesome ❤
@prashantmehta2832
@prashantmehta2832 2 месяца назад
I am preparing for the interview, and I can say you are playing main part. I do make note and follow all steps. Thanks for the supper.
@alokmishra5367
@alokmishra5367 5 месяцев назад
No no sir .. I m honestly following your course.
@manvika
@manvika 11 месяцев назад
Your series is really good,. I started watching it today.. Please keep making it.. really helpful.
@rakeshjadhav_O
@rakeshjadhav_O 2 месяца назад
Great explanation Manish. Thank you very much for such informative videos
@jigsparikh7961
@jigsparikh7961 5 месяцев назад
You are awesome. I make it simple to understand. thanks
@vinayakjaipurkar7331
@vinayakjaipurkar7331 5 месяцев назад
Amazing explanation Sir 🤩
@Varunsharma-sg2nt
@Varunsharma-sg2nt Месяц назад
Sir, Awesome explanation Thank you so much
@shravanshenoy3873
@shravanshenoy3873 2 месяца назад
Fantastic explanation, understood the concepts very clearly. Thanks for making such amazing content.
@lifeisfun9
@lifeisfun9 9 месяцев назад
You are a gentle man seriously :)
@manojkaransingh5848
@manojkaransingh5848 Год назад
Outstanding tutorial....maja aa gaya sir..@manish
@pushkarratnaparkhi2205
@pushkarratnaparkhi2205 4 месяца назад
धन्यवाद मनीष भाई। 🙏
@sureshydv724
@sureshydv724 Год назад
You are outstanding. I love the way you are teaching. Thanks Manish bhai explaining along with flowchart👏
@dilipkuncha5728
@dilipkuncha5728 9 месяцев назад
hi , im learning well through your course . I literally realised after watching this video , about the csv file , because i had to literally make a new one using the data. Thanks for motivation and Please be Motivated !!😊
@samirdeshmukh9886
@samirdeshmukh9886 10 месяцев назад
Very helpful sir.. Thank you..
@ghvendrashra4912
@ghvendrashra4912 4 месяца назад
Bhai..you are the best teacher for data Engineer... concepts are very clear.....thanks
@MsSubhrajeet
@MsSubhrajeet 5 месяцев назад
Awesome 👏
@Azure-Mahesh
@Azure-Mahesh 7 месяцев назад
Your inputs are really helping in understanding complicated things easily, thank you
@user-ks7wl9pc2f
@user-ks7wl9pc2f 11 месяцев назад
Manish You are simply awesome. i'm glad that i have found best video's on RU-vid to learn spark.
@vaibhavmore7936
@vaibhavmore7936 Год назад
We got the file and used it as you taught. Didn't feel need to comment this also, after you are giving this much for free. Thanks for awesome content.
@deeksha6514
@deeksha6514 4 месяца назад
I do not have words to praise this masterpiece. Keep creating this awesome content.
@hubspotvalley580
@hubspotvalley580 11 месяцев назад
You are doing great job. Your lectures are awesome.
@praveenkumarrai101
@praveenkumarrai101 11 месяцев назад
your kadva sach is very very true
@t1mt0m97
@t1mt0m97 Год назад
Manish Ji....excellent hands-on course.. Keep adding and let your channel grow !
@bforbhakti1
@bforbhakti1 6 месяцев назад
Your teaching is really good and helpful.thanks alot
@mmohammedsadiq2483
@mmohammedsadiq2483 9 месяцев назад
I mean to say in my last comment, you have explained very well with simple examples
@user-qn6ud4hs3b
@user-qn6ud4hs3b 4 месяца назад
The way you explained everything made it so simple to understand. Earlier it used to be difficult for me to understand the architecture concept and I would forget that in sometime, but now I feel I wont forget it as this video felt like a story and everything was explained in detail and in very simple way. Its such a great content.
@ayeshaagrawal4987
@ayeshaagrawal4987 11 месяцев назад
Best teacher ever met , thank you sir
@SANJAYYADAV-hm2bs
@SANJAYYADAV-hm2bs 4 месяца назад
Sir you are doing really awesome job. Please making all your videos. As its will help to others in future as well. I am working as ETL tester and still i keep watching your videos to get some insight on data engineer roadmap.
@jbb6906
@jbb6906 10 месяцев назад
Awesome
@shivakrishna1743
@shivakrishna1743 Год назад
I want to tell you that I am following your videos and also implementing them. Whenever possible I saw links from your video and got the doc or data file. Once, I couldn't find data so I asked you :). Please don't loose motivation for doing these videos!!
@souravdas-kt7gg
@souravdas-kt7gg 4 месяца назад
content is very good
@SanjayKumar-rw2gj
@SanjayKumar-rw2gj Месяц назад
Wonderful explanation bro. Read couple of articles about Narrow and Wide transformation but could not grasp it completely but now after watching your video things got clear.
@automationwithwasi
@automationwithwasi Месяц назад
Manish bhaiya your videos are very valuable. I learn new things every time.
@electricalsir
@electricalsir Год назад
Thanks man 😊🎉
@rohit-ll3rj
@rohit-ll3rj 3 месяца назад
Really appreciate your efforts Manish! You teach really well. I work as a Data Engineer in one of the Big 4 and go through your videos whenever i need understanding of topics in depth.
@DpIndia
@DpIndia 10 месяцев назад
Nice Video, got lots of learning
@seethroughmyeyes423
@seethroughmyeyes423 Месяц назад
You make each and every topic very simple to understand. Thank you so much and keep up the work. 👍 If possible, please make some videos on Kafka, Kubernetes, Azure, Databricks. Your videos are really very helpful!
@adityaanand835
@adityaanand835 Год назад
thank you bhai!
@manish_kumar_1
@manish_kumar_1 Год назад
Directly connect with me on:- topmate.io/manish_kumar25
@sanooosai
@sanooosai 3 месяца назад
sir g dont loose motivations, we are fallowing with all dedication, sometime people with all the implementation knowledge also see your video for revision or to brush up basics, thank you for all the hard work and giving these great content in hindi
@ashishkumar9538
@ashishkumar9538 4 месяца назад
Thank you for sharing your knowledge among us and I am really enjoying learning the concepts. I am sure these informative videos are helping a lot of people. Keep on teaching us!!
@anshukumari6616
@anshukumari6616 Год назад
Thanks a lot !!
@apoorvkansal9266
@apoorvkansal9266 Месяц назад
Hello Sir, You are doing a great job and there are very few humble and selfless Data Engineers like you who could dedicate their valuable time for sharing their knowledge and experience. Please continue creating these videos on PySpark. I am preparing for Data Engineering Interviews for the the 3rd time i.e. for my 3rd switch of companies and interviews are getting scheduled and looking upto this series for cracking them. Thank you in advance for creating such informative and detailed videos. Jai Mata Di! 🙏
@shyammohan3611
@shyammohan3611 3 месяца назад
Sir I have got you link in the last video, sir i am following you both playlist practical and theory. please don't stop videos
@abhilovefood4102
@abhilovefood4102 Год назад
Sach bat hai sir
@hasnainakbar8621
@hasnainakbar8621 2 месяца назад
Love you brother
@amarnaik1819
@amarnaik1819 2 месяца назад
Sir your videos are very helpfull
@radheshyama448
@radheshyama448 11 месяцев назад
❤ thank you
@danishthev-log2264
@danishthev-log2264 11 месяцев назад
nice content sir
@lazycool3611
@lazycool3611 3 месяца назад
Ab tak bhaut sahi chl rha hai jo aap smjhaye 100 marks aayenge ab hume
@adityasaini8437
@adityasaini8437 8 месяцев назад
Manish bhai is real 💎!!
@sachinragde
@sachinragde Год назад
keep it up manish
@khurshidhasankhan4700
@khurshidhasankhan4700 8 месяцев назад
Sir aapke lecture se bahut sare doubt clear ho raha hai
@user-rh1hr5cc1r
@user-rh1hr5cc1r 3 месяца назад
well explained
@syeduroojahmed2863
@syeduroojahmed2863 3 месяца назад
Good initiative🙂
@saatoonworld
@saatoonworld Год назад
Good content
@abidsyman
@abidsyman 11 месяцев назад
good morning sir, ur lectures are great
@Finoboost
@Finoboost Год назад
Hi Manish Bhai, csv file was downloaded by the steps which you guided, you are doing great work, i am following and applying all the concepts which you taught.
@manish_kumar_1
@manish_kumar_1 Год назад
Good
@ankitas4019
@ankitas4019 3 месяца назад
Very nice
@karansinghrajpurohit3500
@karansinghrajpurohit3500 Год назад
Please continue this series
@manish_kumar_1
@manish_kumar_1 Год назад
Sure
@pschandekar
@pschandekar Год назад
Thanks Manish! You have explained and described each and every concept very deeply. It would be very helpful if you provide any referral resources and your notes if possible :)
@manish_kumar_1
@manish_kumar_1 Год назад
Notes not possible. Resources I will try to provide
@adityaabhinav4171
@adityaabhinav4171 9 месяцев назад
Most underrated Teacher on RU-vid I wish you all the best and one day you will be famous because the way you teach is awesome.
@manish_kumar_1
@manish_kumar_1 9 месяцев назад
Aap hame jaante hai itna hi kaafi hai😂😂
@akash4606
@akash4606 11 месяцев назад
Manish bhai bhot ache se pdha rhe ho.......lge rho....hm bhi apke sath lge hue h....Maine abhi start Kia pr complete zrur krunga
@VINEETANU1
@VINEETANU1 6 месяцев назад
Hi Manish, really good information, I am going though one by one, hope you have covered all the databricks topics in your videos and please provide us the PPT as well, thank you for your support. Vineet Goel
@HeenaKhan-lk3dg
@HeenaKhan-lk3dg 2 месяца назад
I am Following your course, Where you shared spark Documentations, I didn't see in Anywhere
@Icelander00
@Icelander00 10 месяцев назад
Manish I am implementing all side by side
@dattak-gb7ez
@dattak-gb7ez Месяц назад
Sir App Azure ke bhi kuch session loge to bahot acha hoga..kyuki apki padane ki technique bahot achi hai..
@hubspotvalley580
@hubspotvalley580 11 месяцев назад
Manish Bhai, I would like to request you, please make a real time end to end project. So, that it can help us during the interview.
@user-ln8hk2bx3k
@user-ln8hk2bx3k Месяц назад
great bhai bahut acha concept padha raheho Azure Data Brick ka v lekar aawo aap series
@engineerbaaniya4846
@engineerbaaniya4846 Год назад
Thanks
@mantukumar-qn9pv
@mantukumar-qn9pv Год назад
Sach me ab promise to myself karta hu ab practice karunga
@swarupsarangi734
@swarupsarangi734 2 месяца назад
you should make another video in which you address whether the spark.read methods are transformation or action.
@adilhusain6619
@adilhusain6619 Месяц назад
Hi Sir, sir dono container me data kis condition me enter hoga?
@Nikhil-jj7xf
@Nikhil-jj7xf 9 месяцев назад
Hello Sir its youre 6 lecture and in 5th Lecture you explained about architecture of Spark and not about corrupted records
@nehakamath8321
@nehakamath8321 8 месяцев назад
I have the same doubt. @manishkumar you mentioned in the starting of this video about corrupted records. Is there any video that is missed out between lecture 5 & lecture 6 of the series Spark Fundamental (theory). Please let us know.
@Nikhil-jj7xf
@Nikhil-jj7xf 8 месяцев назад
@@nehakamath8321 Hope he is following our comments so he will upload it..😄
@soumyaranjanrout2843
@soumyaranjanrout2843 7 месяцев назад
​@nehakamath8321He is talking about the Pyspark practical playlist video where he had made a video on handling corrupted records
@ChandanKumar-xj3md
@ChandanKumar-xj3md Год назад
Thanks Bhai JHAAD lagane k liye :). But really great input you provide through your Spark lecture. And please upload the tutorials as always waiting for the next lecture. Aur hm to data hi bana liye the aap k video se :)
@manish_kumar_1
@manish_kumar_1 Год назад
Nice
@prabhatsingh7391
@prabhatsingh7391 Год назад
Hi Manish ,1st thanks for such beautiful content. Can you please explain such questions "How to process 1 TB or any size of data in Apache Spark ?". If already available then just confirm.
@manish_kumar_1
@manish_kumar_1 Год назад
Already created one video on how to process 100gb of data daily
@alokmishra5367
@alokmishra5367 5 месяцев назад
Could you make videos regarding all the data engineering tools and technology.
@adilhusain6619
@adilhusain6619 Месяц назад
Kya yeh possible nhi h ki same ids ka data same container me pehle se hi move ho jae taki data suffeling na krni pade aur wide dependency ki zarurat hi na pade
@NY-fz7tw
@NY-fz7tw 5 месяцев назад
Sir, should i finish your theory lecture of spark and then follow practical videos for practice ?
@tnmyk_
@tnmyk_ 5 месяцев назад
Very well explained! I have a doubt though - What if for example - One partition can handle only 100 records of data but the original input data consists of 150 records all having the same ID. In that case how will the groupby() function work because it wont be possible to bring all data in to the same partition even after shuffling
@EVSprakash
@EVSprakash 4 месяца назад
What is the answer
@vivekjaiswal7316
@vivekjaiswal7316 3 месяца назад
Nice head of
@Rakesh-if2tx
@Rakesh-if2tx Год назад
Hi Manish Bhai, I couldn't find the csv file in the last video and thought you will upload it later. I watched the last video but was unable to practice due to some constraints and will seriously practice from now onwards. Apologies from my side. Thank you for the efforts you're putting into this spark series and Please continue this series. Want to connect with you for guidance. Kindly, let me know.
@manish_kumar_1
@manish_kumar_1 Год назад
Hi Rakesh, This was for all not just you. It's nice that you are paying attention. Keep practicing and rock it in interview 💪
@WorseTGameR
@WorseTGameR Год назад
@@manish_kumar_1 Bro please don't drop the series. I saw you didn't provide the dataset, so I downloaded a random one from kaggle and followed the steps.
@prabhatsingh7391
@prabhatsingh7391 Год назад
Hi In this example you have supposed 2 executor and as per HDFS default block size is 128 mb so there two partitions are there , but if we have supposed 3 executor and have 2 partitions then ,what we will do.
@manish_kumar_1
@manish_kumar_1 Год назад
1 executor will be sitting idle
@anirbanadhikary7997
@anirbanadhikary7997 Год назад
I can find similarity between your tutorial and the book Spark the Definitive Guide by Oriliey . Please correct me if I am wrong.
@manish_kumar_1
@manish_kumar_1 Год назад
Yes I do refer that book.
Далее
DAG and Lazy Evaluation in spark
20:35
Просмотров 23 тыс.
spark architecture | Lec-5
21:03
Просмотров 38 тыс.
Best KFC Homemade For My Son #cooking #shorts
00:58
Просмотров 16 млн
Learn Apache Spark in 10 Minutes | Step by Step Guide
10:47
RDDs: Transformation and Actions
16:11
Просмотров 16 тыс.
Spark Job, Stages, Tasks | Lec-11
24:16
Просмотров 28 тыс.
rdd in spark | Lec-9
24:54
Просмотров 21 тыс.
spark sql engine in spark | Lec-8
17:24
Просмотров 18 тыс.