Тёмный
The Data Channel
The Data Channel
The Data Channel
Подписаться
"The Data Channel”, your go-to destination for unraveling the mysteries of the ever-expanding data universe. In an era where information reigns supreme, understanding the intricacies of data and its related technologies is not just a skill but a necessity. Whether you’re a seasoned data enthusiast, a budding analyst, or someone simply curious about the transformative power of information, this blog aims to be your compass in navigating the dynamic world of data.

Join us at “The Datapedia” as we navigate the exciting intersections of data engineering, data science, and the ever-evolving landscape of cloud platforms. From foundational knowledge to advanced techniques, our mission is to make the complexities of these technologies accessible to all. Embark on this data odyssey with us and discover the limitless possibilities within the data spectrum.
Комментарии
@pavanicowdary8395
@pavanicowdary8395 12 часов назад
I didnt understand the free taril and i am creating databrick acc using microsoft azure but it ask something
@oumaymabenyahya6066
@oumaymabenyahya6066 День назад
Hii, i sent u an email for the dbc file !
@nanivo9804
@nanivo9804 2 дня назад
what's the difference b/w both of this?
@SRINIVASAREDDY-h2f
@SRINIVASAREDDY-h2f 4 дня назад
Seems, Topic 5 is missing..
@SRINIVASAREDDY-h2f
@SRINIVASAREDDY-h2f 6 дней назад
Great Job!! Sent an email for practice notes.
@thedatachannel878
@thedatachannel878 5 дней назад
Sent to your mail ID Thank you
@SRINIVASAREDDY-h2f
@SRINIVASAREDDY-h2f 6 дней назад
Very useful videos, Thanks for knowledge share.
@thedatachannel878
@thedatachannel878 5 дней назад
Thank you for your support Happy learning 😇
@mohammedak-m8k
@mohammedak-m8k 7 дней назад
hi day 2 for requesting all files related to this playlist and even i have mailed you .please help
@thedatachannel878
@thedatachannel878 6 дней назад
Apologies for delay, I have shared now, please check your mail
@neuera9556
@neuera9556 8 дней назад
Please send spark associate developer playlist dbf files i have send u email request still not response
@thedatachannel878
@thedatachannel878 6 дней назад
You mail id?
@mohammedak-m8k
@mohammedak-m8k 8 дней назад
please upload video 5
@mohammedak-m8k
@mohammedak-m8k 8 дней назад
hi thanks for the course i have emailed to requesting for dbc file thanks
@thedatachannel878
@thedatachannel878 6 дней назад
Sent to your mail, please check, Thank you
@vivekdutta7131
@vivekdutta7131 9 дней назад
Sir! Where is the autoloader video. I cannot find it in this playlist! Please upload it.Thanks!🙂
@neuera9556
@neuera9556 11 дней назад
Hi hope you’re doing great 👍 ! After learning basics n understanding ( types of data, 6vs of data, etl vs elt, db vs dw vs dm vs dl, full vs incremental load, fact vs dimensions, dw schemas, scd type n storage layer types, oltp vs olap) in one day , i have moved to databricks instead of data analysis as i want to become data engineer so today i have started this playlist n done with below topics ( 1- data bricks -- simple open source and single seloid teck stack ( all stack under 1 unberla ) lakehouse Data lake + dw = lakehouse (delta lake ) as delta lake is heart of lake house n lakehouse is build on top of delta lake (Topic-2 free trail - 2 ways aws n direct db website register ( need credit card 14 days trail n set termination 120 or 15 minutes as if u unchecked it will consume storage ) 1- trail version single node ( default driver node ) Multi node has worker + driver Topic-3 ( db architecture n services) ( control plane n data plane Control plane :: db cloud admin control Data plane : customer cloud ( aws azure gcp ) all exution transformation compute memory storage physically present here ) High level cluster : 1- general purpose interactive n job cluster Topic-3 db workspace Db page overview ( left side personas ( ml de n sql will b available in premium version) rest create workspace repos compute data workflow will b same in all 3 pasonas ( ds ml sql ) -- right side top ( user settings , admin consle , logout many more ) Topic -4 create n manage clusters Workspace create or compute create new ( single node -) When u click on created cluster u will have many options logs alerts metrics many more where we can mintor all Topic-5 db notebooks 1- create new notebook name n select lang u want 2- execute cell print (“hello”) 3- magic command %sql %python to change language of note book 4- md 5- %run “path” of child notebook or second notebook 6- %run “./notbookname” for exiting folder 7- dbutilies.help 8- display dbutlies.fs.help to display in tabular format Topic-6 git version n repos Create account in git hub settings developer mode acess token create copy then go to db settings add token repos n configure then create a child branch n to avoid conflict ( first always pull latest version n then push) Topic -7 taking brake may b evening i will practice Thanks 😊 once again !!
@thedatachannel878
@thedatachannel878 11 дней назад
Amazed by your dedication, it is really inspiring. Thank you for your learning journey updates
@neuera9556
@neuera9556 12 дней назад
Learn in one day all below topics n return what i learn thanks now starting this playlist Topic -1 Types of data 1- structured ( oltp transactional online banking booking erp ) relation between tables Eg : oracle sql server ibm db2 vertica 2-semi structured- json xml format Dw / data bricks / delta or data lake ( iot sensor data ) 3- unstructured ( images videos ) ( dw / delta lake ) N for statistics data ( Topic -2 6 vs of data 1- volume (gb tb peta bytes) 2- variety (semi structured unstructured) 3- velocity-( 4- value ( analysis dashboard) 5- veracity ( redundancy scalability) 6- variability( change in variable) Topic -3 etl vs elt Etl : extract transform load- specific use case Elt : extract load n transform- many use case Topic -4 Db - rdbms oltp ( relationship primary n foreign key) Dw - storage ( dw / db) fact n dimensional ) Dm - subset of dw for fast recent revision history Dl- large storage Topic -5 full vs incremental load Full: replace full load in target system ( useful when less volume n frequency is often Incremental load : first time full load n from next load it only takes updates or modified record( volume is high n modifications) Topic-6 fact vs dimensions Fact: Measures Transitional Dimensions: Not change statics Dim tables Topic-7 scd Scd -0 to 6(5) Scd -0 fixed dimensions Scd-1 no history Scd -2 Scd -3 Scd -4 Scd -6(5) Topic -8 dw schemas 1- star - straight forward centralised fact n many dim tables but dim tables will not have relation with other dim tables 2- snow flake - Complex query performance low One fact table have many dim tables n each dim tables have relation b/w them know as child tables 3- galaxy star schema
@thedatachannel878
@thedatachannel878 12 дней назад
Appreciate your efforts on compiling all this topics. Which would eventually help many learner’s Thank you, happy Learning
@neuera9556
@neuera9556 12 дней назад
@@thedatachannel878 thanks for making’s such clear content video I request you to make video on real time projects for azure data engineering with step by step notes 📝 n data sets which can easily downloaded n use for practice thanks 😊
@thedatachannel878
@thedatachannel878 12 дней назад
@neuera9556 Noted Thank you for your genuine feedback
@neuera9556
@neuera9556 13 дней назад
I saw thinking due to storage outage
@shubhambhardwaj2119
@shubhambhardwaj2119 16 дней назад
is this sufficient to pass databricks certification and also pls share practice notebook hv shared a mail
@thedatachannel878
@thedatachannel878 14 дней назад
We covered almost all the topics as part of certification. But to attend to certification exam, suggest you to please take exam dumps and understand the pattern of questions
@shubhambhardwaj2119
@shubhambhardwaj2119 16 дней назад
Thanks for the tutorial ...Have mailed for the notebooks and course content. Please share.
@thedatachannel878
@thedatachannel878 14 дней назад
Have shared details of practice notebooks in mail. Thank you
@ChrisC-k7y
@ChrisC-k7y 17 дней назад
Thank you for making the course. Can I ask for the notebooks and data?
@thedatachannel878
@thedatachannel878 17 дней назад
Hi thank you responding. Request you to please send mail request to yt.the.data.channel@gmail.com and we will send required materials for practice
@JeevanKumar-z5k
@JeevanKumar-z5k 21 день назад
Request mail for the .dbc file to practice was sent to your mail, could you please send that file for my mail. Thanks
@thedatachannel878
@thedatachannel878 14 дней назад
Hi, have shared the practice notebook details to your mail id. Thank you
@neuera9556
@neuera9556 24 дня назад
Hi - i wana become data engineer can u suggest ur playlist in alphabetical order
@neuera9556
@neuera9556 24 дня назад
Hi i want to learn data engineering n get job as experience data engineer can you suggest playlist in alphabetical order for de and ru data engineering if yes can i know ur LinkedIn? Thanks 😊
@thedatachannel878
@thedatachannel878 23 дня назад
thedatapedia.com/2024/01/04/azure-data-engineering-comprehensive-learning-guide/
@thedatachannel878
@thedatachannel878 23 дня назад
Refer to this link, where there is complete curated list in order and plan to learn. Hope it helps
@manaskumar9048
@manaskumar9048 26 дней назад
Content is good, but the background music is really annoying. Please don't use it in your lectures.
@thedatachannel878
@thedatachannel878 25 дней назад
Thank you for you valuable feedback . We have already considered and all our latest videos are without background noise Happy learning
@abdoukhadrediop804
@abdoukhadrediop804 26 дней назад
Hello i send you a mail for content notebook
@thedatachannel878
@thedatachannel878 12 дней назад
Hi, have replied you with required practice materials Thank you
@abdoukhadrediop804
@abdoukhadrediop804 27 дней назад
Hello, thanks for tutorial ! i send you a mail for notebook.
@thedatachannel878
@thedatachannel878 14 дней назад
Hi, have shared the practice notebook details to your mail id. Thank you
@MKBH-f7p
@MKBH-f7p 28 дней назад
Topic 5 missing
@thedatachannel878
@thedatachannel878 27 дней назад
Apologies, will try to add that shortly
@neuera9556
@neuera9556 Месяц назад
Is this end of the series ?
@thedatachannel878
@thedatachannel878 28 дней назад
Yes this end of playlist, however if you have any suggestions for any particular topics please let us know
@ARATHI2000
@ARATHI2000 Месяц назад
Good videos. Thank you. Could you pls send the data files etc.? Sent you an email earlier. Thx!
@thedatachannel878
@thedatachannel878 Месяц назад
We have sent you the required dbc file for practice, please check your mail Happy Learning, Thank you
@ZeeShanytt
@ZeeShanytt Месяц назад
Best video to understand the concept. Thank you The Data Channel
@thedatachannel878
@thedatachannel878 Месяц назад
Thank you for your appreciation. Keep supporting Happy Learning 👍
@sarangdoliya8322
@sarangdoliya8322 Месяц назад
Thanks for tutorial ... I have send mail for Notebooks
@thedatachannel878
@thedatachannel878 Месяц назад
Have replied to your mail with required practice notebooks, please check….
@AbhishekKumar036
@AbhishekKumar036 Месяц назад
Thanks Buddy🙏🏻, Can you tell me how can i integrate github with amazon redshift for version control?
@thedatachannel878
@thedatachannel878 Месяц назад
Hi Abhishek, I don’t have AWS subscription to demo right now. But this is Noted, will try to add this content in near future
@NasimKhan-vu8oi
@NasimKhan-vu8oi Месяц назад
The background music is disturbing
@thedatachannel878
@thedatachannel878 Месяц назад
Thank you for feedback, we are already considering this and all our new video are without background music Happy Learning 👍
@andersoncardenas8777
@andersoncardenas8777 Месяц назад
Nice content, looking for the material, i already have sent the email
@thedatachannel878
@thedatachannel878 Месяц назад
Sent to your mail. Happy Learning. Thank you
@rashmips-ht7iw
@rashmips-ht7iw Месяц назад
Hi Can you please share the notebook and course details?
@thedatachannel878
@thedatachannel878 Месяц назад
Please mail me to yt.the.data.channel@gmail.com will share notebooks
@vasutke1187
@vasutke1187 Месяц назад
High clarity, good presentation. Very use full. Thank you Sir
@thedatachannel878
@thedatachannel878 Месяц назад
Thank you Vasu Happy Learning 👍
@sumanthkumaravula7058
@sumanthkumaravula7058 Месяц назад
Good Content! Can you please share the .dbc file for practice?
@thedatachannel878
@thedatachannel878 Месяц назад
Thank you. Please send mail request to yt.the.data.channel@gmail.com
@sumanthkumaravula7058
@sumanthkumaravula7058 Месяц назад
Already email your team.😊
@thedatachannel878
@thedatachannel878 Месяц назад
@sumanthkumaravula7058 sent now please check
@ngobamisteve900
@ngobamisteve900 Месяц назад
Thank you for this helpful tutorial : Good job
@thedatachannel878
@thedatachannel878 Месяц назад
Thank you for supporting Happy Learning 👍
@zerogaming4452
@zerogaming4452 2 месяца назад
Have sent the mail for Notebook and Data Files.
@thedatachannel878
@thedatachannel878 2 месяца назад
Sent to your mail id. Please Subscribe and share, Happy learning
@zerogaming4452
@zerogaming4452 2 месяца назад
Q: Client Mode and Local mode appears to be same. Could you please share a scenario where there is a difference?
@abdoukhadrediop804
@abdoukhadrediop804 28 дней назад
In client mode de spark session is registered into you machine so every result come back to your machine ! So il client mode if your machine fail the computing is done! For debuging is coll to see what contaning the spark session but is not necessy in oder cases ! note in cluster mode spark session is contained by driver program!
@neuera9556
@neuera9556 2 месяца назад
Topic -5 missing
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you bringing it up. Will update the content
@neuera9556
@neuera9556 2 месяца назад
@@thedatachannel878 thanks eagerly waiting for
@saurabh7629
@saurabh7629 2 месяца назад
Tutorial is helpful, requesting you to share the notebooks, Please share.
@thedatachannel878
@thedatachannel878 2 месяца назад
Can you pls drop a mail to yt.the.data.channel@gmail.com . Will share the required notebooks
@ArunkumarAravindhakshan
@ArunkumarAravindhakshan 2 месяца назад
if in this video if you avoid background music means that will be very help full to understand clearly
@thedatachannel878
@thedatachannel878 2 месяца назад
Thanks for sharing feedback, perhaps we have considered this in future video to remove background music
@raghupatil8809
@raghupatil8809 2 месяца назад
Really useful Python course, keep it up bro...
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you. Happy Learning
@madhur2045
@madhur2045 2 месяца назад
Thank you very nice videos sir
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you, happy learning
@higiniofuentes2551
@higiniofuentes2551 2 месяца назад
Thank you for this very useful video!
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you, Please Subscribe . Happy Learning 👍
@kernelab
@kernelab 2 месяца назад
greate video destroyed by pointless distracting music.
@thedatachannel878
@thedatachannel878 2 месяца назад
We acknowledge the issue and unfortunately there is no way to remove background noise. But we have made sure to remove noise in all our upcoming videos
@jenyroy8203
@jenyroy8203 2 месяца назад
Hello sir, By saying the default value of vacuum 30 days, do u mean that all the table data has only maximum 30 days life by default?
@thedatachannel878
@thedatachannel878 2 месяца назад
Yes that is correct
@jenyroy8203
@jenyroy8203 2 месяца назад
@@thedatachannel878 so in real world industry projects how is it going to work where we may need out tables for a longer time
@thedatachannel878
@thedatachannel878 2 месяца назад
@jenyroy8203 In real world project, if there is need of retention period more than 30 days they you can persist data in Blob storage which is cheaper solution or may be in adls etc
@marcobernardo1604
@marcobernardo1604 2 месяца назад
Pretty handy and useful
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you Happy Learning
@susanODilla
@susanODilla 2 месяца назад
Hi Data Channel, thank you for the tutorial! I'm interested in setting limits in Fabric. Can operations that exceed a cost limit be automatically shut down? For example, if a company accidentally leaves a process running over the weekend, resulting in charges of over 30,000 Euros, would implementing a kill-cost limit of, let's say, 1,000 Euros have prevented this? I'd appreciate any insights you can provide. Thank you!
@thedatachannel878
@thedatachannel878 2 месяца назад
Absolutely, that’s great questions. Currently I don’t see any supporting MS documentation to support setting limits. However, I am definitely sure that there will enterprise level and even subscription level limit can be set. Need to explore and if I come across supporting documentation will post it here Thank you. Happy Learning
@susanODilla
@susanODilla 2 месяца назад
@@thedatachannel878 Thanks for the reply. Unfortunately there is not much about this yet!
@thedatachannel878
@thedatachannel878 2 месяца назад
Yes, but will wait for updates to come from MS
@alanpaul7505
@alanpaul7505 2 месяца назад
This is master piece course and expecting really interesting topics coming out.,,,!
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you. Happy Learning..
@raghupatil8809
@raghupatil8809 2 месяца назад
Interesting and need of the hour, Kudos for you effort and knowledge
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you. Happy Learning..
@Madhumathi-gk3ew
@Madhumathi-gk3ew 2 месяца назад
Thank you for always brining such amazing quality content
@thedatachannel878
@thedatachannel878 2 месяца назад
Thank you. Happy Learning..