Learn Apache Spark in Microsoft Fabric in the 30 days of September.
Here's the playlist for this series if you want to catchup: • Learn Apache Spark in ...
Link to the GitHub for this series: github.com/LearnMicrosoftFabr...
Kaggle Dataset: www.kaggle.com/datasets/dines...
pySpark Machine Learning documentation:
spark.apache.org/docs/latest/...
Spark is the engine behind both the Data Engineering AND the Data Science experiences in Microsoft Fabric, so in September I'll be walking you through Apache Spark: what it is, why you should learn it, how to use it, and how it integrates into Microsoft Fabric.
No previous Spark knowledge is required, some basic Python would be useful!
#pyspark #microsoftfabric #apachespark
Here's the schedule:
1 Welcome - • DAY ONE - Learn Apache...
2 Why Spark? • Why you should learn S...
3 Components of Spark - • FIVE components of the...
4 Spark DataFrame - • Introducing the Spark ...
5 Read Files into DataFrame - • How to read CSV, JSON,...
6 Read/Write to Lakehouse Table - • How to write data to L...
7 Basic DataFrame Operations - • Learn THESE important ...
8 DataFrame Filtering - • 14 ways to filter your...
9 GroupBy and Aggregate Functions - • How to use GROUPBY and...
10 Handling missing values - • NO MORE NULLS - how to...
11 Joining DataFrames - • How to get better insi...
12 Time-series data - • Analyzing DATES and TI...
13 Spark SQL - • Using your SQL knowled...
14-16 Spark Machine Learning - • Spark Machine Learning...
20 Configuring Spark - • Customising Fabric Run...
21 Autotuning Spark Configuration - • Autotuning your Spark ...
22 Library Management - • Manage your Python pac...
23 High-concurrency mode - • Running multiple noteb...
24 Spark Scala - • Writing Spark Scala in...
25 Fabric MSSparkUtils - • Powerful notebook util...
26 Monitoring Spark - • Monitoring Spark (erro...
30 FINALE QnA - • How to use Spark in Mi...
Timeline
0:00 Coming up...
0:22 Intro to SparkML
2:08 Reviewing the documentation
4:28 Introducing the Kaggle dataset
5:44 Read data & Train/Test Split
7:47 Basic Concepts of SparkML
9:55 Feature Engineering in SparkML
15:55 SparkML Pipelines
16:30 Logistic Regression Model
17:26 Model testing and evaluation
19:45 Hyperparameter tuning and cross-validation
20.50 Future work: SynapseML
21:15 Future work: Saving models in Fabric
21:48 Wrapup
-BROWSE MY OTHER FABRIC PLAYLISTS-
DATA ENGINEERING • Data engineering (Micr...
END-TO-END FABRIC PROJECT • Playlist
INTRO TO MICROSOFT FABRIC • Intro to Microsoft Fabric
DATA FACTORY • Data Factory (Microsof...
-LINKEDIN-
Not following the LinkedIn page yet? Here's the link: / learnmicrosoftfabric
-ABOUT WILL-
Hi, I'm Will! I'm hugely passionate about data and using it to create a better world. I currently work as a Consultant, focusing on Data Strategy, Data Engineering and Business Intelligence (within the Microsoft/Azure/Fabric environment). I have previously worked as a Data Scientist. I started Learn Microsoft Fabric to share my learnings on how Microsoft Fabric works and help you build your career and build meaningful things in Fabric.
-SUBSCRIBE-
Not subscribed yet? You should! There are lots of new videos in the pipeline covering all aspects of Microsoft Fabric.
youtube.com/@learnmicrosoftfa...
1 авг 2024
