Understanding RisingWave Streaming Database Internals
In this episode of the Geek Narrator podcast, host Kaivalya Apte discusses the technical architecture and usage of the RisingWave streaming database with Yingjun, the founder and CEO of RisingWave Labs. Touching on everything from ingestion techniques to storage layers and optimization measures, this conversation provides a detailed overview of the RisingWave platform and its capabilities. It's an insightful and technical deep dive into the intricacies of creating and managing a streaming database in the modern data landscape.
Introduction
About RisingWave and Yingjun
Understanding Streaming Databases
RisingWave in E-commerce applications
How RisingWave works
Cache Management in RisingWave
Postgres Compatibility and Multi-cloud Support
Challenges in shifting from Batch to Streaming
Conclusion
Chapters
00:00 Introduction and Guest Background
01:47 Understanding Streaming Databases
02:54 RisingWave: A Streaming Database
03:22 Use Cases for Streaming Databases
04:00 RisingWave in the Context of Existing Architectures
20:30 RisingWave's Cloud-Native and Rust-Based Architecture
25:34 Data Ingestion and Storage in RisingWave
29:05 Fault Tolerance in RisingWave
31:02 Understanding S3 and Item Potency
31:53 Dealing with Duplicate Entries and Network Issues
32:33 Data Format and Storage in S3
33:05 The Role of Streaming Ingestion and Random Access
33:37 Joining Different Streams and Data Access
34:24 The Transition from Row Store to Column Store
36:43 Understanding Predefined and Ad hoc Queries
37:25 The Importance of Materialized Views
40:25 The Role of Caching in Query Processing
40:25 Understanding the Stateless Behavior of Compute Nodes
45:27 The Role of Preloading in Query Processing
47:36 The Importance of Postgres Compatibility
51:23 Future Plans for Supporting Multiple Cloud Offerings
52:48 Understanding the Challenges of Moving from Batch to Streaming
53:14 The Importance of Real-Time Data Processing
58:57 Conclusion and Future Collaborations
References:
RisingWave blog: risingwave.com/blog/building-...
risingwave.com/
===============================================================================
For discount on the below courses:
Appsync: appsyncmasterclass.com/?affil...
Testing serverless: testserverlessapps.com/?affil...
Production-Ready Serverless: productionreadyserverless.com...
Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount.
===============================================================================
Follow me on Linkedin and Twitter: / kaivalyaapte and / thegeeknarrator
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: • Write-ahead-logging
Popular playlists:
Realtime streaming systems: • Realtime Streaming Sys...
Software Engineering: • Software Engineering
Distributed systems and databases: • Distributed Systems an...
Modern databases: • Modern Databases
Stay Curios! Keep Learning!
28 июн 2024
