The Databricks Change Feed enables CDC, or Change Data Capture, in the spark environment - this is pretty huge. Keeping track of changed records can be a hugely inefficient exercise, comparing masses of records to determine which ones have been changed by upstream events. With the delta table change feed, we can keep an efficient track of exactly which records have changed!
In this video, Simon looks at the initial release, looking at how the change feed works under the covers, what happens in the transaction log and how you get started with CDC in your Lake!
For more details on the new change feed functionality - check out the docs here: docs.microsoft...
As always - don't forget to hit those like & subscribe buttons, and get in touch if we can help with your advanced analytics objectives!
22 авг 2024
