Automated data profiling and quality scan via Dataplex 

Data quality is a critical concern within a complex data environment, particularly when dealing with a substantial volume of data distributed across multiple locations. To systematically identify and visualise potential issues, establish periodic scans, and notify the relevant teams at an organisational level on a significant scale, where should one begin?
This is precisely where the automated data profiling and data quality scanning capabilities of Dataplex on Google Cloud can prove invaluable. Requiring no infrastructure setup and offering a straightforward method for defining and implementing rules for data profiling and quality checks, it could serve as an excellent foundation for your large-scale data quality framework.
01:16 - Data Profiling vs Data Quality Scan
02:37 - Dataplex auto profiling
08:15 - Dataplex auto data quality scan
10:47 - Profiling hinted quality rules & YAML via CLI
18:36 - Other options to create scans
21:08 - Sensitive data considerations
22:02 - Summary
Slide: drive.google.com/file/d/13khs...
Repo: github.com/rocketechgroup/dat...

Наука