Data Council is the "No BS" data conference. Since 2013 we've been bringing together the brightest minds in data to share insider industry knowledge, technical architectures and best practices on building the cutting edge data processing systems and tools of the future.
We are deeply technical, vendor neutral & community-driven, and we exhibit our values each year during our flagship global event in Austin, TX. Across 3 days, join top data scientists, lead engineers, CTOs, founders, AI researchers, Heads of Data, executives, investors and community organizers who are all coming together IRL to share valuable insights as they build the future of data together.
We also operate Zero Prime Ventures - a first check VC fund for Day 0 engineer-founders.
2nd Legendary talk, I can't remember how many years it's been since I last actually watched a tech video at 1x speed, and had my attention completely captured / enjoyed it, this was fascinating. This guy is in the Venn diagram of smart person, who knows how to properly present/communicate, and was willing to do the prep work. VS many other smart people suck at communication/presentation or aren't willing to do the prep work.
all this jank just to solve the issue which is basically Python. Just write a fully statically compiled binary and shove that on a NFS, then just use rsync between dev machines and NFS. Have a shell script watch binary file changes and relaunch when file is changed. Look ma, I just replaced entire solid with a few bash scripts 😂
Very cool talk! The idea of learning hueristics was very cool! I didn't quite understand how the criterion for splitting down multiple paths! I will check out the source code! Thank you for hosting this talk!
This was an extremely informative talk - especially the section on challenges - and one I wish would receive more attention due to how useful it is as an overview to quite a few complex and highly relevant issues. It would be nice if it were re-elaborated and presented in a non-live presentation format.
Great, very much needed and promising project ! However, it is not quiet clear what do you mean when you are talking about data versioning (DV) - do you version the data as LakeFS does or you are just versioning the source code which is producing this data. Also the diagrams in the presentation (Virtual/Physical layers) I find confusing and not easy to grasp at first glance. It will be nice in the next iteration if you use some real world/practical entities to describe demo objects like customer, product, sales etc. instead of just “source” and wrap the demo in some quick story like “Meet Alex, the data engineer at TechCorp, a rapidly growing tech company. Alex is responsible for managing the company’s data pipelines, ensuring that data from various sources is clean, consistent, and available for analysis” etc. you got the idea. Finally I would suggest you switch the sequence and the time you spend on the theory and the demo part - show your fantastic open source project demo first and how easy is implementing the 3 concepts in meaningful story then after each segment just mention the theoretical part, but don’t allow the theory to consume 75% of your presentation unless you want to be considered as one of the many Data Governance “gurus” which are presenting on this channel. Whishing you all good luck with this fantastic project !
If someone can explain to me how you’re supposed to do a major version DB upgrade with a Debezium connector. It’s such an unbelievable pain that it’s a total dealbreaker. Unless I’m missing something
I loved this and wish there was more of it. Thank you! But as noted: 'invoice reconciliation is boring'. I feel like the survival of our species will pivot not on our curiosity, but on our capacity to constrain our desire for novelty enough to solve boring problems.
Pointer vs. Value discussion: Based on the Method vs. Function discussion, ADT should be strictly adhered to. Operations that modify the ADT are modeled as functions that take the old state as an argument and return the new state as part of the result. In other words, a function should enforce immutability. The ADT approach helps with concurrency, making the code cleaner and easier to read. As an API user, I shouldn't worry about the state changing when I pass a structure. Of course, the pure ADT model's problem is memory consumption. That's why ADT models are generally implemented in VMs that can routinely find old structures without references and remove them from memory.
The method vs. function debate is absurd. The presenter needs to learn or spend time with OO programming. Class methods don't have to be logically connected to states. I developed in C during the 80s. The problem with structs is that the data is the point of coupling. The class hides data. In OO, the focus is on behavior and not the state. The OO state can be anywhere and can change. The strategy allows the implementation of the module to be changed without disturbing the client programs.
Really good and informative. I congratulate PeerDB for their recent seed round secured . I see there is a lot of potential in PeerDB where organisations are looking to stream their data to warehouse. I have had a very unique need , I wish PeerDB was a wonderful choice back then.
