Polars: Blazingly Fast DataFrames in Rust and Python

Подписаться 112 тыс.

Просмотров 28 тыс.

50% 1

This talk will introduce Polars a blazingly fast DataFrame library written in Rust on top of Apache Arrow. Its a DataFrame library that brings exploratory data analysis closer to the lessons learned in database research.
CPU's today's come with many cores and with their superscalar designs and SIMD registers allow for even more parallelism. Polars is written from the ground up to fully utilize the CPU's of this generation.
Besides blazingly fast algorithms, cache efficient memory layout and multi-threading, it consist of a lazy query engine, allowing Polars to do several optimizations that may improve query time and memory usage.
Read more:
github.com/pola-rs/polars www.ritchievink.com/blog/2021...
Join the talk to learn more.
Connect with us:
Website: databricks.com
Facebook: / databricksinc
Twitter: / databricks
LinkedIn: / data. .
Instagram: / databricksinc

Наука

Опубликовано:

18 июл 2022

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 28

@JeremyChone 2 года назад

Great video. I have been playing with Polars, Python, and Rust, and it is a fantastic technology. Also, I think Polars' Python facade with Rust backend / clean API is a great model for other data-centric technologies that need performance and scripting facade interfaces. Big thanks to the Polars team for this gem.

@user-qm6up7kz4n 2 года назад

Polars is just great! After trying it I won't go back to pandas.

@incremental_failure Год назад

Haven't been so excited about data analysis for a while. Now comes the chore of converting pandas to polars in a 15k line project.

@ControlTheGuh Год назад

Learn rust, python and rust synergies will be all over the place in data science soon. Great work with Polars

@jmnunezd1231 Год назад

Why do you think that? Can you elaborate more please? Thanks!

@ptarslatinum2806 Год назад

Can you please provide some insight? I am thinking of learning rust and I wish to enter this field.

@saaddahmani1870 2 года назад

Really great crate and great video, thanks.

@swalsh1123 Год назад

I'm going to try this out. Thanks

@Josh-wb7ii Год назад

For a lot of my job its not possible to use eagerly evaluated DFs, polars is a fantastic option.

@josedejesuslopezdiaz Год назад

great API

@headbangingidiot Год назад

This is great, can we use polars on a databricks cluster?

@rakeshjamdare2049 Год назад

Sure, it terms of python, its just another library so can use it very conveniently!!!

@milo4433 2 года назад

I wonder if this could ever be used with multiple machines similar to spark. Polars looks very fast and with the addition of distributed compute it would be very powerful. Perhaps it was not designed to allow that though.

@user-qm6up7kz4n 2 года назад

Polars uses all logical cores of your machine in parallel. Doesn't that mean distributed compute?

@milo4433 2 года назад

@@user-qm6up7kz4n I am asking specifically about cross machine distribution. For example if we had multiple 8-core machines (like VMS in aws) available, can we distribute polars work across those vms instead of being limited to a single machine and its cores.

@user-qm6up7kz4n 2 года назад

@@milo4433 aah I see the difference now