4 ms·

Datafusion is undoubtedly one of the best open source projects of all time, it's so incredibly powerful and well designed. The extensibility is insane, you can

Datafusion is undoubtedly one of the best open source projects of all time, it's so incredibly powerful and well designed. The extensibility is insane, you can create your own query language that compiles to logical plans.

ssinchenko 3mo ago
I agree 100%! DataFusion is beautiful and easy to extend in any direction. For the second version of my "out-of-core" graph algorithms project, for example, I implemented my own "co-partitioning" to speed up joins and achieved a performance improvement of two times! It was also easy to modify the physical plan and declare partitioning.
- stuhood 2mo ago
  Co-partitioning of range-partitioned data is now supported on `main` too: see https://github.com/apache/datafusion/issues/23183 https://github.com/apache/datafusion/issues/23183 and the epic https://github.com/apache/datafusion/issues/22395 https://github.com/apache/datafusion/issues/22395