sammysidhu

5 ms·

1. 1.▲
   Finding a Needle in the Haystack: Querying Physical AI Data with Daft(eventual.ai)

   2 points by sammysidhu 3mo ago|0 comments
2. 2.▲
   Scaling As-Of Joins(daft.ai)

   1 points by sammysidhu 5mo ago|0 comments
3. 3.▲
   by sammysidhu 6mo ago

   Amazing work leveraging Daft for this!
4. 4.▲
   by sammysidhu 11mo ago

   Part of the Daft team here! Happy to answer any questions
5. 5.▲
   DeepSeek smallpond, 3FS and data processing for AI(blog.getdaft.io)

   10 points by sammysidhu 2y ago|0 comments
6. 6.▲
   by sammysidhu 2y ago

   Hi, One of the authors of Daft here! Happy to answer any questions
7. 7.▲
   by sammysidhu 2y ago

   Congrats on the launch! It's been great working with you from the Daft side
8. 8.▲
   Eventual (YC W22) Is Hiring a Developer Relations Manager for Daft (SF)(jobs.ashbyhq.com)

   1 points by sammysidhu 2y ago
9. 9.▲
   by sammysidhu 2y ago

   https://github.com/Eventual-Inc/Daft Is also great at these types of workloads since it’s both distributed and vectorized!
10. 10.▲
    by sammysidhu 2y ago

    We're actually using pyiceberg to retrieve metadata! All our IO and decoding happens in the rust side once the data has been passthrough. We expose something called a ScanOperator which allows integration into various catalogs through
11. 11.▲
    Eventual (YC W22) Is Hiring Software Engineers to Build a Query Engine in Rust(jobs.ashbyhq.com)

    1 points by sammysidhu 2y ago
12. 12.▲
    Adversarial file reading: from 10k small CSVs to Parquet files(blog.getdaft.io)

    7 points by sammysidhu 3y ago|1 comments
13. 13.▲
    by sammysidhu 3y ago

    Eventual | Engineers; Rust | SF | https://www.ycombinator.com/companies/eventual/jobs We're the people behind Daft (On the front page today!), a distributed dataframe built in Rust. We're VC backed and f
14. 14.▲
    by sammysidhu 3y ago

    we love rust too!
15. 15.▲
    by sammysidhu 3y ago

    This would be more comparable to being a backend for ibis. We're working on adding the remaining operations (like regex on strings or trigonometry functions) that ibis requires!
16. 16.▲
    by sammysidhu 3y ago

    horizontal scaling however provides you with more aggregate network bandwidth. Most enterprises run workloads that downloads data from a data lake (usually S3), which is usually the bottleneck. Having horizontal scaling here allows the quer
17. 17.▲
    by sammysidhu 3y ago

    One of the daft developers here - Thank you!
18. 18.▲
    Daft 0.2: 10x faster IO from S3(blog.getdaft.io)

    3 points by sammysidhu 3y ago|0 comments
19. 19.▲
    by sammysidhu 3y ago

    Love the data driven approach to massive problem
20. 20.▲
    by sammysidhu 3y ago

    (one of the Daft maintainers here) Great call out! I went ahead and make an issue for us to work on this: https://github.com/Eventual-Inc/Daft/issues/1016
21. 21.▲
    by sammysidhu 3y ago

    Hi! (one of the Daft maintainers here), thanks for the feedback. Ultimately you're right that supporting the full Polars syntax in a distributed fashion is very difficult. There are libraries out there that do "Pandas but distribu
22. 22.▲
    by sammysidhu 3y ago

    Hi (one of the maintainers here), that is a good suggestion! I wasn't aware of that project. I went ahead and made an issue to add `export DO_NOT_TRACK=1` as one of the variables we track! https://github.com/Eventual-In
23. 23.▲
    by sammysidhu 3y ago

    Hi Sammy here, one of the Daft maintainers. Happy to answer any questions that you all might have!