4 ms·
I think this will be a hit with the big name audit companies. I know some use databricks for pyspark on the M&A side. As deals move forward and they get more da
by film42 2y ago
I think this will be a hit with the big name audit companies. I know some use databricks for pyspark on the M&A side. As deals move forward and they get more data, they have to scale up their instances which isn't cheap. If polars enables serverless compute where you pay by the job, that could be a big win.
And sure, databricks has an idle shutdown feature, but suppose it takes ~6 hours to process the deal report, and only the first hour needs the scaled up power to compute one table, and the rest of the jobs only need 1/10th the mem and cores. Polars could save these firms a lot of money.
- serced 2y agoMay I ask what part in M&A needs this much data processing? I am quite familiar with the field but did not yet see such tasks.
- lmeyerov 2y agoI thought databricks has serverless recently already? Or is it by the notebook, while this is by the job?
- Centigonal 2y agoDatabricks supports serverless for both interactive notebooks and jobs.