3 ms·
Generally speaking your data-science stuff wouldn't fit into the serverless bucket anyway though. Also generally Python is just used for terminal processing in
by jpgvm 3y ago
Generally speaking your data-science stuff wouldn't fit into the serverless bucket anyway though.
Also generally Python is just used for terminal processing in the Big Data world, all the pieces in between are JVM anyway.
- disgruntledphd2 3y agoYeah, I mean if you're already a JVM shop then you could probably run Spark yourself (which can be painful) and use SparkSQL to dump to Python (maybe on databricks, though I'm not a massive fan) and do your fancy ML stuff there. Alternatively, you could just use a read-only PG replica and Python here, which would suffice for most DS needs (unless your core product is DS related or you have huuuuuggggggggeeee data (which you probably don't)).
- jpgvm 3y agoYeah that is fair too. Most people have tiny data. Then there is places I have worked... where 10TB qualified as small data that you "just query with Presto" and where more exotic things were needed for actually interacting with the "big data". All relative in the end.
- disgruntledphd2 3y agoYeah, snowflake is super nice for analytics, but PG will get you a long way with less complexity (given that you're already using it for OLTP).
- sanderjd 3y agoThis doesn't mesh with my experience of the current breakdown in usage between python and the jvm for these use cases.