3 ms·
Yeah, I mean if you're already a JVM shop then you could probably run Spark yourself (which can be painful) and use SparkSQL to dump to Python (maybe on databri
by disgruntledphd2 4y ago
Yeah, I mean if you're already a JVM shop then you could probably run Spark yourself (which can be painful) and use SparkSQL to dump to Python (maybe on databricks, though I'm not a massive fan) and do your fancy ML stuff there.
Alternatively, you could just use a read-only PG replica and Python here, which would suffice for most DS needs (unless your core product is DS related or you have huuuuuggggggggeeee data (which you probably don't)).
- jpgvm 4y agoYeah that is fair too. Most people have tiny data. Then there is places I have worked... where 10TB qualified as small data that you "just query with Presto" and where more exotic things were needed for actually interacting with the "big data". All relative in the end.
- disgruntledphd2 4y agoYeah, snowflake is super nice for analytics, but PG will get you a long way with less complexity (given that you're already using it for OLTP).