3 ms·
Ah yes then I am very familiar with Athena- like tools. At my old job I used pyspark and I must say I did enjoy how easy it was to make custom python UDFs in co
by therobots927 2y ago
Ah yes then I am very familiar with Athena- like tools. At my old job I used pyspark and I must say I did enjoy how easy it was to make custom python UDFs in combination with SQL so I miss that in a pure SQL environment. But for a lot of applications SQL gets the job done.
- fifilura 2y agoYeah this is why i am hoping that EMR serverless would be the next step. You get the freedom of not having to set up the resources. It is cheap because you are basically eating the breadcrumbs of available cpu capacity "oh i have 10 idle cpus here in Italy to do one part of you map job" etc. But you are not locked in to SQL. (Athena has UDFs but they are not very user friendly).