2 ms·
Yeah I'm familiar with Snowpark and I think their mistake is they implemented an API very similar to PySpark but not exactly the same. I can actually see that t
by dayflyer 2y ago
Yeah I'm familiar with Snowpark and I think their mistake is they implemented an API very similar to PySpark but not exactly the same. I can actually see that the 2nd most common engine looked at in the code of my repo is Snowflake so I think there is interest in it even with Snowpark. I will be adding Snowflake support next.
In terms of not liking PySpark, I get that the API is not for everyone and if someone doesn't like PySpark then they are unlikely to like SQLFrame. I think it is great we have a variety of options available to us and ideally each team/organization can find an API that works best for their skillsets/needs but also reduces vendor lock. SQLFrame is a great fit for those teams/organizations who want to standadize on PySpark.