3 ms·
Yes I'm familiar with Ibis and we use the same technology to do the SQL generation (SQLGlot). Like most products, I made this due to two problems I commonly fac
by dayflyer 2y ago
Yes I'm familiar with Ibis and we use the same technology to do the SQL generation (SQLGlot). Like most products, I made this due to two problems I commonly faced:
1. Generate SQL that represents a PySpark DataFrame pipeline that I maintain so others who prefer SQL can easily understand it
2. I'm already proficient with the PySpark DataFrame API but not all projects I work on require Spark.
In both cases Ibis would not have been a solution for my issue. I think if you haven't learned either then it would come down to if you are more interested in the learning the PySpark DataFrame API or Ibis DataFrame API.