Spark/PySpark/polars/dbt/sqlmesh are all data engineering frameworks for data transformation (and some data scientists). If you don't have a data warehouse / O

Spark/PySpark/polars/dbt/sqlmesh are all data engineering frameworks for data transformation (and some data scientists). If you don't have a data warehouse / OLAP system you are generally not in the niche for those tools.