4 ms·
Use SQL for data pipelines, Airflow Sucks for MLOps, Draw plots on the terminal
- lorenzotenti 4y agoAnd Data-Centric AI!
- brahbrah 4y agoI know it’s not the focus of the article, but a personal pet-peeve of mine is when pandas gets mentioned in articles and discussions regarding alternatives to sql and data pipeline tools. Pandas was never meant to be a replacement for sql, it was meant to be a replacement for excel in highly dynamic quickly changing financial and econometric excel models. It just so happens there’s a lot of overlap between this and more proper data pipeline tools. But for its original purpose I don’t know if anything that still comes close to the useability of pandas.
- lorenzotenti 4y agoYou're right, but pandas today is used a bit everywhere, and in data pipelines too, even if it wasn't its main goal.