I see some python / pandas hate here and I have to defend it a bit: We all agree a query like: tb_1 = pd.read_sql(big_table_1) tb_2 = pd.read_sql(big_table_2)

I see some python / pandas hate here and I have to defend it a bit: We all agree a query like: tb_1 = pd.read_sql(big_table_1) tb_2 = pd.read_sql(big_table_2) table_i_want = pd.merge(tb_1, tb_2) table_i_want =table_i_want[table_i_want.my_id.isin([1,2,3])] is completely stupid, put that in the DB! And a lot of (very very) unexperienced people will write such a query, since they dont know anyhting about DBs and SQl. However, pandas and python is here for what comes after that! Like a very complicated groupby logic, or invoking some apis to enhance data or complicated string processing or some kind of error measure or invoking some ML models or sending a slack message or building features for ranking or whatever. I know a lot of people here love their SQL but queries like mentioned above are either impossible or become horrible horibly complicated and impossible to debug. EDIT: And I want to add one extremely important thing. Whenever you can put stress away from your DB it is a good thing. Since your DB has to run all of the time and will be already stressed from Frontend requests. Combined with the fact that we are often talking about non-time sensitive work wouldn't it be better if your data scientists work on a database dump instead of continously hitting the DB?