> I use LLMs for annoying shit that used to take an inordinate amount of time. For example an analyst gives me a CSV, it has some data clean up issues, eventual

> I use LLMs for annoying shit that used to take an inordinate amount of time. For example an analyst gives me a CSV, it has some data clean up issues, eventually it needs to become a series of SQL inserts. The CSV to SQL for analysts problem is a data integrity problem that is domain specific and not tool specific. Remember that a 'relation' in relational databases is just a table, specifically named columns and tuples (rows). A CSV is also just tuples (lines), but obviously SQL also typically has multiple normalized tables etc... Typically bad data is worse than missing data. For analysts, missing data can lead to bias and reduced statistical challenges, but methods exist and it can often be handled. Bad data, on the other hand, can be misleading, deceptive and/or harmful. An LLM will be its very nature, be likely to produce bad data when cleaning. The risk of using an LLM here is that it doesn't have context or nuances to deal with that. Data cleaning via (sed,grep,tr,awk), language tools or even ETL can work.... I promise you that fixing that bad data will be far worse. But using it in a red-green-refactor model may help with the above, but you will actively need to be engaged and dig through what it produces. Personally I find it takes more time to do that than to just view it as tuple repacking...and use my favorite tools to do so. Data cleaning is hard, but it is the context specific details that make it so.