3 ms·
This quote is ridiculous. Some people enjoy plumbing high speed reliable/transparent data pipes. I could care less what goes thru the pipes I make.
by humbleMouse 8y ago
This quote is ridiculous. Some people enjoy plumbing high speed reliable/transparent data pipes. I could care less what goes thru the pipes I make.
- m_ke 8y agoThe "T" in ETL is important here. You might not care but the end users of that data should. It's very easy to take raw data and strip it of a lot of useful information by transforming and normalizing it. The person doing modeling or data analysis should ideally be dealing with the raw data, know how it was collected and understand what each field really means.
- closeparen 8y agoThe unglamorous ETL work is the config and query writing to apply infrastructure building blocks to particular pairs of tables, not the creation of the generic infrastructure.
- pishpash 8y agoExactly. They are talking about manual, custom, one-use ETL that needs to be maintained forever. Don't nobody got time for that. On the other hand, sometimes you can't get away from that because different orgs/humans generate trash data in idiosyncratic forms. Things will get much better once we pry all the human hands off of data and let engineers redesign all of them across the world. Not going to happen soon.