5 ms·
I agree; in my experience, pretty much everything is ETL. We take data from one thing, change it a bit, and put it somewhere else. Sometimes, as a treat, we tak
by taberiand 2y ago
I agree; in my experience, pretty much everything is ETL. We take data from one thing, change it a bit, and put it somewhere else. Sometimes, as a treat, we take data from two things, put them together, and then put that somewhere else.
Frontend, backend, databases, services, reports, whatever - ETL.
In that context, the types and transformations between types are the most important feature. (Everything is actually Category Theory)
- necovek 2y agoExcept that "ETL" is a terrible name for it: one of those generic acronyms, that even when expanded to "Extract, Transform, Load", needs explanation for at least the "E" and "L" parts. To me, looking at it as "functional" approach instead (data in, operate over that data, data out) is cognitively simpler.
- namaria 2y agoI remember working on an ETL pipeline in Airflow years ago and thinking: "we're defining and programming an abstract computer". I guess with general computers everything we do is basically defining nested specific computers. That's, I think, the insight behind SmallTalk and the original concept of objects it used: the objects were supposed to represent computers and the message passing was an abstract network layer.
- whattheheckheck 2y agohttps://rachit.pl/post/you-have-built-a-compiler/ https://rachit.pl/post/you-have-built-a-compiler/