3 ms·
All software is effectively a set of ETL jobs with a user-friendly interface wrapped around. Those fancy algorithms, languages, frameworks, etc are simply a ver
by bob1029 3y ago
All software is effectively a set of ETL jobs with a user-friendly interface wrapped around. Those fancy algorithms, languages, frameworks, etc are simply a very ceremonious way to get data from A to B.
The schema/data are completely agnostic to the computation. You don't even need a computer. 100% of schemas are possible to print out on sheets of physical paper and iterate offline.
- gridspy 3y agoComputer used to be the job title for people who DID do this by hand. It was co-opted by the technology which replaced the job. Of course modern computation is often impractical to do by hand. It might even be so complex that humans would make too many errors and take to long to ever complete correctly.
- esafak 3y agoCommon use cases were abstracted into libraries and services because nobody should have to reinvent the wheel. This let people operate at a higher level of abstraction and concentrate on more complicated tasks. These too became abstracted, and the cycle repeated.
- sirsinsalot 3y agoWhile true, I think this is reductive. When you're dealing with most modern software, there's so many of these ETL-like pipelines, abstractions, black-boxes and just sprawling complexity that the job becomes much more about complexity management than anything. No component exists in a vacuum, I would argue software is as much about managing the complexity of the data and algorithms towards a purpose as it is about the purpose, data or algorithms. Entropy comes for us all, after all.