4 ms·
Agree with this. There is a lot more information to process, and it's being generated at greater and greater rates. Ideas that used to be an entire paper in the
by maxFlow 4y ago
Agree with this. There is a lot more information to process, and it's being generated at greater and greater rates. Ideas that used to be an entire paper in the 1970s have now become an extra paragraph in an undergrad textbook from the 2020s. Further and further specialization in every field--e.g. in SWE: SysAdmin branched out into DevOps and SRE, Data Science branched out into Data Engineering, Analytics Engineering, ML Engineering, etc. It seems we have built more efficient tools, but the barrier to entry to use them is taller; and once you can use them they seem to require longer and longer hours of tinkering and supervision. Yet if you don't put in the hours your tools/systems start to decay simply by virtue of progress[1]. Tool-integreation costs are also higher: new tools don't exist in a vacuum but rather need to be adapted and vetted by the host system.
Somewhat related, but I believe this is the reason why academia will continue to lag behind industry. Academia has a bias for fundamental knowledge, but as information grows, fundamental knowledge represents a smaller proportion of all knowledge[2]. Anecdotally I have observed a 2-5 year gap between innovation in industry vs academia (CS). Academia often far behind looking to monetize the success of industry via newly minted formulaic programs. This was not always the case, e.g. databases came mostly from academia into industry in the 1970s, but nowadays innovation in the field seems industry-driven. To be fair, perhaps academia has already explored much of the relevant concepts, and it's easier to innovate by way of implementing these same ideas in different programming languages and business models.
[1] What are we left with then? More efficient but also less robust systems? Being more efficient, i.e. generating more output, seems to be the chosen trade-off today, when you can afford extra human intervention, but will it ever be the case when we reach an inflexion point and economics favor robustness/stability over output?
[2] Not to mention the issues with defining something as fundamental. Should fundamental knowledge be frozen in time, or instead act a sliding window? I.e. when, if at all, does something go from fundamental to obsolete? and when does something go from innovation to fundamental? Is it merely popularity driven?