4 ms·
I studied using ML just over a decade ago. I actually compared MLPs to SVMs and had a similar thought to this. It does seem like there is a regression on unders
by gargablegar 3y ago
I studied using ML just over a decade ago. I actually compared MLPs to SVMs and had a similar thought to this. It does seem like there is a regression on understanding some of the fundamentals and older tools of the trade.
I guess everyone gets focused on the newer things.
Really does seem like people rediscovering older endpoints.
- uoaei 3y agoThere's been a huge flood of vanilla software engineers into ML, retconning it as "a subfield of computer science" (computability is a minor concern compared to the statistical underpinnings). They pretend to know the math because they can read the equations, then claim with utmost confidence that actually they're doing all the hard work in ML because they are experts in calling APIs and integrating into products, however useful or useless.
- jjtheblunt 3y agoRetconning? What does that mean?
- chriswarbo 3y agoIt's a term from literature/fiction, where later installments try to explain something from earlier installments; usually in a way that's nothing like the original intent. Applying this to real history is more like "revisionism".
- ReactiveJelly 3y ago"RETroactive CONtinuity", in addition to what the sibling comment said. The Wikipedia top example is Sherlock Holmes dying in a fight with Moriarty and then coming back later when the author relented and decided to write more stories. https://en.wikipedia.org/wiki/Retroactive_continuity https://en.wikipedia.org/wiki/Retroactive_continuity
- jjtheblunt 3y ago(thanks to the sibling comments for explaining...)
- visarga 3y agoThere's a similar trend of accusing LLMs of not really understanding, being just pattern machines. Funny that a whole group of people get the same treatment.
- Q6T46nT668w6i3m 3y agoYou’re comparing a subjective claim to an objective claim. Let’s be clear, transformers do not “understand.”
- famouswaffles 3y agoThey do in every practical, testable definition of the word. "Transformers don't understand" is not an objective claim and in fact any attempt to objectify it leads to the opposite assertion.
- jprete 3y agoI'm not sure what your point is. Many software engineers blatantly pattern-match and copy-paste code to try to get their own stuff to work without understanding what's really going on. This is a long-standing complaint in the industry.
- uoaei 3y agoYou will note the phrasing is tautological in nature and does not apply to all software engineers.
- consilient 3y ago> "a subfield of computer science" (computability is a minor concern compared to the statistical underpinnings) Computability theory is not all of computer science. It's just one subfield among many.
- uoaei 3y agoI'm referring obliquely to a specific nitpick from select CS folks who argue that because the theoretical optimum is not computable in finite time/memory that the statistical basis for understanding ML is irrelevant.
- noduerme 3y agoReally? Here I thought it was a flood of academics retconning neural net code into a "science" now that programmers had made it run in Python for them fast enough to be useful.
- uoaei 3y agoAs hacky as it ends up being in practice, there are some pretty solid theoretical fundamentals to the field of statistical learning. The problem is the theory is constrained either to the micro-scale (individual layers/"simple" models, etc.) or to the supra-scale (optimization/learning theory, etc.). Not much concrete can be said about the macro-scale (individual networks) in theoretical terms, only that empirically they seem tend toward the things the supra-scale theory says they should do. The current controversy in the academia v engineers tussle is 1) what exactly do the empirical results imply and 2) how much does the theory really matter given the practical outcomes. The only thing the two sides broadly agree upon is that some amount of error will always exist because NNs can be broadly understood as lossy compression machines.
- Q6T46nT668w6i3m 3y agoThat’s science. You can’t expect everyone to know everything. It’s a preprint so this is the first opportunity to provide feedback.