6 ms·
Fuck, imagine how many doctoral theses I could've written every time I tweaked a few lines of code to try some abstract way of recombining outputs I didn't full
by noduerme 3y ago
Fuck, imagine how many doctoral theses I could've written every time I tweaked a few lines of code to try some abstract way of recombining outputs I didn't fully understand. I missed the boat. All this jargon is absolutely for show, though. Purely intended to create the impression that there's some kind of moat to the "discovery". There are much clearer ways to express "we fucked around with putting the outputs of this black box back into the inputs", but I guess that doesn't impress the rubes.
- ResearchCode 3y agoYou're not wrong. Applied ML articles are not worth reading.
- constantly 3y agoI wouldn’t go this far, applied ML articles are my favorite articles. If you’re in the arena, it’s good to see things that other people have done from a practical perspective so you can ape it in your own work or not give it further consideration.
- kristopolous 3y agoI really really wish this culture of expressing simple things in ornate ways would die. All it does is make knowledge less accessible
- awesomeMilou 3y agoPeople have to get their PhD's somehow... ;)
- tinco 3y agoIf the excuse is true and the "ornate" language really is a dense representation of information then it should be fairly trivial to have an LLM agent unsummarize it. There could be a webservice that offers a parallel track of layman's translations of any paper.
- heydenberk 3y agoSounds like you're referring to https://www.explainpaper.com/ https://www.explainpaper.com/ :)
- noduerme 3y agoAn LLM won't tell you that the authors obfuscated it because they don't know what the fuck they did. You need a human for that.
- estebarb 3y agoI haven't tried other models, but if you prompt a recent ChatGTP with "academic style" and ask it to "review and provide feedback" of a paragraph you wrote it will reword it using the most fancy, overselling words it can find. I liked to use it for improving grammar and style, but in later iterations ChatGTP started writing garbage... I'm not sure if that is because training, feedback from users or an attempt to make usage is LLMs obvious to teachers.
- noduerme 3y agoI think this applies to everything right now. Papers like this are just ridiculous examples. In like, 6th grade I won second place at the LA county science fair for coding a simulation of a coyote's life in hypercard (with tons of graphs). Yay. Y'know what? That shit and those graphs would've been incomprehensible to the judges if it hadn't been written in plain language, in an attempt to make them understand what they were looking at. My entire career since has been an attempt to communicate and alleviate the pain points in communication between parties, by way of writing software that encapsulated their descriptions of what they needed. And likewise I never pretended to be smarter or know more than my clients did: Everything must be explained and comprehensible in normal people language. People need to know how shit works, especially if they're paying for it. Or they should. Or if they don't know and don't care, they're fucking negligent. Especially if they say "wow that sounds smart, let's let these guys run our weapons program". To your point, the reason this ornate language thrives and people get away with complacency about how their own systems work, boils down to a silent pact between managers and engineers to sweep everything under the rug out of laziness and ill-will. There's something blatantly mendacious and evil (in the banal way) about the agreement that managers approve black boxes which were approved by complex-sounding papers so that upper management can wash their hands of the results. [edit] maybe I'm just bitter because I spent hours today pondering exactly how many engineers at Monsanto must have known about the dangers of the astroturf, and how many raised their hand, or hid behind a spreadsheet https://frontofficesports.com/investigation-links-astroturf-to-deaths-of-six-former-phillies/ https://frontofficesports.com/investigation-links-astroturf-...
- pedrosorio 3y ago“Engineers who like to pretend to be mathematicians” - I heard once.
- kristopolous 3y agoIn math, this is mostly an English problem I think. Next time you find a Wikipedia math page to be an impenetrable wall of jargon, click the Wikipedia language tool and choose another language, any will do. Then use Chrome's tool to machine translate the foreign language version back to English. I've found invariably this makes the article more coherent then the native English language Wikipedia math page. It says something about the culture for sure.
- thechao 3y agoI know a lot of professional mathematicians who can't make their way through the Wikipedia articles of adjacent fields. The English entries appear to be written by the ghost of Bourbaki!
- deleted 3y ago[deleted]
- consilient 3y agoSome of them are: I've run into sections lifted straight of of Lang's Algebra several times.
- novalis78 3y agoThis works for philosophers too. Try Schopenhauer (already easy to read in comparison) in English translation.
- djmips 3y agoSince LLM are good at translating perhaps that's one of the reasons they seem to do a decent job of explaining things.
- CuriouslyC 3y agoThat's literally the entire field of philosophy after the ~18th century.
- FrustratedMonky 3y agoYep ~18 century Didn't Wittgenstein and/or Nietzsche say something similar. Words are in-adequate for communication, and all philosophy is playing with words. But, Language is all we have to communicate, so guess we are stuck with it.
- TrackerFF 3y ago"Not math-y enough"/ "Needs more math" is a very common feedback ML/AI researchers get when writing papers. The other day I was watching a live-stream of a doctoral defense, as the thesis was quite relevant to my work. So one of the committee members would really pick and criticize the math - ask questions like "You are supposed to be the bleeding edge on this topic, why was the math so simple? Did you research more rigorous theories to explain the math?" etc. (He was awarded the doctorate though) So, I dunno, if that's how things are now - it makes sense to me that the authors go overboard with complicated notation, even if they could have written it much simpler. Probably makes the work seem more rigorous and legit. Doesn't really take that much more time, and it covers your ass from "not rigorous enough" gotchas - though at the expense of readability.
- kristopolous 3y agoGo read any article in the first 200 years or so of philtrans. There's lots of crucial science there written in a way that doesn't have the modern trappings of the form. It's good reading. Maybe some style perturbations borrowed from earlier eras would be good https://www.biodiversitylibrary.org/bibliography/62536 https://www.biodiversitylibrary.org/bibliography/62536 menu on the right Benjamin Franklin, Robert Boyle, Isaac Newton, Maxwell, Ohm and Volt - they're all there. If that style was good enough for them ...
- FrustratedMonky 3y agoI wish also. When I was young and new, so much wasted time trying to parse the 'arcane' math that was really something simple bug dressed up as complicated to give it weight.
- mattew 3y agoWhat I find entertaining/confounding is how difficult the abstracts to these new AI papers are to understand. It feels like academia is pushing this style, so it’s hard to blame the authors since they have to play the game. For reference I have an undergrad degree in computer science, have been working professionally for 25 years, and am fairly data centric in my work. I’m hoping when I run this through GPT4 to get an explanation for a mortal software developer something sensible comes out the other end.
- necroforest 3y agowhich jargon here is "just for show"?
- deleted 3y ago[deleted]
- ftxbro 3y ago> we show that over-parameterization catalyzes global convergence by ensuring the feasibility of the SVM problem and by guaranteeing a benign optimization landscape devoid of stationary points does this mean 'an over-parameterized transformer problem is a convex svm problem'?
- dongecko 3y agoI read it the same way as you did, or at least it's an approximation. In general that's not really surprising. I remember discussions from some years ago about larger networks leading to smother loss surfaces.
- tensor 3y agoThe irony is that your "simplification" uses even more "jargon." But yes, thats how I would read that, and I also see no issue at all with the language in the paper. These terms are used for precision, and have meaning to those in the field. Papers are written for other experts, not laymen.
- ftxbro 3y agoOK but why they write "benign optimization landscape devoid of stationary points" instead of "convex" other than "just for show"? In my understanding it's not better for either audience experts or laymen. For experts it would be more clear to just say convex and they would know the implications, and if someone doesn't know what convex means they probably also aren't going to be on board with 'stationary points'. Also I'm not trying to pick on the authors I'm just trying to answer the question of which specific parts could be seen as 'just for show'.
- agent327 3y agoYou think it's that easy, but, as frequently the case with transformers, I believe there's more here than meets the eye.
- gargablegar 3y agoI studied using ML just over a decade ago. I actually compared MLPs to SVMs and had a similar thought to this. It does seem like there is a regression on understanding some of the fundamentals and older tools of the trade. I guess everyone gets focused on the newer things. Really does seem like people rediscovering older endpoints.
- uoaei 3y agoThere's been a huge flood of vanilla software engineers into ML, retconning it as "a subfield of computer science" (computability is a minor concern compared to the statistical underpinnings). They pretend to know the math because they can read the equations, then claim with utmost confidence that actually they're doing all the hard work in ML because they are experts in calling APIs and integrating into products, however useful or useless.
- jjtheblunt 3y agoRetconning? What does that mean?
- chriswarbo 3y agoIt's a term from literature/fiction, where later installments try to explain something from earlier installments; usually in a way that's nothing like the original intent. Applying this to real history is more like "revisionism".
- ReactiveJelly 3y ago"RETroactive CONtinuity", in addition to what the sibling comment said. The Wikipedia top example is Sherlock Holmes dying in a fight with Moriarty and then coming back later when the author relented and decided to write more stories. https://en.wikipedia.org/wiki/Retroactive_continuity https://en.wikipedia.org/wiki/Retroactive_continuity
- jjtheblunt 3y ago(thanks to the sibling comments for explaining...)
- meltyness 3y agoOr, optimistically, this is really how they think about these things, and you should simply be happy they're not trying to obfuscate their findings.
- thechao 3y agoWatching the AI community rediscover automatic differentiation 20+ years after the field was considered "mature" was equal parts frustrating & fascinating. The frustration was them rewriting the history of discovery, but without any sort of sense or rigor ... and it was also the most fascinating!
- AndrewKemendo 3y agoThis is indeed the frustration I'm waiting for some fresh group of grad students to make a breakthrough using a reinvented version of Pearls "Do" calculus or maybe they make some narrow breakthrough using BayesNets and everyone geeks out on those for a while *I do think transformers (much like ff networks + backprop from 2012-2018) are probably a lasting software architecture for inference applications until we come up with new hardware, and move beyond GPU focused computing It's exciting to see it all working, but disheartening how a-historical this last few years has been in AI - with the exception of Brooks, Sutton and a few other greybeards in the field who say similarly
- barelyauser 3y agoScarcity is not a myth.
- AndrewKemendo 3y agoThere is no scarcity of fundamental human needs of water, food, shelter and love. The only reason someone lacks them is because someone else is hoarding them. This is well established in global trade metrics.
- capableweb 3y agoThe funny thing is that this constantly happens in every field ever, humans truly excel at repeating history without learning from the past. Another example: - HTML served by static file servers - HTML generated by backend - HTML enhanced with small JS snippets - HTML generated by frontend, but served by backend - Go to step one, not learning why anyone moved on from the previous method
- deleted 3y ago[deleted]
- Q6T46nT668w6i3m 3y agoIt’s a preprint on arXiv, not the Magna Carta.
- deleted 3y ago[deleted]
- maf12 3y agoI think the main motivation in ml theory that touches current SOTA is not "expressing simple ideas with a jargon for show". Jargon is necessary, as much as some (mostly very practical) engineers or software people cannot see it due to how unnecessary it seems to them (as they are used to practically and quickly express themselves). It's a jargon for the mathematics of machine learning, which is pretty unstandardized so to speak. So you need to define yourself. And without a jargon and clear proofs, what you do is just brainstorming at most. The value of such work is that their statement is pretty clear, proved and contain hypotheses which can be tested by the future papers. Here is an example: to explain the existence of adversarial example, there are 2 suggestions without a jargon: 1) that the decision boundary is too nonlinear, 2) that the decision boundary is too linear. Both of these explanations contradict and stated without any real proof and unfortunately can be widely heard in most of the adversarial example papers. If we were to have clear formulations of these two statements, we could have tested both of these claims but unfortunately the papers that suggested these theories didn't put effort for defining a jargon and putting their suggestion as a clear-formal statement.
- r-zip 3y agoWhat are you talking about? Did we read the same abstract?