9 ms·
Lots of great insight. Here’s one: “Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other
by dr_dshiv 4y ago
Lots of great insight. Here’s one:
“Given the long timelines of a PhD program, the vast majority of early ML researchers were self-taught crossovers from other fields. This created the conditions for excellent interdisciplinary work to happen. This transitional anomaly is unfortunately mistaken by most people to be an inherent property of machine learning to upturn existing fields. It is not.
Today, the vast majority of new ML researcher hires are freshly minted PhDs, who have only ever studied problems from the ML point of view. I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML.”
- michaelrpeskin 4y agoobligatory XKCD: https://xkcd.com/793/ https://xkcd.com/793/
- fknorangesite 4y agoSimilarly: https://www.smbc-comics.com/comic/2012-03-21 https://www.smbc-comics.com/comic/2012-03-21
- javajosh 4y agoThat's a great xkcd, but there are 2 upsides to this arrogant approach. First, arrogance is a nerd-snipe maximizer. Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. It doesn't happen often, but when it does happen and there is no clout like "emporer's new clothes" clout. EDIT: The downside, of course, is that you appear arrogant, and people won't like you. This can hurt your reputation because it is apparently anti-social behavior on several levels. I think its fair to call it a little bit of an intellectual punk rock move that is probably better left to the young. It's an interesting emotional anchor to mapping a new field, though.
- jojosbuddy 4y agoNot laughing! /s (physicist here) Actually most applied physicists like myself go down that path cause we're pretty efficient, lazy folk & skip through as fast as possible--I call it the principle of maximum laziness.
- anonymouskimmer 4y ago> Second, there is a small chance you're absolutely right, and you've just obviated a whole field from first principles. Mostly when I read about things like this happening, it's happening to a formerly intractable problem in mathematics. Do you have examples outside of math?
- passer_byer 4y agoAlfred Wegener as the initial author on the theory of plate tectonics comes to mind. He was a trained meteorologist who observed the similarities between geological formations between the South American east coast and African west coast. He was lucky, in that his father in-law was a prominent geologist and helped him defend this thesis.
- anonymouskimmer 4y agoOh yeah, revolutionary insights are very important for the advancement of knowledge and the elimination of wrong ideas. But as you wrote, this was the work of a thesis, not a random commenter from another field.
- ramraj07 4y agoThe only time I’ve seen this happen is AlphaFold. Every other time some math or software guy came into biology it worked exactly like in xkcd.
- anonymouskimmer 4y agoThat's a good one.
- deltree7 4y agoThe biggest innovation of this decade, LLMs was created by Math/CS not linguistics
- jltsiren 4y agoBiology has already been transformed by mathematics, statistics, and CS for decades. These days, if something like in that xkcd happens, those people are probably not even familiar with the relevant parts of math / stats / CS.
- CogitoCogito 4y agohttps://xkcd.com/1831/ https://xkcd.com/1831/
- Nevermark 4y agoAn introductory book on neural networks, written a few decades ago, starts with a short history of the field at that time. This included a phase where some physicists began having opinions about the subject, anticipating they might quickly find a model for the brain, with fields, or other physics-like paradigms. Their expectations were that with their superior understanding of all things fundamental, they would rush in, and rush out. Leaving the stunned machine learning researchers dazzled, frazzled, and asking "Who was that masked physicist?!?" Except their ideas went nowhere. I can't find the book, but this story made for a memorable foreword.
- qumpis 4y agoI'm yet to see an ML PhD be required to learn chemistry to a similar extent that chemists would need to doing ML (especially at research level)
- kevviiinn 4y agoThat's because application and research are quite different. If one does a PhD in ML they learn how to research ML. Someone with a PhD in chemistry learns how to research chemistry, they only need to apply ML to that research
- ghaff 4y agoBack when O'Reilly was still hosting events (sigh), at one of their AI conferences, someone from Google gave a talk about differences between research/academic AI and applied AI. I think she had a PhD in the field herself but basically she made the argument that someone who is just looking to more or less apply existing tools to business or other problems mostly doesn't need a lot of the math-heavy theory you'll get in a PhD program. You do need to understand limitations etc. of tools and techniques. But that's different from the kind of novel investigation that's needed to get a PhD.
- frozenport 4y ago>>math-heavy theory you'll get in a PhD program Lol. With the exception of niche groups in compressed sensing, math doesn't get too hard. Furthermore, ML isn't math driven in the sense people are trying things and somebody tries to come up with the explanation after the fact.
- selimthegrim 4y agoWell I think the issue is more of if you’re Genentech and you need ML people and can’t afford to pay them you’re probably better off retraining chemistry PhDs.
- kevviiinn 4y ago
- ternaus 4y agoLoved this argument as well. With respect to more mature research fields the entry point to ML is much lower. Hence I always recommended people to have major in Physics, Chemistry, Biology etc but look for projects in these fields that could benefit in ML (I have a number of them about Physics). So that argument was not novel. But the fact that pure ML PhDs will have significantly lower multidisciplinary knowledge is a good one. It could be compensated by the fact that ML is growing fast and all kinds of people join the ride, but still.
- asciimov 4y ago> "[...] I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML.” That's good ol' academic gatekeeping for ya, available wherever PhD's are found.
- mattkrause 4y agoThere’s more to it than that. CS is unusually easy to learn on your own. You can mess around, build intuition, and check your progress—-all on your own and in your pyjamas. It’s easy to roll things back if you make a mistake, and hard to do lasting damage. There are tons of useful resources, often freely available. Thus, you can get to an intermediate level quickly and cheaply. Wet-lab fields have none of that. Hands-on experience and mentorship is hard for beginners to get outside of school. There are a few introductory things online, but what’s the Andrew Ng MOOC for pchem?
- cratermoon 4y agoOn the flip side, software development and engineering rigor is largely absent in academics, as has been discussed previously here on HN. This is enough of an issue to make replicating research even when the code and data are provided, but it's an even bigger issue trying to turn academic code into a product.
- jltsiren 4y agoThis is more about ML than CS. ML is fundamentally about developing general-purpose algorithms applicable to a wide range of problems. If your job is using ML to solve problems in chemistry, it's more about chemistry than ML, and a chemistry background is more important than an ML background. It's unlikely that you have to develop novel ML methods for the problems you are facing. I've seen the opposite in bioinformatics. While dedicated bioinformatics programs are now common, you still see many CS / mathematics / statistics / physics / EE people moving to bioinformatics after bachelor's / masters's / PhD / postdoc. In some bioinformatics jobs, you often have to solve new computational problems, and it's easier to teach enough biology to people with a methodological background than the other way around.
- knorker 4y ago> I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML. Haha, I've seen that for so many topics. "It's much easier for someone used to circuit switched phone networks to learn IP than the other way around", says the person who started with circuit switched. I just thought "dude, you're literally the worst at IP networking that I've ever met. Your misunderstandings are dug into everything I've seen you do with IP".
- dekhn 4y agoAs somebody who has crossed the line between ML and chemistry many times, I would love to see: more ML researchers who know chemistry, more chemistry researchers who know ML, and best of all, fully cross-disciplinary researchers who are both masters of chemistry and ML, as those are the ones who move the field farthest, fastest.
- whymauri 4y agoYou could probably fit all the people who fit the last criteria in the same room (chemistry side is probably the bottleneck, especially drugs which is a effectively a specialization).
- pama 4y agoPresent. I think there exist many of us. Chemistry is a very wide field though, so not sure if organic synthesis vs theoretical chemistry vs physical chemistry vs biochemistry will end up more useful to help tackle drug discovery problems or other chemistry applications. Same with ML I suppose; even though the specialties are less concrete nowadays, the breadth of publications has far exceeded the breadth of modern chemistry.
- quickthrower2 4y agoSociety is not structured to encourage this. Getting a job sooner is more lucrative. Any breakthough you make having studied for a couple of decades is property of a corporation not you.
- randmeerkat 4y ago> Society is not structured to encourage this. I wish more people understood this and the value that it would add to society. For some reason most people understand how roads and transit improve commerce, but fail to understand how education and social support for the individual does as well.
- quickthrower2 4y agoSport incentives work out nicely for getting a supply of top talent. Big money through sponsorship/ads/tickets, fairly formulaic system to get better at a sport (top coaches/teams, get fitter, practice a lot), an easy way to identify progress (wins games, scores goals). But for sciences, someone could do great work for 20 years and produce obscure papers that might stay obscure, or may unlock the mystery of the universe or help us meet the world's energy demands without pollution! So the pay off if magnitudes higher than sport, but magnitudes less probable. Someone will run 100m fast, guaranteed, but will we solve science problems X Y Z and when?
- SkyBelow 4y ago>I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML. Perhaps this is selection bias. Among all the chemists, the ones who will dabble in ML will likely be the chemists with the highest ML related aptitude. In contrast, a ML expert on a chemist project is more likely not being internally driven to explore it but instead has been assigned the work, which means that there is less bias in selection and thus they have less chemistry aptitude.
- kergonath 4y ago> I’ve seen repeatedly that it’s much harder for a ML PhD to learn chemistry than for a chemist to learn ML I can confirm. We regularly look for people to write some computational physics code, and recently for people using ML to solve solid state physics problems. It’s way easier to bring a good physicist or chemist to a decent CS level (either ML or HPC) than the other way around.
- hgsgm 4y agoThis is because software developers are too good at automation themselves out of a job.
- MichaelZuo 4y agoIt's also because nobody goes to get a phd in solid state physics for the money or career prospects, at least not in the last decade. So it's a small and self selected group.
- kergonath 4y agoWe love automation. There is just too much to do, and the field is unbounded. More automation means more papers and more interesting science.
- epolanski 4y agoIt's the same reason analysts come from math rather than economy degrees. You can teach a mathematician what he needs to know about finance, you can hardly do the opposite.
- kenjackson 4y agoChemistry is a centuries old discipline, that people study undergrad a full four years before getting a PhD in the field of chemistry. ML is a, practically speaking, 15 year old field that PhDs often begin to study after a couple of AI courses in undergrade and a specific track in grad school (while they study other parts of CS as part of their early graduate CS work). There's just way less context in ML than Chemistry.
- smaddox 4y agoI don't think that's really true, but you don't need the full context in ML in order to apply it to your field of interest. Just like you don't need the full context in Physics to apply it to an engineering problem. Some of the most successful ML researchers have read several decades of research papers. It's not uncommon to see references to papers from the 70s or 90s. Edit: No doubt that the relevant parts of stats, random matrix theory, and ML is a newer field than Physics or Chemistry, though.
- PartiallyTyped 4y agoDL is a 15 year old field, ML is far older than that.
- kenjackson 4y agoThat’s why I said practically speaking. And realistically I don’t think you can even generously consider anything prior to perceptrons.
- PartiallyTyped 4y agoML is so much more than DL / NNs. I recommend that you open a book like Bishop or ESL, the overwhelming majority of ML has been non DL going back to the 70s.
- kenjackson 4y agoMost of that I learned doing non-ML stats work. But regardless, my point was the history of it is far less than chemistry. Let’s say ML started at the turn of the 1990s to be generous. Still much younger than chemistry.
- Quekid5 4y agoDo you have any anecdata on topologists?
- fancy_pantser 4y agoTopologists will occasionally come to work with their clothes on inside out and insist nothing is wrong.
- cratermoon 4y agoTopologists can't tell the difference between a coffee cup and a donut, and will occasionally make a mess trying to dunk their donut in their coffee.
- Fomite 4y agoIronically, in my field I'd argue this isn't necessarily true - so long as the ML PhD sees value in that field. Part of this is because public health as an undergraduate discipline is extremely new, so the field is used to having to teach Folks From Elsewhere about our field, rather than the fields built on the assumption of a large foundation of undergraduate coursework.
- j45 4y agoInterdisciplinary and intersectional skillsets are critically valuable. Separation of concerns especially at the beginning innovation stages can be more of an inhibitor than accelerator of success. Scaling and growth is another thing. A similar pattern existed in the late 90s with web developers coming from many different industries with their domain knowledge and domain insight. The code and frameworks were early but the insights of what was a problem most pressing to solve.
- theGnuMe 4y agoThe ML folks pretty much nailed protein folding though. Where any of them phds in molecular bio or chemistry?
- angarg12 4y agoNot so dramatic, but I have a traditional CS background and got into ML much later in my career. Last year my company formalized the process to hire ML Engineers. The interview is the same format as the software engineer round, but with an extra theoretical ML round. I've observed two distinct groups of candidates in the process. One are recent grads with PhDs in ML. Other are people from diverse backgrounds that happened to start working in ML in their current job. The first group tends to excel in the ML theory part, but flunk the leetcode style coding questions. The second group tends to do better in coding, but do worse in the theory part. This is exactly what the process has been designed to do. I am very put off from looking for ML Engineer positions in other companies if they follow a similar process. I know I would fail the interview for my current job. They could get away with it for a while, but I doubt it's sustainability or desirability in the long term.