7 ms·
they want to get well-paid, high prestige jobs using machine learning. whereas doing the exact same task with a regression in Excel turns it into low-prestige
by currymj 8y ago
they want to get well-paid, high prestige jobs using machine learning.
whereas doing the exact same task with a regression in Excel turns it into low-prestige clerical work.
i think they’re acting pretty rationally once you consider their own incentives. (plus of course, the general cultural perception that machine learning is cool, smart, and cutting edge must rub off on them too and how they perceive themselves.)
- humanrebar 8y ago> i think they’re acting pretty rationally once you consider their own incentives Is that even true? Or are candidates and companies just caught up in a cycle of signaling? In my experience, there is amazing demand for pretty straightforward software engineering, and the talented leaders there are coveted and paid very well. Analytics, ML, etc. are desired as well, but are a minority portion of the demand.
- HumanDrivenDev 8y agoIf you're in a '2nd tier' first world country like Canada, Australia, the UK, Germany etc, Machine Leanring is one of the only ways you can get a salary comparable to a 'normal' Software Dev in the US. Keep that in mind.
- kamikaz1k 8y agoWhich countries are first tier?
- HumanDrivenDev 8y agoNot many, Switzerland and Norway are all that come to mind. The real point I am trying to get across is the huge gulf between the career of a US software engineer vs almost anywhere else in the world.
- goatlover 8y agoI thought the well-paid, high prestige jobs required a Phd in the field? Or has machine learning become widespread enough for the unwashed programming masses?
- fragmede 8y agoThere are point and click GUI tools and frameworks and libraries that simplify a great deal of complexity involved, but domain expertise is still required to be able to understand what all the parameters actually mean, eg how not to overfit the model to the training data. There are well paying ML jobs that don't involve getting a PhD first.
- PeterisP 8y agoWe now can teach applied ML to undergrads and have them complete decent practical projects using the various ML frameworks which now are quite high-level and abstract away most of the math. Properly understanding your data and what's possible is still a bit of a challenge, but that's more about general intelligence and less about any particular skill.
- gaius 8y agoI am not looking to move right now but I still have a couple of job searches saved from before and everything it seems is data science, machine learning, big data etc now. Even jobs that are very obviously Excel or similar and paying accordingly.
- rjsw 8y agoIt is also difficult to get any funding or interest in what I would call "medium data".
- rspeer 8y agoEven though every nearly every enterprise runs on medium data, and can't admit it because that would make them seem inferior to their competitors, who they think have big data.
- gaius 8y agoI remember when we used to just call that “data”
- gaius 8y agoI suspect a smart engineer can make the vast majority of big data problems fit in memory by appropriate filtering and pre-processing but you’d never get your purchase order for that 200-node Hadoop cluster approved if you admitted it...
- currymj 8y ago“It takes a big man to admit his data is small.”
- mr_toad 8y agoSometimes those terms are added to advertisements to attract people who want to be doing those things, not because they are actual job requirements.
- ryandrake 8y agoWhy solve the problem in Excel when you can solve it with a 1000 line C program and earn 4X what the Excel analyst makes? Why solve the problem with a deterministic and efficient C program when you can solve it with 99% confidence using TensorFlow on a 100 node Spark cluster and earn 4X the C programmer? Incentives are currently aligned toward everyone choosing to solve problems using the most complicated tools available.
- kriro 8y agoMy counterclaim would be that doing ML in code is actually easier these days than doing stuff in Excel (and a 1080 is sufficient). So why not solve the problem with a sufficiently easy to use tool? Your other point is valid. If it can be solved deterministically than you should probably do that (but I'd probably not reach for C since I don't need the C-efficiency hammer right away)
- TeMPOraL 8y agoBecause it's likely that when solving things in Excel, you actually know what you're doing. The popular ML approach these days is throwing data at whatever you downloaded from Kaggle and hoping it sticks.