14 ms·

Machine learning is still too hard for software engineers

178 points by gk1 5y ago

yobbo 5y ago
The article contains lists with bullet items and then suggests that if these were "easy", ML would be more accessible. Well true, but much of the "difficulty" is in establishing faith that the models are meaningful and valuable. The article is talking about applications where the value is less-or-equal to the work of learning and applying some library (it mentions days or weeks). What is the actual value of applying ML in these cases?
kellymore 5y ago
I think it goes without saying that this is too relative. One thing that is common in both fields is the level of research and reading docs it entails. I used to tell my students "software engineering is 70% reading and 30% coding". This remained consistent as I dove into Data Science and now ML with computer vision at Roboflow. Of course the first time I was exposed to it during a fellowship, I thought I was out of my depths, but this comes with everything new. To @deepsun point, I've found Kaggle's intro courses quite excellent as well.
Irishsteve 5y ago
I'm doing ML a long time, practice makes perfect etc. If anything is too hard, it is front end.
- marcinzm 5y ago
  Yeah, it felt like a sales person being confused why can't become an expert fullstack software engineer on up to date modern stacks without a lot of effort and time.
- tiborsaas 5y ago
  As a senior frontend engineer, practice makes perfect, frontend is easy-peasy. ML/AI seems to have this thick wall of math around it, if I really want to understand what makes the models going.
rchaves 5y ago
I really like the premise, and I really agree with it, but I don't think a SaaS is a solution, the solution is trying to find better abstractions that makes things simple and easier for the developers, using code, and without limiting their flexibility, but that's the hardest thing to do! As an experienced software developer who used to learn a new framework every week, I thought ML was going to be piece of cake. In reality, I went down this rabbit hole 4 years ago, and I'm still in there, I was so innocent back then
- oblak 5y ago
  Good points. But truth is SaaS can be a great solution if you pick Nyckel! Sorry, it had to be done.
- lbriner 5y ago
  The issue with abstractions is that ML isn't just a few variables or dissimilar systems to choose from, the whole problem domain that could be helped with ML has so many dimensions. Natural language, speech processing, pattern recognition etc. are all completely different. The only thing they have in common is that you might use computers for them. I think that is why people find ML so hard. It isn't a single sausage machine to create insight from, it is a set of entire stacks from philosophy all the way down to the electronics.
version_five 5y ago
I'd argue software engineering is still too hard for ML engineers. Most of ML (not research but commercial applications of what's already been demonstrated) is now well within the realm of engineering, but there are few standard practices, bodies of knowledge, or agreed on processes for doing anything. These are problems that engineering formalism solve, not another auto-ml tool. Maybe I'm saying the same thing as the article, just framed differently?
- yldedly 5y ago
  In my experience, both are true. I'm more on the ML side, and I can tell I don't have the kind of routine and habits that good software engineers have, though I'm learning. But on the other hand, and I've seen this from software engineers who've made the transition to ML, and clearly have a good handle on the concepts (in one case even published papers in ML journals), they don't seem to have the intuition that allows them to select the right tool for the job, or understand when a given technique is appropriate; you could call that "modeling maturity" - a combination of mathematical maturity and the skill of relating domain knowledge to modeling choices.
  - antman 5y ago
    Absolutely agree, the major challenge has been to explain to software engineers that the pick up an algo from github, monkey see monkey do a tutorial followed by automl or grid search is pretty irrelevant nowadays. If you can’t implement any existing algorithm from a scientific paper it is pretty hard to understand the flexibility that is required designing a training pipeline. In terms of tech stacks, cloud providers’ sample architectures and code samples are a base for cloud components and API understanding and not actual implementations.
  - 7h3kk1d 5y ago
    I think the above commenter is more critiquing the notion of relying on intuition all together.
    - yldedly 5y ago
stevenalowe 5y ago
So is software engineering
etangent 5y ago
One might as well write an article claiming that "UI design is still too hard for software engineers" or "controlling a nuclear power plant is still too hard for software engineers" -- which are true (and it is equally true that software engineering is hard for UI designers and nuclear power plant operators). Who came up with this silly idea that something that is a valid knowledge domain of its own is suddenly going to become "easy"?
zwieback 5y ago
Missing in these advertisements-disguised-as-blogs is an estimate of effort or time. Let's say ML is critical to a new product or internal tool, how many man-years is it reasonable to invest? If your expectation is using ML like a drag-and-drop app builder then you're probably best off using a canned tool but then you won't really have any competitive advantage.
ramesh31 5y ago
Can confirm. Data science team just schooled the entire engineering department in our Deep Racer tournament.
- exdsq 5y ago
  Makes sense though, right? Data team did data stuff better. If you had a different task with an engineering focus you'd see the opposite.
- fxtentacle 5y ago
  In the gocoder Bomberland competition, my DL AI trained on 1.5 billion timesteps just lost against someone handcrafting algorithms in Python in one day. You're absolutely right, sometimes, simple and predictable solutions are much better than AI magic ^^
deleted 5y ago
[deleted]
uoaei 5y ago
Wow, it's almost like machine learning and software engineering are different disciplines entirely! Just because both involve coding doesn't mean software engineers should be expected to have the math chops (stats/prob, linalg, calc, etc.) to make machine learning work for them... Vice versa is a little more complicated, because ML/DS can be done very inefficiently without the proper coding practices, but understanding the math is independent of that so that point still holds for this comparison.
- jstx1 5y ago
  > machine learning and software engineering are different disciplines entirely I don't think so. Or more precisely, they might look different academically but you need to have both as a skill to build something useful.
  - uoaei 5y ago
    In-built is the assumption that one person must have both competencies. This may be true for cash-strapped startups but this hardly plays well as general advice.
    - jstx1 5y ago
      It's not an assumption, that's my main point - you need both skills in the same person. And it's true everywhere including in the biggest companies.
      - ngc248 5y ago
        Definitely not ... I am a SWE in a big company and working on ML projects for the past 4 years. We have a data scientist team which does the data exploration and comes up with a model. Their output would be a jupyter notebook. Then we have a team of "Applied ML practitioners", which I am a part of, we productionize the jupyter notebook, by setting up pipelines, services etc. We understand ML algos, stats, probability etc, but not as much as our data scientist team does. Having both in the same person would be good, but is not necessary.
howmayiannoyyou 5y ago
The best tutorials for indoctrinating software engineers in ML are written accessibly & use real-world business-case data, or easily comprehended data sets. One great example of this, from graph theory, was this fantastic 2013 article by Kieran Healy: https://kieranhealy.org/blog/archives/2013/06/09/using-metadata-to-find-paul-revere/ https://kieranhealy.org/blog/archives/2013/06/09/using-metad... There's no getting around the complexities of fit, bias and customized models for many ML problems, so my observation above is obviously limited in its applicability.
tehsauce 5y ago
Another perspective, someone with no programming experience whatsoever today can in 1 minute create a new ML application for natural language tasks with performance that would blow most SOTA systems from 5 years ago out of the water, using things like the openai API. And this trend will almost certainly continue, where many tasks can be programmed by simply asking/describing the problem to a massive model and letting it work magic.
- lbriner 5y ago
  That might be true but my experience is that people are much better at building tooling than they are at using it properly. So many organisations can't even get the basics of efficiency and custom service right but they still invest heavily into cutting edge tech. I can imagine plenty of companies joining the ML bandwagon and still not even really knowing what they do as a business.
- dTal 5y ago
  This seems a trifle hyperbolic. No programming experience whatsoever? Remind me what the 'P' in 'API' stands for?
  - tehsauce 5y ago
    If one were to build with the API from scratch. But in theory all you really need to do to deploy a “new model” just is write a short prompt and click a “create endpoint” button.
vorpalhex 5y ago
I'm ok with this. ML should remain expensive and hard to use while we grasp how to regulate it and understand it. We need less ML, not more of it.
deepsun 5y ago
I've dived into ML (and DL) with 17 years of software development experience. I'd say it's much easier than software. Yes, there's A TON to learn and experiment with, but still much less than with software. I was able to feel confident enough after just 1.5 years learning and kaggling, and passed easily ML interviews to SF Bay Area companies (hint -- all data science people are extremely glad to see software experience, much more than data science). "Good pipeline and bad model is much better than bad pipeline and good model" (c) someone
- Eridrus 5y ago
  Beyond what makes a good model, IME at a FAANG building an ML product, the bulk of the work in practice tends to be general software engineering. There need to be a sufficient number of people who understand the actual ML pieces under the hood, but even when you're making changes to the models, the bulk of the actual work is not complex ideation, but implementing the ideas in software, and this implementation is usually something you can learn in a few months.
- monkeybutton 5y ago
  Bad pipeline means bad data, and bad data means bad model. Its as simple as that.
  - jacobr1 5y ago
    Bad pipeline is too strong. But good enough pipeline + good enough models > bad pipeline + great model
    - exdsq 5y ago
      My wife is a researcher at Stanford doing some ML stuff and the constant painpoint isn't the researching of novel models or maths or whatever, it's the IT hell of managing a pipeline and data acquisition. I've helped her introduce things like Docker to the lab which seems to have helped but still - software engineering is a totally different skill that really hinders ML work.
- 5y ago
swframe2 5y ago
In general, getting a phd is the best way to go but it is not the only way. "The AI Epiphany" channel by Aleksa Gordić is worth watching. Check out his origin story: https://www.youtube.com/watch?v=SgaN-4po_cA https://www.youtube.com/watch?v=SgaN-4po_cA He works at DeepMind. He is self taught; without a phd.
- alexcnwy 5y ago
  Aleksa is great. Also check out Jeremy Howard from fast.ai - also no PhD but amazing teacher and contributes actively to research. Chris Olah (Google Brain, OpenAI, etc) go to university at all. PhD definitely not required.
  - baobabKoodaa 5y ago
    You're heralding literal geniuses like Jeremy Howard as examples of "PhD not required". Yes, if you are Jeremy Howard or Leonardo DaVinci, sure you'll excel at this stuff without studying a degree.
  - xiphias2 5y ago
    I tried to build on top of fast.ai, and it was very easy to start, but all the hooks and magic in fast.ai 2 just made it extremely hard for me to understand and extend the code. I believe it went in a bad direction.
    - swframe2 5y ago
      When tensorflow was the dominate api, fast.ai made a lot of sense. Nowadays pytorch seems to have won and it is easy to use directly and there is PyTorch Lightning if you want things even easier to write. There are a several videos on youtube where the creator reviews a paper and then implements it from scratch. For example this channel is pretty good: https://www.youtube.com/c/AladdinPersson/playlists https://www.youtube.com/c/AladdinPersson/playlists
      - p1esk 5y ago
fxtentacle 5y ago
"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure that the pre-trained embedding won't distort your results. "For a software engineer, the hardest thing about developing Machine Learning functionality should be finding clean and representative ground-truth data, but it often isn’t." That is (in my opinion) an entirely bogus request. Machine learning is a mathematical / statistical tool for modeling large unknown functions. I feel like this sentence is akin in usefulness to: "For a nuclear power plant, the hardest thing about building one should be to draw how the finished building will look like in the press release". Someone "doing" machine learning without the requisite math knowledge is effectively driving blind. And worse than that, they don't even know what they don't see, because they lack the skills to identify their blind spots. That's how you end up with a "tank detection AI" that in reality just classifies the weather into bright vs. dark. [1] Companies like this who promise advanced mathematical algorithms with no prior skill or knowledge are how we unleash a plague of buggy unverified automatons upon the world. [1] https://www.lesswrong.com/posts/5o3CxyvZ2XKawRB5w/machine-learning-and-unintended-consequences https://www.lesswrong.com/posts/5o3CxyvZ2XKawRB5w/machine-le...
- glouwbug 5y ago
  Resume driven development is real. Who wants a crud app on their resume when they can have a crud + ML app on their resume? I remember back in like 2016 recruiters devoured anyone with the slightest bit of ML experience on their resumes: it fed back into the ego of developers, and suddenly everyone was an ML expert who could do no better than load a JSON of data and import keras. What a strange trip that time was
  - fxtentacle 5y ago
    I feel like we're seeing the next round of that now with all those "build your own advanced Deep Learning AI in 3 simple steps" websites. Clarifai, Amazon Rekognition, Google AutoML Vision, Nykel (the article here), Amazon Comprehend, Google AutoML Natural Language, MonkeyLearn, Lateral, BigML, Azure ML, Lobe, DataRobot, Rapidminer, Dataiku ... Did I forget anyone? EDIT: H2O’s Driverless AI, Floyd, AWS SageMaker, Databricks EDIT2: Pega Platform, MLFlow, Comet.ml EDIT3: $SNOW SnowFlake, Spell.ml, Cloudera ML, Alibaba Cloud
rememberlenny 5y ago
This is timely. I just compiled a few resources I found useful on groking computer vision and deep learning. Short list I’ve enjoyed recently: - PyImageSearch blog: https://pyimagesearch.com/blog/ https://pyimagesearch.com/blog/ - Fastai library and course: https://www.fast.ai/ https://www.fast.ai/ - Yan LeCun’s 2021 Spring NYU class: https://m.youtube.com/playlist?list=PLLHTzKZzVU9e6xUfG10TkTWApKSZCzuBI https://m.youtube.com/playlist?list=PLLHTzKZzVU9e6xUfG10TkTW...
saintarian 5y ago
Author of the blog post here - it's very cool to see this on HN! I wrote this as someone who considers himself a half-decent software engineer trying to use ML for a side project and feeling frustrated by all the effort and "accidental complexity" involved. Why focus on software engineers and ML in this post/rant/company? Because "software is eating the world" and having ML be more accessible to software engineers will broaden the range of problems they can solve. Thanks for all the comments - I acknowledge all/most of the criticisms as valid. A SaaS/AutoML solution won't work for everyone and definitely not for every problem, and it won't be the only answer to making ML more approachable.
jakupovic 5y ago
The hardest thing about ML is getting right data in right format. This is the same of pretty much any other program, but ML has a special place.
garyfirestorm 5y ago
lol I did an ML course in 2017. I am a mechanical engineer with some computer skills and managed to do lot of tensor flow models quite successfully. It’s funny to read that this could be hard for software guys.
surrTurr 5y ago
Nice read. However, it seems a bit fishy that the website publishing this article is a company offering ML as a service. They as a company directly profit from people not wanting to learn ML themselves. I‘m not saying they wrote this article to increase sales, but it‘s a thing to keep in mind.
sbeckeriv 5y ago
Can we not apply Machine Learning to teaching Machine Learning to software engineers?
surrTurr 5y ago
I‘d argue that integrating ML into a project isn‘t the hard problem, as it was described in this article. Most people will, after some research, be able to build some basic ML functionality using popular libraries and a bit of python. The hard thing about ML is, when you try to actually understand it. I‘ve met many „ML engineers“ who do their job but do not actually understand what they‘re doing. Understanding ML doesn‘t have to do a lot with coding. It‘s math and statistics.
Thaxll 5y ago
I view ML the same as crypto, a fad that will go away in the future and remains in certain nich realms.
- binarymax 5y ago
  ML has already added actual value to so many industries. While the AGI fantasy may never happen and we may even see another AI winter, the existing products and solutions that you use every day are powered by ML.
PLenz 5y ago
Physically training the models isn't the hard part. It knowing why you using x method, how to interpret the result, and what dials to turn that's the hard part. And there's no automating that.
torbTurret 5y ago
Amazon has some cool interactive articles that make it easy to understand some concepts, eg: https://mlu-explain.github.io/decision-tree/ https://mlu-explain.github.io/decision-tree/
Xcelerate 5y ago
Coming from a background in computational quantum chemistry, it’s interesting to see all of the people who say ML is “easy” after taking a few online courses and reading some books on data science. If it’s so easy, invent AGI then, since that is the holy grail of machine learning. Most of these people claiming expertise do not have a deep grasp of the mathematical fundamentals required for state of the art research in the field. Can you develop a neural network with features that are invariant to permutational and rotational symmetries? If so, how do you efficiently generate the irreproducible representations of the product of the symmetric and special orthogonal groups for use in a fast Fourier transform? What is minimum description length and why is it so fundamental? How do you solve trust-region problems on Riemannian manifolds? Throwing an off-the-shelf PyTorch library at a problem does not make one an expert in machine learning.
- uejfiweun 5y ago
  Not to disagree with your point, but your comment really reminded me of the following scene from Breaking Bad: https://www.youtube.com/watch?v=W_dxteeedgs https://www.youtube.com/watch?v=W_dxteeedgs
  - Xcelerate 5y ago
    Haha, I’d never seen that. I suppose my comment is a bit over-the-top compared to my comment history, but it’s somewhat of a touchy point for me when someone jumps into a new field and claims it’s easy and that they’re an expert without being able to back up the claim (some rare people can do this with a new field; most cannot).
    - antifa 5y ago
      Is it though? When I run the hello world for whatever GAN or pytorch, what exactly am I going to do if I get an unexpected error message? Or if the results are off? Would I even know the results are off? I don't even know if pytorch is used for GAN.
      - 5y ago
xavxav 5y ago
Sort of hijacking, but I've always wondered: Where are our 'neural binutils'? I want to be able to compose these tools like I would random unix ones: Something like 'Identify album covers in this image | extract the text in said covers | spotify api'. It seems like there are so many breakthrough models but both due to technical (size/compute) and industrial ($$$) concerns they remain out of reach for random devs, let alone packageable into a `grep` style composable tool.
- jldugger 5y ago
  > Where are our 'neural binutils'? Assuming you mean coreutils. binutils is for managing/inspecting binary executables. But to your point: there were two key innovations and criteria of UNIX pipelines: a common and understandable data format, and writing programs to send and receive anonymous data. Crucially, the input and output formats were the same: plain text, separated by newlines. In contrast neural networks are applied to a variety of data formats. Images, video, audio, text, social networks etc. each with their own encoding into something an NN can work with, with varying dimensions, features, metadata etc. So it doesn't make sense to bundle them as 'neural utils' but rather utils along whatever pipeline already exists, like GraphicsMagick. Which does leave a huge blind spot for the domain transforms like text recognition. If you stay within the AI ecosystem, you _can_ set up reusable layers for tensorFlow, but typically you cant swap out something in the middle without retraining all layers below it. Which you might treat as a violation of the anonymity criterion, since the behavior / performance of a layer relies on the specific behavior of those above it.
  - mprovost 5y ago
    There's still a lot of opportunity to introduce ML into the classic plain text Unix utilities. Even if you have to retrain, there's still room to improve over existing tools. One example is learned sort, which outperforms radix sort, even including the time it takes to train the model. https://blog.acolyer.org/2020/10/19/the-case-for-a-learned-sorting-algorithm/ https://blog.acolyer.org/2020/10/19/the-case-for-a-learned-s...
clircle 5y ago
No, this article is wrong on the first sentence. The hardest part of machine learning _is_ in fact curating quality data.
- disgruntledphd2 5y ago
  That's the essential complexity. The OP is correct that there's far more accidental complexity involved than seems strictly necessary. In an ideal world, I wouldn't have had to spend the last few years learning software engineering in order to be a better data scientist.
spaetzleesser 5y ago
I still remember when SQL was a dark science that could only be managed by administrators that were initiated in secret rooms. Nowadays a lot of people use SQL without deep understanding. They do useful stuff but if you are very skilled you can do way more. And SQL experts are often frustrated with all the amateurs that use their database in such a sub-optimal way. I expect the same for ML. The tooling will improve until regular developers can use it without understanding the fundamentals. They will do useful things but some skilled people will be able to do way more advanced stuff. That's pretty much the path of all technologies. They get simplified to a level where they useful for a lot of people but some experts will be able to get way more out it. And usually the experts look down on the amateurs.
- 317070 5y ago
  As ML expert, I am often frustrated at the sub-optimal things I find in the wild or even publications. But you need to understand everyone is on a growing trajectory.
  - spaetzleesser 5y ago
    And there is also the concept of "good enough".
monksy 5y ago
Machine learning isn't for SWEs. It's for data scienctists and in a limited sense of Data Engineers.
binarymax 5y ago
This line "Set up low-latency, elastic, highly-available, and cost-effective inference close to your data." is the problem I've found that annoys everyone - Software Eng don't understand MLOps, Data Scientists don't understand systems programming, and everything ends up costing way too much money and taking way too much time. Even with a magic API the latency still isnt good enough, so the choice is often entire ML solutions like the OPs product, months of development time, or a really expensive container. Shameless plug since my Show HN went completely ignored earlier today ;) ...but that's why i built this: https://news.ycombinator.com/item?id=30428664 https://news.ycombinator.com/item?id=30428664
master_yoda_1 5y ago
OP first define which software engineers you are talking about. Otherwise stop bullshitting just to market your company?
modernpink 5y ago
Most CS grads coming onto the engineer market now will have ML exposure through their chosen college courses. As this wave of knowledge makes its way through the industry, the value of ML specialist knowledge will decline, especially as off-the-shelf pre-trained models improve. Very few companies then will be able to justify the luxury of a dedicated in-house data science/ML engineering team. In other words, most software engineers will be ML engineers, in the same way most software engineers today are Docker/cloud proficient engineers.
- LAC-Tech 5y ago
  > in the same way most software engineers today are Docker/cloud proficient engineers oh shit, are we all supposed to be now?
  - exe163 5y ago
    Docker is a tool where one reads the documentation and be able to use it. ML, much like math, is a discipline. Sure you can read textbooks and the latest papers. But deep understanding and experience are what make it useful. Sadly these will take much longer.
bob1029 5y ago
We made an experimental foray into the ML space in attempts to accelerate authoring of SQL queries. After about a week of reading through literature and playing with open ai, it became pretty obvious we were still super far away from being able to build something the business could actually find value in. My problem scales horribly with the ML training angle we have today, because it's the super complex one-off queries we need the most help with, not the simple ones we can anticipate and train against. What we need is actual intelligence for many of our problems. Things like subjective criteria are important to us. Realizing maybe a recursive query is a fair compromise to reduce a 400 line monster to 30 lines. Assuming the 400 "looks nasty", that is. I guess you could train that bit too, but then your solution space gets even more impossible to target.
lysecret 5y ago
So, I went directly into data science after an econ degree, worked there for 2 years and then transitioned to SWE (at startups). First, I am 100% certain ML will become a part of the standard SWE toolkit (just like apis, docker, sql, etc..). However, to the relative "hardness" I would say ML currently is much less things but they can be really hard to get your head around (like starting to think in embeddings and vectors), and many APIs are wonky, because they are new. And the whole data science workflow space it in an early spot (we are still missing a "create-data-app", simmilar to the create-react-app.
- antisthenes 5y ago
  Your career path seems really interesting. Do you have a blog or some kind of work showcase where we can follow your journey and work?
  - lysecret 5y ago
    happy to hear that. I fell in the trap of "to write a blog i have to code it myself haha" but il get back to you when its public:)
- borroka 5y ago
  Standard ML applications will be commoditized in the same way that creating a website has been commoditized. But for anything that isn't standard, and that's the vast majority of what businesses need, ML isn't going to become part of SWE's standard toolkit anytime soon. ML is kind of like tennis in the sense that you look at Nadal and Federer and all the greats and you can say, "man, I could do that," but you can't, not even close in this life and the next after this one. In fact, most SWEs who have developed some sort of "intuition" about ML have been quite dangerous, as they tend to be condescending to the real experts, have built things that make no sense at all or fall apart when 1 data point our of 100,000 changes, and when presented with the fact that they have no clue about ML, resort to the "the AI is all hype anyway" comeback. And vice versa (ML practitioners like me who think they can do SWE with one eye closed, "what's the big deal?") is also true.
dhairya 5y ago
The challenge for the engineers at our AI startup is that deterministic testing paradigms don't adapt well to probabilistic models that are continually being retrained. As a scientist is hard to convey the acceptable range of variance and often the random change of individual predictions at the decision boundary. It's also hard debug behavioral issues that actually are systematic model failure versus those that are traditional infrastructure bugs. Often times the band aid is that build lookup tables to ensure certain behavior which in turn also underlying issues from being discovered. Testing paradigms are either too high level or too specific. Recent work on evolving behavioral tests addresses this but it requires more manual effort and interpretation which kinda defeats to point of automated tests.
- deepsquirrelnet 5y ago
  Good discussion. I have often wondered about that interface. This bears more resemblance to traditional manufacturing actually. I think there may be some value in borrowing ideas from statistical process control, rather than trying to force predictions into deterministic cases.
beijbom 5y ago
I'm the ML guy at Nyckel. Great to see all thoughts and comments! Feel free to AMA.
- htrp 5y ago
  Can you give a succinct summary of what makes you guys different from datarobot etc? In general, I feel like ML platforms have the problem outlined by https://xkcd.com/927/ https://xkcd.com/927/ (Standards)
  - beijbom 5y ago
    Good question. We have been doing this for almost 2 years now and we still find new players almost every week! It's a bit of a wild west for sure. I can't say what we do different from everyone, but a few things that we focus on: * Speed: we train models based on DL in seconds. So you get real-time feedback on your model/data as you annotate and upload more. This is true for a few, but far from all of our competitors. In our benchmarking we find that we still perform on par with the competition (at least in the "low-data" regime https://www.nyckel.com/blog/automl-benchmark-nyckel-google-huggingface/ https://www.nyckel.com/blog/automl-benchmark-nyckel-google-h...) * Level of abstraction: Many competitors expose some ML knobs for their users thinking it will improve the experience. We found that this induces "ML anxiety" for many. As a result we have zero knobs. Just focus on your data, we do the rest. * API: we have spend a ton of time developing clean API abstractions. Some competitors have great APIs, other don't. * Cost: we are super cheap. Our lowest tier if $50. We don't charge for training or per function/model.
nnoitra 5y ago
Become better instead of whining.
faangiq 5y ago
Look it’s just buzzwords all the way down. There’s only a handful of people who can do this stuff. {Your business} will drop some buzzwords, someone will write an if statement, management high fives all around and fat bonuses. Deal with it.
joelbondurant1 5y ago
henning 5y ago
> basic like image classification Yes, it is going to be laborious to have state-of-the-art deep learning implemented in your infrastructure.
danielmorozoff 5y ago
It seems to me the biggest challenge for SE transitioning into ML is that ML is a very broad topic and people conflate a lot of roles together. From purely research based questions (backbones, optimizers, initializers etc), to more 'MLOps' like pipelining questions, which tend to fall into the classical engineering / dev ops buckets. So the real question is what type of ML do you want to do? If you're looking to land a job at FAIR / Deepmind or Google Brain/ Nvidia Research as a researcher or ML scientist the expectations of knowledge are very different than 'data science'. These are research lab groups, that work on pushing the state of the art forward. They are also supported by great engineers, building awesome tools that improve ML research. So transitioning into this sort of role requires more than doing Kaggle competitions, it requires developing an intuition for the respective ML subfield / and trying new things and usually failing. i.e. this is a research role and will require a lot of study and learning If on the other hand you are looking for datascience / take model and build pipeline to run AI, or perform hyper param sweeps or simply modify some model code, then on I would say that is much more engineering than research ML. This has a much lower barrier to entry coming from engineering and could be a good stepping stone to a transition into pure ML research. On a more general note to consider when thinking of transitioning to ML is that these systems are probabilistic in nature vs purely deterministic as they are in more general software systems. People (ie humans) are bad at wrapping their heads around distributional processes - you can see this in all fields that deal with them (Quantum vs Classical Physics, Biological Systems etc). In general I guess what I have seen is when engineers try to dip their toes into ML, what's required is a mindset shift in how to approach problems. Once that happens the depth of that shift determines the type of role with ML you wish to pursue.
- deepsquirrelnet 5y ago
  It's interesting that you mention this, because there's quite an impressive resurgence of privately funded R&D going on in the ML space. We're in an interesting phase, where the field is moving too fast to have 'canonical' methodologies (though we're getting close). To be an engineer in the deep learning space often requires reading and keeping up with research. Everything just gets dumped on arXiv, because the peer-reviewed publication cycle is almost too slow for the field. Making a successful transition to ML in my opinion, depends a lot on the individual. Without a strong background in calculus, linear algebra and statistics, it's going to be difficult. Training a model is what people tend to focus on, but in my opinion, that's the easy part. Evaluating/validating a model, analyzing and preparing your data, anticipating model performance, understanding what to do to improve your fit, model selection or architecture. Developing custom deep learning architectures at times requires a bit of an abstract mathematical intuition that I think will suit many engineers very well. A lot of engineers are well equipped to be successful in making a transition, but on the other hand, at least as many aren't. In the future, I think the field will have many varying degrees of expertise, with the barriers to entry becoming lower all the time. We're reaching a point where some common use cases can be solved adequately in a nearly automated fashion. Some "autoML" tools don't really require any real understanding of ML, though I think it's not wise to get in the habit of using them without understanding how to evaluate a fit. These tools will be great for people who want to occasionally use ML to solve some smaller problems, but as a part of their larger job function. In some middle area, ML engineers and practitioners will be training and operationalizing models, and keeping up with major developments in research. But there will be some significant changes in the next decade. I predict the nebulous of data science, data analysis and machine learning will become formalized into 3 major skills - exploratory data analysis, machine learning and advanced computational statistics. At the lowest level, researchers will continue developing the field, which like you say, is probably not something you transition directly into.
bgroat 5y ago
Maybe I'm alone here... but the biggest hurdle I have with ML as a software engineer is the python ecosystem
Khelavaster 5y ago
This is myopic...ML.Net takes care of these issues.

178 points by gk1 5y ago

The article contains lists with bullet items and then suggests that if these were "easy", ML would be more accessible. Well true, but much of the "difficulty" is in establishing faith that the models are meaningful and valuable. The article is talking about applications where the value is less-or-equal to the work of learning and applying some library (it mentions days or weeks). What is the actual value of applying ML in these cases?

kellymore 5y ago

I think it goes without saying that this is too relative. One thing that is common in both fields is the level of research and reading docs it entails. I used to tell my students "software engineering is 70% reading and 30% coding". This remained consistent as I dove into Data Science and now ML with computer vision at Roboflow. Of course the first time I was exposed to it during a fellowship, I thought I was out of my depths, but this comes with everything new. To @deepsun point, I've found Kaggle's intro courses quite excellent as well.

Irishsteve 5y ago

I'm doing ML a long time, practice makes perfect etc. If anything is too hard, it is front end.

marcinzm 5y ago
Yeah, it felt like a sales person being confused why can't become an expert fullstack software engineer on up to date modern stacks without a lot of effort and time.
tiborsaas 5y ago
As a senior frontend engineer, practice makes perfect, frontend is easy-peasy. ML/AI seems to have this thick wall of math around it, if I really want to understand what makes the models going.

rchaves 5y ago

I really like the premise, and I really agree with it, but I don't think a SaaS is a solution, the solution is trying to find better abstractions that makes things simple and easier for the developers, using code, and without limiting their flexibility, but that's the hardest thing to do! As an experienced software developer who used to learn a new framework every week, I thought ML was going to be piece of cake. In reality, I went down this rabbit hole 4 years ago, and I'm still in there, I was so innocent back then

oblak 5y ago
Good points. But truth is SaaS can be a great solution if you pick Nyckel! Sorry, it had to be done.
lbriner 5y ago
The issue with abstractions is that ML isn't just a few variables or dissimilar systems to choose from, the whole problem domain that could be helped with ML has so many dimensions. Natural language, speech processing, pattern recognition etc. are all completely different. The only thing they have in common is that you might use computers for them. I think that is why people find ML so hard. It isn't a single sausage machine to create insight from, it is a set of entire stacks from philosophy all the way down to the electronics.

version_five 5y ago

I'd argue software engineering is still too hard for ML engineers. Most of ML (not research but commercial applications of what's already been demonstrated) is now well within the realm of engineering, but there are few standard practices, bodies of knowledge, or agreed on processes for doing anything. These are problems that engineering formalism solve, not another auto-ml tool. Maybe I'm saying the same thing as the article, just framed differently?

yldedly 5y ago
In my experience, both are true. I'm more on the ML side, and I can tell I don't have the kind of routine and habits that good software engineers have, though I'm learning. But on the other hand, and I've seen this from software engineers who've made the transition to ML, and clearly have a good handle on the concepts (in one case even published papers in ML journals), they don't seem to have the intuition that allows them to select the right tool for the job, or understand when a given technique is appropriate; you could call that "modeling maturity" - a combination of mathematical maturity and the skill of relating domain knowledge to modeling choices.
- antman 5y ago
  Absolutely agree, the major challenge has been to explain to software engineers that the pick up an algo from github, monkey see monkey do a tutorial followed by automl or grid search is pretty irrelevant nowadays. If you can’t implement any existing algorithm from a scientific paper it is pretty hard to understand the flexibility that is required designing a training pipeline. In terms of tech stacks, cloud providers’ sample architectures and code samples are a base for cloud components and API understanding and not actual implementations.
- 7h3kk1d 5y ago
  I think the above commenter is more critiquing the notion of relying on intuition all together.
  - yldedly 5y ago

stevenalowe 5y ago

So is software engineering

etangent 5y ago

One might as well write an article claiming that "UI design is still too hard for software engineers" or "controlling a nuclear power plant is still too hard for software engineers" -- which are true (and it is equally true that software engineering is hard for UI designers and nuclear power plant operators). Who came up with this silly idea that something that is a valid knowledge domain of its own is suddenly going to become "easy"?

zwieback 5y ago

Missing in these advertisements-disguised-as-blogs is an estimate of effort or time. Let's say ML is critical to a new product or internal tool, how many man-years is it reasonable to invest? If your expectation is using ML like a drag-and-drop app builder then you're probably best off using a canned tool but then you won't really have any competitive advantage.

ramesh31 5y ago

Can confirm. Data science team just schooled the entire engineering department in our Deep Racer tournament.

exdsq 5y ago
Makes sense though, right? Data team did data stuff better. If you had a different task with an engineering focus you'd see the opposite.
fxtentacle 5y ago
In the gocoder Bomberland competition, my DL AI trained on 1.5 billion timesteps just lost against someone handcrafting algorithms in Python in one day. You're absolutely right, sometimes, simple and predictable solutions are much better than AI magic ^^

deleted 5y ago

[deleted]

uoaei 5y ago

Wow, it's almost like machine learning and software engineering are different disciplines entirely! Just because both involve coding doesn't mean software engineers should be expected to have the math chops (stats/prob, linalg, calc, etc.) to make machine learning work for them... Vice versa is a little more complicated, because ML/DS can be done very inefficiently without the proper coding practices, but understanding the math is independent of that so that point still holds for this comparison.

jstx1 5y ago
> machine learning and software engineering are different disciplines entirely I don't think so. Or more precisely, they might look different academically but you need to have both as a skill to build something useful.
- uoaei 5y ago
  In-built is the assumption that one person must have both competencies. This may be true for cash-strapped startups but this hardly plays well as general advice.
  - jstx1 5y ago
    It's not an assumption, that's my main point - you need both skills in the same person. And it's true everywhere including in the biggest companies.
    - ngc248 5y ago
      Definitely not ... I am a SWE in a big company and working on ML projects for the past 4 years. We have a data scientist team which does the data exploration and comes up with a model. Their output would be a jupyter notebook. Then we have a team of "Applied ML practitioners", which I am a part of, we productionize the jupyter notebook, by setting up pipelines, services etc. We understand ML algos, stats, probability etc, but not as much as our data scientist team does. Having both in the same person would be good, but is not necessary.

howmayiannoyyou 5y ago

The best tutorials for indoctrinating software engineers in ML are written accessibly & use real-world business-case data, or easily comprehended data sets. One great example of this, from graph theory, was this fantastic 2013 article by Kieran Healy: https://kieranhealy.org/blog/archives/2013/06/09/using-metadata-to-find-paul-revere/ https://kieranhealy.org/blog/archives/2013/06/09/using-metad... There's no getting around the complexities of fit, bias and customized models for many ML problems, so my observation above is obviously limited in its applicability.

tehsauce 5y ago

Another perspective, someone with no programming experience whatsoever today can in 1 minute create a new ML application for natural language tasks with performance that would blow most SOTA systems from 5 years ago out of the water, using things like the openai API. And this trend will almost certainly continue, where many tasks can be programmed by simply asking/describing the problem to a massive model and letting it work magic.

lbriner 5y ago
That might be true but my experience is that people are much better at building tooling than they are at using it properly. So many organisations can't even get the basics of efficiency and custom service right but they still invest heavily into cutting edge tech. I can imagine plenty of companies joining the ML bandwagon and still not even really knowing what they do as a business.
dTal 5y ago
This seems a trifle hyperbolic. No programming experience whatsoever? Remind me what the 'P' in 'API' stands for?
- tehsauce 5y ago
  If one were to build with the API from scratch. But in theory all you really need to do to deploy a “new model” just is write a short prompt and click a “create endpoint” button.

vorpalhex 5y ago

I'm ok with this. ML should remain expensive and hard to use while we grasp how to regulate it and understand it. We need less ML, not more of it.

deepsun 5y ago

I've dived into ML (and DL) with 17 years of software development experience. I'd say it's much easier than software. Yes, there's A TON to learn and experiment with, but still much less than with software. I was able to feel confident enough after just 1.5 years learning and kaggling, and passed easily ML interviews to SF Bay Area companies (hint -- all data science people are extremely glad to see software experience, much more than data science). "Good pipeline and bad model is much better than bad pipeline and good model" (c) someone

Eridrus 5y ago
Beyond what makes a good model, IME at a FAANG building an ML product, the bulk of the work in practice tends to be general software engineering. There need to be a sufficient number of people who understand the actual ML pieces under the hood, but even when you're making changes to the models, the bulk of the actual work is not complex ideation, but implementing the ideas in software, and this implementation is usually something you can learn in a few months.
monkeybutton 5y ago
Bad pipeline means bad data, and bad data means bad model. Its as simple as that.
- jacobr1 5y ago
  Bad pipeline is too strong. But good enough pipeline + good enough models > bad pipeline + great model
  - exdsq 5y ago
    My wife is a researcher at Stanford doing some ML stuff and the constant painpoint isn't the researching of novel models or maths or whatever, it's the IT hell of managing a pipeline and data acquisition. I've helped her introduce things like Docker to the lab which seems to have helped but still - software engineering is a totally different skill that really hinders ML work.
5y ago

swframe2 5y ago

In general, getting a phd is the best way to go but it is not the only way. "The AI Epiphany" channel by Aleksa Gordić is worth watching. Check out his origin story: https://www.youtube.com/watch?v=SgaN-4po_cA https://www.youtube.com/watch?v=SgaN-4po_cA He works at DeepMind. He is self taught; without a phd.

alexcnwy 5y ago
Aleksa is great. Also check out Jeremy Howard from fast.ai - also no PhD but amazing teacher and contributes actively to research. Chris Olah (Google Brain, OpenAI, etc) go to university at all. PhD definitely not required.
- baobabKoodaa 5y ago
  You're heralding literal geniuses like Jeremy Howard as examples of "PhD not required". Yes, if you are Jeremy Howard or Leonardo DaVinci, sure you'll excel at this stuff without studying a degree.
- xiphias2 5y ago
  I tried to build on top of fast.ai, and it was very easy to start, but all the hooks and magic in fast.ai 2 just made it extremely hard for me to understand and extend the code. I believe it went in a bad direction.
  - swframe2 5y ago
    When tensorflow was the dominate api, fast.ai made a lot of sense. Nowadays pytorch seems to have won and it is easy to use directly and there is PyTorch Lightning if you want things even easier to write. There are a several videos on youtube where the creator reviews a paper and then implements it from scratch. For example this channel is pretty good: https://www.youtube.com/c/AladdinPersson/playlists https://www.youtube.com/c/AladdinPersson/playlists
    - p1esk 5y ago

fxtentacle 5y ago

"Machine Learning is too difficult" ... says company selling "ML platform [which] can be used by anyone and it only takes minutes to train your first model." No, actually, you're just being dishonest. Even if you hide TensorFlow and the keras models behind a nice GUI, people still need that mathematics knowledge to succeed. And yes, pre-training is great. But you need a shitload of stochastic analysis to make sure that the pre-trained embedding won't distort your results. "For a software engineer, the hardest thing about developing Machine Learning functionality should be finding clean and representative ground-truth data, but it often isn’t." That is (in my opinion) an entirely bogus request. Machine learning is a mathematical / statistical tool for modeling large unknown functions. I feel like this sentence is akin in usefulness to: "For a nuclear power plant, the hardest thing about building one should be to draw how the finished building will look like in the press release". Someone "doing" machine learning without the requisite math knowledge is effectively driving blind. And worse than that, they don't even know what they don't see, because they lack the skills to identify their blind spots. That's how you end up with a "tank detection AI" that in reality just classifies the weather into bright vs. dark. [1] Companies like this who promise advanced mathematical algorithms with no prior skill or knowledge are how we unleash a plague of buggy unverified automatons upon the world. [1] https://www.lesswrong.com/posts/5o3CxyvZ2XKawRB5w/machine-learning-and-unintended-consequences https://www.lesswrong.com/posts/5o3CxyvZ2XKawRB5w/machine-le...

glouwbug 5y ago
Resume driven development is real. Who wants a crud app on their resume when they can have a crud + ML app on their resume? I remember back in like 2016 recruiters devoured anyone with the slightest bit of ML experience on their resumes: it fed back into the ego of developers, and suddenly everyone was an ML expert who could do no better than load a JSON of data and import keras. What a strange trip that time was
- fxtentacle 5y ago
  I feel like we're seeing the next round of that now with all those "build your own advanced Deep Learning AI in 3 simple steps" websites. Clarifai, Amazon Rekognition, Google AutoML Vision, Nykel (the article here), Amazon Comprehend, Google AutoML Natural Language, MonkeyLearn, Lateral, BigML, Azure ML, Lobe, DataRobot, Rapidminer, Dataiku ... Did I forget anyone? EDIT: H2O’s Driverless AI, Floyd, AWS SageMaker, Databricks EDIT2: Pega Platform, MLFlow, Comet.ml EDIT3: $SNOW SnowFlake, Spell.ml, Cloudera ML, Alibaba Cloud

rememberlenny 5y ago

This is timely. I just compiled a few resources I found useful on groking computer vision and deep learning. Short list I’ve enjoyed recently: - PyImageSearch blog: https://pyimagesearch.com/blog/ https://pyimagesearch.com/blog/ - Fastai library and course: https://www.fast.ai/ https://www.fast.ai/ - Yan LeCun’s 2021 Spring NYU class: https://m.youtube.com/playlist?list=PLLHTzKZzVU9e6xUfG10TkTWApKSZCzuBI https://m.youtube.com/playlist?list=PLLHTzKZzVU9e6xUfG10TkTW...

saintarian 5y ago

Author of the blog post here - it's very cool to see this on HN! I wrote this as someone who considers himself a half-decent software engineer trying to use ML for a side project and feeling frustrated by all the effort and "accidental complexity" involved. Why focus on software engineers and ML in this post/rant/company? Because "software is eating the world" and having ML be more accessible to software engineers will broaden the range of problems they can solve. Thanks for all the comments - I acknowledge all/most of the criticisms as valid. A SaaS/AutoML solution won't work for everyone and definitely not for every problem, and it won't be the only answer to making ML more approachable.

jakupovic 5y ago

The hardest thing about ML is getting right data in right format. This is the same of pretty much any other program, but ML has a special place.

garyfirestorm 5y ago

lol I did an ML course in 2017. I am a mechanical engineer with some computer skills and managed to do lot of tensor flow models quite successfully. It’s funny to read that this could be hard for software guys.

surrTurr 5y ago

Nice read. However, it seems a bit fishy that the website publishing this article is a company offering ML as a service. They as a company directly profit from people not wanting to learn ML themselves. I‘m not saying they wrote this article to increase sales, but it‘s a thing to keep in mind.

sbeckeriv 5y ago

Can we not apply Machine Learning to teaching Machine Learning to software engineers?

surrTurr 5y ago

I‘d argue that integrating ML into a project isn‘t the hard problem, as it was described in this article. Most people will, after some research, be able to build some basic ML functionality using popular libraries and a bit of python. The hard thing about ML is, when you try to actually understand it. I‘ve met many „ML engineers“ who do their job but do not actually understand what they‘re doing. Understanding ML doesn‘t have to do a lot with coding. It‘s math and statistics.

Thaxll 5y ago

I view ML the same as crypto, a fad that will go away in the future and remains in certain nich realms.

binarymax 5y ago
ML has already added actual value to so many industries. While the AGI fantasy may never happen and we may even see another AI winter, the existing products and solutions that you use every day are powered by ML.

PLenz 5y ago

Physically training the models isn't the hard part. It knowing why you using x method, how to interpret the result, and what dials to turn that's the hard part. And there's no automating that.

torbTurret 5y ago

Amazon has some cool interactive articles that make it easy to understand some concepts, eg: https://mlu-explain.github.io/decision-tree/ https://mlu-explain.github.io/decision-tree/

Xcelerate 5y ago

Coming from a background in computational quantum chemistry, it’s interesting to see all of the people who say ML is “easy” after taking a few online courses and reading some books on data science. If it’s so easy, invent AGI then, since that is the holy grail of machine learning. Most of these people claiming expertise do not have a deep grasp of the mathematical fundamentals required for state of the art research in the field. Can you develop a neural network with features that are invariant to permutational and rotational symmetries? If so, how do you efficiently generate the irreproducible representations of the product of the symmetric and special orthogonal groups for use in a fast Fourier transform? What is minimum description length and why is it so fundamental? How do you solve trust-region problems on Riemannian manifolds? Throwing an off-the-shelf PyTorch library at a problem does not make one an expert in machine learning.

uejfiweun 5y ago
Not to disagree with your point, but your comment really reminded me of the following scene from Breaking Bad: https://www.youtube.com/watch?v=W_dxteeedgs https://www.youtube.com/watch?v=W_dxteeedgs
- Xcelerate 5y ago
  Haha, I’d never seen that. I suppose my comment is a bit over-the-top compared to my comment history, but it’s somewhat of a touchy point for me when someone jumps into a new field and claims it’s easy and that they’re an expert without being able to back up the claim (some rare people can do this with a new field; most cannot).
  - antifa 5y ago
    Is it though? When I run the hello world for whatever GAN or pytorch, what exactly am I going to do if I get an unexpected error message? Or if the results are off? Would I even know the results are off? I don't even know if pytorch is used for GAN.
    - 5y ago

xavxav 5y ago

Sort of hijacking, but I've always wondered: Where are our 'neural binutils'? I want to be able to compose these tools like I would random unix ones: Something like 'Identify album covers in this image | extract the text in said covers | spotify api'. It seems like there are so many breakthrough models but both due to technical (size/compute) and industrial ($$$) concerns they remain out of reach for random devs, let alone packageable into a `grep` style composable tool.

jldugger 5y ago
> Where are our 'neural binutils'? Assuming you mean coreutils. binutils is for managing/inspecting binary executables. But to your point: there were two key innovations and criteria of UNIX pipelines: a common and understandable data format, and writing programs to send and receive anonymous data. Crucially, the input and output formats were the same: plain text, separated by newlines. In contrast neural networks are applied to a variety of data formats. Images, video, audio, text, social networks etc. each with their own encoding into something an NN can work with, with varying dimensions, features, metadata etc. So it doesn't make sense to bundle them as 'neural utils' but rather utils along whatever pipeline already exists, like GraphicsMagick. Which does leave a huge blind spot for the domain transforms like text recognition. If you stay within the AI ecosystem, you _can_ set up reusable layers for tensorFlow, but typically you cant swap out something in the middle without retraining all layers below it. Which you might treat as a violation of the anonymity criterion, since the behavior / performance of a layer relies on the specific behavior of those above it.
- mprovost 5y ago
  There's still a lot of opportunity to introduce ML into the classic plain text Unix utilities. Even if you have to retrain, there's still room to improve over existing tools. One example is learned sort, which outperforms radix sort, even including the time it takes to train the model. https://blog.acolyer.org/2020/10/19/the-case-for-a-learned-sorting-algorithm/ https://blog.acolyer.org/2020/10/19/the-case-for-a-learned-s...

clircle 5y ago

No, this article is wrong on the first sentence. The hardest part of machine learning _is_ in fact curating quality data.

disgruntledphd2 5y ago
That's the essential complexity. The OP is correct that there's far more accidental complexity involved than seems strictly necessary. In an ideal world, I wouldn't have had to spend the last few years learning software engineering in order to be a better data scientist.

spaetzleesser 5y ago

I still remember when SQL was a dark science that could only be managed by administrators that were initiated in secret rooms. Nowadays a lot of people use SQL without deep understanding. They do useful stuff but if you are very skilled you can do way more. And SQL experts are often frustrated with all the amateurs that use their database in such a sub-optimal way. I expect the same for ML. The tooling will improve until regular developers can use it without understanding the fundamentals. They will do useful things but some skilled people will be able to do way more advanced stuff. That's pretty much the path of all technologies. They get simplified to a level where they useful for a lot of people but some experts will be able to get way more out it. And usually the experts look down on the amateurs.

317070 5y ago
As ML expert, I am often frustrated at the sub-optimal things I find in the wild or even publications. But you need to understand everyone is on a growing trajectory.
- spaetzleesser 5y ago
  And there is also the concept of "good enough".

monksy 5y ago

Machine learning isn't for SWEs. It's for data scienctists and in a limited sense of Data Engineers.

binarymax 5y ago

This line "Set up low-latency, elastic, highly-available, and cost-effective inference close to your data." is the problem I've found that annoys everyone - Software Eng don't understand MLOps, Data Scientists don't understand systems programming, and everything ends up costing way too much money and taking way too much time. Even with a magic API the latency still isnt good enough, so the choice is often entire ML solutions like the OPs product, months of development time, or a really expensive container. Shameless plug since my Show HN went completely ignored earlier today ;) ...but that's why i built this: https://news.ycombinator.com/item?id=30428664 https://news.ycombinator.com/item?id=30428664

master_yoda_1 5y ago

OP first define which software engineers you are talking about. Otherwise stop bullshitting just to market your company?

modernpink 5y ago

Most CS grads coming onto the engineer market now will have ML exposure through their chosen college courses. As this wave of knowledge makes its way through the industry, the value of ML specialist knowledge will decline, especially as off-the-shelf pre-trained models improve. Very few companies then will be able to justify the luxury of a dedicated in-house data science/ML engineering team. In other words, most software engineers will be ML engineers, in the same way most software engineers today are Docker/cloud proficient engineers.

LAC-Tech 5y ago
> in the same way most software engineers today are Docker/cloud proficient engineers oh shit, are we all supposed to be now?
- exe163 5y ago
  Docker is a tool where one reads the documentation and be able to use it. ML, much like math, is a discipline. Sure you can read textbooks and the latest papers. But deep understanding and experience are what make it useful. Sadly these will take much longer.

bob1029 5y ago

We made an experimental foray into the ML space in attempts to accelerate authoring of SQL queries. After about a week of reading through literature and playing with open ai, it became pretty obvious we were still super far away from being able to build something the business could actually find value in. My problem scales horribly with the ML training angle we have today, because it's the super complex one-off queries we need the most help with, not the simple ones we can anticipate and train against. What we need is actual intelligence for many of our problems. Things like subjective criteria are important to us. Realizing maybe a recursive query is a fair compromise to reduce a 400 line monster to 30 lines. Assuming the 400 "looks nasty", that is. I guess you could train that bit too, but then your solution space gets even more impossible to target.

lysecret 5y ago

So, I went directly into data science after an econ degree, worked there for 2 years and then transitioned to SWE (at startups). First, I am 100% certain ML will become a part of the standard SWE toolkit (just like apis, docker, sql, etc..). However, to the relative "hardness" I would say ML currently is much less things but they can be really hard to get your head around (like starting to think in embeddings and vectors), and many APIs are wonky, because they are new. And the whole data science workflow space it in an early spot (we are still missing a "create-data-app", simmilar to the create-react-app.

antisthenes 5y ago
Your career path seems really interesting. Do you have a blog or some kind of work showcase where we can follow your journey and work?
- lysecret 5y ago
  happy to hear that. I fell in the trap of "to write a blog i have to code it myself haha" but il get back to you when its public:)
borroka 5y ago
Standard ML applications will be commoditized in the same way that creating a website has been commoditized. But for anything that isn't standard, and that's the vast majority of what businesses need, ML isn't going to become part of SWE's standard toolkit anytime soon. ML is kind of like tennis in the sense that you look at Nadal and Federer and all the greats and you can say, "man, I could do that," but you can't, not even close in this life and the next after this one. In fact, most SWEs who have developed some sort of "intuition" about ML have been quite dangerous, as they tend to be condescending to the real experts, have built things that make no sense at all or fall apart when 1 data point our of 100,000 changes, and when presented with the fact that they have no clue about ML, resort to the "the AI is all hype anyway" comeback. And vice versa (ML practitioners like me who think they can do SWE with one eye closed, "what's the big deal?") is also true.

dhairya 5y ago

The challenge for the engineers at our AI startup is that deterministic testing paradigms don't adapt well to probabilistic models that are continually being retrained. As a scientist is hard to convey the acceptable range of variance and often the random change of individual predictions at the decision boundary. It's also hard debug behavioral issues that actually are systematic model failure versus those that are traditional infrastructure bugs. Often times the band aid is that build lookup tables to ensure certain behavior which in turn also underlying issues from being discovered. Testing paradigms are either too high level or too specific. Recent work on evolving behavioral tests addresses this but it requires more manual effort and interpretation which kinda defeats to point of automated tests.

deepsquirrelnet 5y ago
Good discussion. I have often wondered about that interface. This bears more resemblance to traditional manufacturing actually. I think there may be some value in borrowing ideas from statistical process control, rather than trying to force predictions into deterministic cases.

beijbom 5y ago

I'm the ML guy at Nyckel. Great to see all thoughts and comments! Feel free to AMA.

htrp 5y ago
Can you give a succinct summary of what makes you guys different from datarobot etc? In general, I feel like ML platforms have the problem outlined by https://xkcd.com/927/ https://xkcd.com/927/ (Standards)
- beijbom 5y ago
  Good question. We have been doing this for almost 2 years now and we still find new players almost every week! It's a bit of a wild west for sure. I can't say what we do different from everyone, but a few things that we focus on: * Speed: we train models based on DL in seconds. So you get real-time feedback on your model/data as you annotate and upload more. This is true for a few, but far from all of our competitors. In our benchmarking we find that we still perform on par with the competition (at least in the "low-data" regime https://www.nyckel.com/blog/automl-benchmark-nyckel-google-huggingface/ https://www.nyckel.com/blog/automl-benchmark-nyckel-google-h...) * Level of abstraction: Many competitors expose some ML knobs for their users thinking it will improve the experience. We found that this induces "ML anxiety" for many. As a result we have zero knobs. Just focus on your data, we do the rest. * API: we have spend a ton of time developing clean API abstractions. Some competitors have great APIs, other don't. * Cost: we are super cheap. Our lowest tier if $50. We don't charge for training or per function/model.

nnoitra 5y ago

Become better instead of whining.

faangiq 5y ago

Look it’s just buzzwords all the way down. There’s only a handful of people who can do this stuff. {Your business} will drop some buzzwords, someone will write an if statement, management high fives all around and fat bonuses. Deal with it.

joelbondurant1 5y ago

henning 5y ago

> basic like image classification Yes, it is going to be laborious to have state-of-the-art deep learning implemented in your infrastructure.

danielmorozoff 5y ago

It seems to me the biggest challenge for SE transitioning into ML is that ML is a very broad topic and people conflate a lot of roles together. From purely research based questions (backbones, optimizers, initializers etc), to more 'MLOps' like pipelining questions, which tend to fall into the classical engineering / dev ops buckets. So the real question is what type of ML do you want to do? If you're looking to land a job at FAIR / Deepmind or Google Brain/ Nvidia Research as a researcher or ML scientist the expectations of knowledge are very different than 'data science'. These are research lab groups, that work on pushing the state of the art forward. They are also supported by great engineers, building awesome tools that improve ML research. So transitioning into this sort of role requires more than doing Kaggle competitions, it requires developing an intuition for the respective ML subfield / and trying new things and usually failing. i.e. this is a research role and will require a lot of study and learning If on the other hand you are looking for datascience / take model and build pipeline to run AI, or perform hyper param sweeps or simply modify some model code, then on I would say that is much more engineering than research ML. This has a much lower barrier to entry coming from engineering and could be a good stepping stone to a transition into pure ML research. On a more general note to consider when thinking of transitioning to ML is that these systems are probabilistic in nature vs purely deterministic as they are in more general software systems. People (ie humans) are bad at wrapping their heads around distributional processes - you can see this in all fields that deal with them (Quantum vs Classical Physics, Biological Systems etc). In general I guess what I have seen is when engineers try to dip their toes into ML, what's required is a mindset shift in how to approach problems. Once that happens the depth of that shift determines the type of role with ML you wish to pursue.

deepsquirrelnet 5y ago
It's interesting that you mention this, because there's quite an impressive resurgence of privately funded R&D going on in the ML space. We're in an interesting phase, where the field is moving too fast to have 'canonical' methodologies (though we're getting close). To be an engineer in the deep learning space often requires reading and keeping up with research. Everything just gets dumped on arXiv, because the peer-reviewed publication cycle is almost too slow for the field. Making a successful transition to ML in my opinion, depends a lot on the individual. Without a strong background in calculus, linear algebra and statistics, it's going to be difficult. Training a model is what people tend to focus on, but in my opinion, that's the easy part. Evaluating/validating a model, analyzing and preparing your data, anticipating model performance, understanding what to do to improve your fit, model selection or architecture. Developing custom deep learning architectures at times requires a bit of an abstract mathematical intuition that I think will suit many engineers very well. A lot of engineers are well equipped to be successful in making a transition, but on the other hand, at least as many aren't. In the future, I think the field will have many varying degrees of expertise, with the barriers to entry becoming lower all the time. We're reaching a point where some common use cases can be solved adequately in a nearly automated fashion. Some "autoML" tools don't really require any real understanding of ML, though I think it's not wise to get in the habit of using them without understanding how to evaluate a fit. These tools will be great for people who want to occasionally use ML to solve some smaller problems, but as a part of their larger job function. In some middle area, ML engineers and practitioners will be training and operationalizing models, and keeping up with major developments in research. But there will be some significant changes in the next decade. I predict the nebulous of data science, data analysis and machine learning will become formalized into 3 major skills - exploratory data analysis, machine learning and advanced computational statistics. At the lowest level, researchers will continue developing the field, which like you say, is probably not something you transition directly into.

bgroat 5y ago

Maybe I'm alone here... but the biggest hurdle I have with ML as a software engineer is the python ecosystem

Khelavaster 5y ago

This is myopic...ML.Net takes care of these issues.