33 ms·

The age of machine learning as code has arrived

117 points by gk1 5y ago

snek_case 5y ago
My experience working in a deep learning outfit for a few years has been that the software stack is incredibly immature and fragile. You're generally tied to proprietary nvidia drivers that are a pain to install, and when they change version, it breaks software such as PyTorch, you're forced to update. PyTorch itself often has breaking changes. Then, most of the code is written in Python, often by academics who know and care very little about good software engineering practices. That also leads to fragile and breakable code. You'll often find yourself trying to install code that's just DOA because of multiple broken dependencies. The quality of the code can be terrible, because academics are in a constant rush to publish the next thing, and tend to view code as throwaway. Worse yet, it's often written by students, and these students are taught by professors who view software engineering as an inferior, unscientific discipline. I've seen many github repos with no open source license, and no README, just a random code dump with no comments. I'm personally not super enthusiastic by the idea of replacing a lot of code by machine learning just yet, because not only is the software ecosystem fragile, but the models can be as well. You're likely to introduce weird, unpredictable modes of failure. I can see that there's clearly a lot of people who dream of being able to basically create software without coding, and not have to deal with pesky software developers, but IMO, we're not there yet. This guy talks about cloud-based machine learning solutions. You won't have to deal with drivers and general "IT" concerns there. The problem is that, besides being very expensive, these cloud-based solution will introduce extra layers of vendor lock-in. If you're training on the google cloud and using TPUs that you can't even physically buy/own, you might have a hard time porting your code to run without a TPU. If Google revokes your access for some reason, because they claim you've violated their TOS, you are fucked.
- jakear 5y ago
  The trick is to get a disk that works; wire it up to boot up, run your job, then shut down, on whatever schedule you need; then never touch it for anything besides business logic changes. Brownie points are awarded for persisting the setup of the image in the form of a dockerfile, shell script, etc., but such documents are to be considered non-authoritative sources, and must be expected to break at a rate roughly proportional to their size.
- cjv 5y ago
  Can you name some breaking changes PyTorch made?
  - snek_case 5y ago
    Not off-hand because it's been a while, but IIRC the names of some methods were changed, arguments were added/removed. I will say that was before version 1.0, so it may be more stable now. Last I saw the nvidia driver situation did not improve though. I wanted to come back to some old code, installed a new version of PyTorch. It complained that my GPU drivers were not recent enough, had to do this long tedious process that took over an hour to get newer drivers working.
    - deleted 5y ago
      [deleted]
- zby 5y ago
  I have tried using Google Cloud - I spent a lot of time finding a location and getting the permissions to use GPUs. Then one time after I switched off the vm to economize on costs I could not switch it on again because of lack of resources. I don't know if I could migrate it somewhere else - but most probably not, because the GPUs setups are pretty unique. After a few weeks I gave up. I bought a laptop with nvidia. I switched off everything that I thought was to switch off. Then for 4 months I was constantly being reminded about something more to switch off. Now I still have a static IP address and I don't know how to switch it off because the VM it was attached to is deleted. By the way as a seasoned (and actually kind of retired) programmer I started learning ML by myself after deciding that I don't like the code I was getting from hired companies. It is a strange land with many software writing practicies surprising someone coming from different programming culture. I wish I could find a community of people who made a similar path - here is my blog: https://zby.github.io/ https://zby.github.io/.
- qcoh 5y ago
  To add to that my current struggle: Most stuff is geared towards conda and terrible engineering practices (requirements.txt and not even locked versions). If you want to actually build and deploy something on actual hardware, you are in a world of hurt. And even for training the dependency situation is so ridiculous that most repos come with their own Dockerfile. I marvel at the cool machine learning demos but I'm kind of sick working on this stuff, tbh.
  - snek_case 5y ago
    Also, nvidia-docker...
ragnot 5y ago
I've heard that software engineers are the automobile workers of the 50s-70s: Extremely well-compensated professionals who were ultimately thrown to the wayside when technology came. I wonder if data scientists and AI will do a similar thing to software engineers.
- snek_case 5y ago
  In my opinion, unless we develop AGI, not at all... And if we did develop AGI, then data scientists would find themselves out of a job too. I think you'd be very hard pressed to automate systems programming, and most forms of programming for that matter. So, as long as we develop new hardware, new products, new platforms, new programming languages, then there will be lots of programming work to be done. The jobs that machine learning is replacing right now (if any?) are fairly repetitive jobs that don't really require any reasoning. Maybe machine learning is also helping automate phishing schemes, but, is there any programming job out there that's really repetitive and thoughtless, where the work is predictable and akin to factory work? Do you think a transformer model is going to be able to handle debugging?
  - ragnot 5y ago
    You obviously have more knowledge than me on this subject, so take what I say with that in mind. But I look at the march of progress and can't help but draw parallels. Websites used to be made by hand. Then came frameworks and then came code-free e-commerce sites like Shopify. Github is testing the Copilot AI. How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec. REST API's are already defined by their model and have code generated to match it. Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
    - snek_case 5y ago
      However, despite the existence of products like Shopify and Wordpress, web developers have no trouble finding jobs, as far as I know. > How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec. Would that person not be, essentially, a programmer? And what would they do if they needed to interface with external hardware that's not yet supported, or some network protocol or piece of software that's not built into AI-code-generator-3000? > Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering? In my opinion, people who probably think little of software developers, keep imagining that software engineering is this primitive, simple, reducible thing, and they are simply wrong. Can there be a use for AI-generated code, and can that take over things like generating user-facing forms and website? Sure. However, what I think is likely to happen is that in 15-25 years, the world will be changing just as fast as it is now, and the software engineering discipline will just expand. Companies that hire "architects" who only know how to use AI-code-generator-3000 will be at a disadvantage compared to companies that hire more fully fledged software engineers. Imagine you run a business and you only have architects who know how to use some code generator code that runs on the Google cloud. You pay some monthly bill to Google and hefty fees to use that tool. You're locked-in to that ecosystem. If you need a new feature or access to a new API, you may need to wait for Google to add support. In contrast, your competitors who hired pesky software engineers are paying higher salaries but save on Google cloud fees, and they don't have to wait for Google to fix problems to implement new features.
didibus 5y ago
> A few years ago, Forbes wrote that "Software ate the world, now AI is eating Software" As far as I know, AI is Software, so this type of reasoning really confuses me. I think it is possible that "developers" might have some of their work find automation using AI and ML, but Computer Science will obviously stick around as long as computers are around. For example, someone has to code the algorithms which are used to teach the computer how to learn. Someone has to code the pipelines for capturing data from the physical world into the digital world. Someone has to code the storage and exchange of that data to the ML models, and someone has to plug the inference into the production processes and systems. Someone has to put alarms and monitors around all these things to assert their operation. Someone has to code the UIs and interfaces that users will use to interact with all those systems. Finally, someone needs to code compatibility and optimizations of all these models and their implementation to continue to work and leverage new hardware capabilities. If people think that developers were spending 90% of their time hand tuning business rules and coming up with manually implemented inference and decision models they are highly mistaken. That could have been 5% of the job in some circles, but 95% of the job has always been all these peripheral tasks. If anything, there will be even more work now, since ML models require a lot more periphery to develop, train and ship.
- indrax 5y ago
  Software is also part of the world, just a very different part.
  - didibus 5y ago
    True, though I'm assuming by "the world", they mean that traditional processes which would have involved pen and paper and other non computer based tools and systems are all being redesigned to make use of computers, thus Software is eating them. But AI is not replacing processes that rely on software with something that no longer relies on software. What AI is doing is it lets you replace even more processes that were not yet able to be handled by a computer with one that a computer can do, and thus it just helps Software eat even more of the world. Decision making and judgement tasks were not able to be successfully moved to computers prior to the recent advancements in AI, now some of those can be.
justicezyx 5y ago
In the relentless pursuing of fame and profit, the knowledge output from companies are outpacing serious academic research institutions. Because of the company's inherent goal for profit, knowledge is also more and more being sugar-coated and pushed out prematurely (which had nothing to do with the morality of the people involved, as the environment of operation are different). Weaker and weaker knowledge is being used nowadays across the society in easier and easier accessible fashion. Skills are replace by automated answers both the asker and answerer might not have the mental capacity and tenacity to truly understand and verify. Cracks are forming between all corners of the intellectual babel tower that is being building and deteriorating at the same time. Of course, we have not figure out how to reliably writing dumb code yet, but we are going to replace that with ML. Now everyone involved can truly shed the responsibility of program failures. After all, no one understand ML anyway...
deleted 5y ago
[deleted]
guard0g 5y ago
This is the best quote I've seen in a while: "Do machine learning like the great engineer you are, not like the great machine learning expert you aren't."
sershe 5y ago
This feel like a slow end of an era. I work in a mixed ML/dev team, on the dev side (distributed systems) and, looking at real-world ML work (and having tried it a bit at home), I really don't want to do ML unless that becomes the only choice... I hope I can get standard dev gigs till I can retire ;) There was a good quote either from Joel or someone like that (or someone he was interviewing on a podcast), along the lines of "I went into software because I like tinkering with intricate clockwork; not because I like training a puppy to not pee on the carpet". On top of that, you tend to not even spend most of the time on the puppy, most of the time is spent collecting and organizing carpet samples.
ngc248 5y ago
"AI is increasingly being applied to mission critical infrastructure like national electric grids and automated supermarket warehousing calculations during pandemics. However, there are questions about whether the maturity of the industry has caught up with the enormity of its growing deployment." I surmise "being applied" means outcome of an ML model is used to augment the decision taken by a human. I doubt they are used to handle everything automatically for critical infra, since ML models are neither 100% accurate and fail miserably when they fail. But the insinuation is that the field has already been won by AI.

117 points by gk1 5y ago

My experience working in a deep learning outfit for a few years has been that the software stack is incredibly immature and fragile. You're generally tied to proprietary nvidia drivers that are a pain to install, and when they change version, it breaks software such as PyTorch, you're forced to update. PyTorch itself often has breaking changes. Then, most of the code is written in Python, often by academics who know and care very little about good software engineering practices. That also leads to fragile and breakable code. You'll often find yourself trying to install code that's just DOA because of multiple broken dependencies. The quality of the code can be terrible, because academics are in a constant rush to publish the next thing, and tend to view code as throwaway. Worse yet, it's often written by students, and these students are taught by professors who view software engineering as an inferior, unscientific discipline. I've seen many github repos with no open source license, and no README, just a random code dump with no comments. I'm personally not super enthusiastic by the idea of replacing a lot of code by machine learning just yet, because not only is the software ecosystem fragile, but the models can be as well. You're likely to introduce weird, unpredictable modes of failure. I can see that there's clearly a lot of people who dream of being able to basically create software without coding, and not have to deal with pesky software developers, but IMO, we're not there yet. This guy talks about cloud-based machine learning solutions. You won't have to deal with drivers and general "IT" concerns there. The problem is that, besides being very expensive, these cloud-based solution will introduce extra layers of vendor lock-in. If you're training on the google cloud and using TPUs that you can't even physically buy/own, you might have a hard time porting your code to run without a TPU. If Google revokes your access for some reason, because they claim you've violated their TOS, you are fucked.

jakear 5y ago
The trick is to get a disk that works; wire it up to boot up, run your job, then shut down, on whatever schedule you need; then never touch it for anything besides business logic changes. Brownie points are awarded for persisting the setup of the image in the form of a dockerfile, shell script, etc., but such documents are to be considered non-authoritative sources, and must be expected to break at a rate roughly proportional to their size.
cjv 5y ago
Can you name some breaking changes PyTorch made?
- snek_case 5y ago
  Not off-hand because it's been a while, but IIRC the names of some methods were changed, arguments were added/removed. I will say that was before version 1.0, so it may be more stable now. Last I saw the nvidia driver situation did not improve though. I wanted to come back to some old code, installed a new version of PyTorch. It complained that my GPU drivers were not recent enough, had to do this long tedious process that took over an hour to get newer drivers working.
  - deleted 5y ago
    [deleted]
zby 5y ago
I have tried using Google Cloud - I spent a lot of time finding a location and getting the permissions to use GPUs. Then one time after I switched off the vm to economize on costs I could not switch it on again because of lack of resources. I don't know if I could migrate it somewhere else - but most probably not, because the GPUs setups are pretty unique. After a few weeks I gave up. I bought a laptop with nvidia. I switched off everything that I thought was to switch off. Then for 4 months I was constantly being reminded about something more to switch off. Now I still have a static IP address and I don't know how to switch it off because the VM it was attached to is deleted. By the way as a seasoned (and actually kind of retired) programmer I started learning ML by myself after deciding that I don't like the code I was getting from hired companies. It is a strange land with many software writing practicies surprising someone coming from different programming culture. I wish I could find a community of people who made a similar path - here is my blog: https://zby.github.io/ https://zby.github.io/.
qcoh 5y ago
To add to that my current struggle: Most stuff is geared towards conda and terrible engineering practices (requirements.txt and not even locked versions). If you want to actually build and deploy something on actual hardware, you are in a world of hurt. And even for training the dependency situation is so ridiculous that most repos come with their own Dockerfile. I marvel at the cool machine learning demos but I'm kind of sick working on this stuff, tbh.
- snek_case 5y ago
  Also, nvidia-docker...

ragnot 5y ago

I've heard that software engineers are the automobile workers of the 50s-70s: Extremely well-compensated professionals who were ultimately thrown to the wayside when technology came. I wonder if data scientists and AI will do a similar thing to software engineers.

snek_case 5y ago
In my opinion, unless we develop AGI, not at all... And if we did develop AGI, then data scientists would find themselves out of a job too. I think you'd be very hard pressed to automate systems programming, and most forms of programming for that matter. So, as long as we develop new hardware, new products, new platforms, new programming languages, then there will be lots of programming work to be done. The jobs that machine learning is replacing right now (if any?) are fairly repetitive jobs that don't really require any reasoning. Maybe machine learning is also helping automate phishing schemes, but, is there any programming job out there that's really repetitive and thoughtless, where the work is predictable and akin to factory work? Do you think a transformer model is going to be able to handle debugging?
- ragnot 5y ago
  You obviously have more knowledge than me on this subject, so take what I say with that in mind. But I look at the march of progress and can't help but draw parallels. Websites used to be made by hand. Then came frameworks and then came code-free e-commerce sites like Shopify. Github is testing the Copilot AI. How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec. REST API's are already defined by their model and have code generated to match it. Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering?
  - snek_case 5y ago
    However, despite the existence of products like Shopify and Wordpress, web developers have no trouble finding jobs, as far as I know. > How much longer before some architect tells an AI to stitch together code that fits some sort of functional spec. Would that person not be, essentially, a programmer? And what would they do if they needed to interface with external hardware that's not yet supported, or some network protocol or piece of software that's not built into AI-code-generator-3000? > Is it that much a stretch to say that in the near future (15-25 years), AI might take over a significant part of software engineering? In my opinion, people who probably think little of software developers, keep imagining that software engineering is this primitive, simple, reducible thing, and they are simply wrong. Can there be a use for AI-generated code, and can that take over things like generating user-facing forms and website? Sure. However, what I think is likely to happen is that in 15-25 years, the world will be changing just as fast as it is now, and the software engineering discipline will just expand. Companies that hire "architects" who only know how to use AI-code-generator-3000 will be at a disadvantage compared to companies that hire more fully fledged software engineers. Imagine you run a business and you only have architects who know how to use some code generator code that runs on the Google cloud. You pay some monthly bill to Google and hefty fees to use that tool. You're locked-in to that ecosystem. If you need a new feature or access to a new API, you may need to wait for Google to add support. In contrast, your competitors who hired pesky software engineers are paying higher salaries but save on Google cloud fees, and they don't have to wait for Google to fix problems to implement new features.

didibus 5y ago

> A few years ago, Forbes wrote that "Software ate the world, now AI is eating Software" As far as I know, AI is Software, so this type of reasoning really confuses me. I think it is possible that "developers" might have some of their work find automation using AI and ML, but Computer Science will obviously stick around as long as computers are around. For example, someone has to code the algorithms which are used to teach the computer how to learn. Someone has to code the pipelines for capturing data from the physical world into the digital world. Someone has to code the storage and exchange of that data to the ML models, and someone has to plug the inference into the production processes and systems. Someone has to put alarms and monitors around all these things to assert their operation. Someone has to code the UIs and interfaces that users will use to interact with all those systems. Finally, someone needs to code compatibility and optimizations of all these models and their implementation to continue to work and leverage new hardware capabilities. If people think that developers were spending 90% of their time hand tuning business rules and coming up with manually implemented inference and decision models they are highly mistaken. That could have been 5% of the job in some circles, but 95% of the job has always been all these peripheral tasks. If anything, there will be even more work now, since ML models require a lot more periphery to develop, train and ship.

indrax 5y ago
Software is also part of the world, just a very different part.
- didibus 5y ago
  True, though I'm assuming by "the world", they mean that traditional processes which would have involved pen and paper and other non computer based tools and systems are all being redesigned to make use of computers, thus Software is eating them. But AI is not replacing processes that rely on software with something that no longer relies on software. What AI is doing is it lets you replace even more processes that were not yet able to be handled by a computer with one that a computer can do, and thus it just helps Software eat even more of the world. Decision making and judgement tasks were not able to be successfully moved to computers prior to the recent advancements in AI, now some of those can be.

justicezyx 5y ago

In the relentless pursuing of fame and profit, the knowledge output from companies are outpacing serious academic research institutions. Because of the company's inherent goal for profit, knowledge is also more and more being sugar-coated and pushed out prematurely (which had nothing to do with the morality of the people involved, as the environment of operation are different). Weaker and weaker knowledge is being used nowadays across the society in easier and easier accessible fashion. Skills are replace by automated answers both the asker and answerer might not have the mental capacity and tenacity to truly understand and verify. Cracks are forming between all corners of the intellectual babel tower that is being building and deteriorating at the same time. Of course, we have not figure out how to reliably writing dumb code yet, but we are going to replace that with ML. Now everyone involved can truly shed the responsibility of program failures. After all, no one understand ML anyway...

deleted 5y ago

[deleted]

guard0g 5y ago

This is the best quote I've seen in a while: "Do machine learning like the great engineer you are, not like the great machine learning expert you aren't."

sershe 5y ago

This feel like a slow end of an era. I work in a mixed ML/dev team, on the dev side (distributed systems) and, looking at real-world ML work (and having tried it a bit at home), I really don't want to do ML unless that becomes the only choice... I hope I can get standard dev gigs till I can retire ;) There was a good quote either from Joel or someone like that (or someone he was interviewing on a podcast), along the lines of "I went into software because I like tinkering with intricate clockwork; not because I like training a puppy to not pee on the carpet". On top of that, you tend to not even spend most of the time on the puppy, most of the time is spent collecting and organizing carpet samples.

ngc248 5y ago

"AI is increasingly being applied to mission critical infrastructure like national electric grids and automated supermarket warehousing calculations during pandemics. However, there are questions about whether the maturity of the industry has caught up with the enormity of its growing deployment." I surmise "being applied" means outcome of an ML model is used to augment the decision taken by a human. I doubt they are used to handle everything automatically for critical infra, since ML models are neither 100% accurate and fail miserably when they fail. But the insinuation is that the field has already been won by AI.