4 ms·

No it won’t transform manufacturing. Deep RL in physical systems is a pipe dream. Deep RL is one of the most data hungry methods. And in physical systems you d

by jpfr 5y ago

No it won’t transform manufacturing. Deep RL in physical systems is a pipe dream. Deep RL is one of the most data hungry methods. And in physical systems you don’t get enough samples. There are loads of data. But no manager of a shopfloor lets you produce scrap just to potentially learn. There is an opportunity to combine ML with classical engineering models for manufacturing. Think differential equations for chemical processes. Then you end up with something like ML augmented control theory. That does work.

thoughtstheseus 5y ago
That’s why you simulate the process.
- Frost1x 5y ago
  Your simulated environment needs to accurately represent everything you're simulating for RL to be useful, otherwise, it will simply find artifacts of flaws of the sim itself or may fail to discover n-th order effects since they won't ever occur. This is exactly where DNNs have excelled and is most desired to be used: in all the places we don't know the foundational theory to. That may be why they're so successful compared to our traditional reductionist approaches: they might catch all the non-reduced aspects we never add into sims as we know them. We don't need DNNs doing QED or Newtonian mechanics (in general), we have nice solutions to these that in many (not all) cases are pretty darn efficient. In some cases, you could be computationally bound and never achieve a sim representing your desired environment. In many, you lack theory to correctly build a sim. You need RL building the sims that you plan to use RL to explore optimal processes you seek within. We have a few disciplines that are well formulated enough where sims can be useful in but the vast majority are simply going to give you garbage or are just computationally bound. At best, many sims provide guidance that experts need to interpret.
jgalt212 5y ago
> Deep RL is one of the most data hungry methods. What's good for AWS is good for America.
dimatura 5y ago
It's not easy (or it would have been done already!) but I don't think it's a pipe dream. A lot of the computer vision capabilities that are taken for granted today would've been pipe dreams a few years ago. It's true that the process is data hungry. But this isn't such a big problem in some cases. Having a few dozens or more of robot arms just figuring out stuff by trial and error isn't such a big deal (you don't have to do this at the customer's shop floor...). For self driving cars and drones this isn't such a great idea (though it has been tried to some extent). That's where simulation and more clever algorithms come in. Including, of course some prior knowledge, like you are implying. I don't think anybody expects to "transform manufacturing" by just throwing REINFORCE into a random KUKA arm.
LudwigNagasena 5y ago
Manufacturers already install all sorts of sensors to control the production process. Of course if you produce high end luxury cars there is probably not enough data for a useful model, but if you operate e.g. a hot rolling mill it may be useful.
- jpfr 5y ago
  Hot rolling is a good example. For the last 30 years neural networks have been used to control the process. There’s a nice NeurIPS paper from 1991 [1]. And even with the non-deep networks from back in the day, people felt compelled to include prior knowledge and even used Bayesian priors to deal with a lack of data. [1] http://papers.neurips.cc/paper/447-neural-control-for-rolling-mills-incorporating-domain-theories-to-overcome-data-deficiency.pdf http://papers.neurips.cc/paper/447-neural-control-for-rollin...

by jpfr 5y ago

thoughtstheseus 5y ago

That’s why you simulate the process.

Frost1x 5y ago
Your simulated environment needs to accurately represent everything you're simulating for RL to be useful, otherwise, it will simply find artifacts of flaws of the sim itself or may fail to discover n-th order effects since they won't ever occur. This is exactly where DNNs have excelled and is most desired to be used: in all the places we don't know the foundational theory to. That may be why they're so successful compared to our traditional reductionist approaches: they might catch all the non-reduced aspects we never add into sims as we know them. We don't need DNNs doing QED or Newtonian mechanics (in general), we have nice solutions to these that in many (not all) cases are pretty darn efficient. In some cases, you could be computationally bound and never achieve a sim representing your desired environment. In many, you lack theory to correctly build a sim. You need RL building the sims that you plan to use RL to explore optimal processes you seek within. We have a few disciplines that are well formulated enough where sims can be useful in but the vast majority are simply going to give you garbage or are just computationally bound. At best, many sims provide guidance that experts need to interpret.

jgalt212 5y ago

> Deep RL is one of the most data hungry methods. What's good for AWS is good for America.

dimatura 5y ago

It's not easy (or it would have been done already!) but I don't think it's a pipe dream. A lot of the computer vision capabilities that are taken for granted today would've been pipe dreams a few years ago. It's true that the process is data hungry. But this isn't such a big problem in some cases. Having a few dozens or more of robot arms just figuring out stuff by trial and error isn't such a big deal (you don't have to do this at the customer's shop floor...). For self driving cars and drones this isn't such a great idea (though it has been tried to some extent). That's where simulation and more clever algorithms come in. Including, of course some prior knowledge, like you are implying. I don't think anybody expects to "transform manufacturing" by just throwing REINFORCE into a random KUKA arm.

LudwigNagasena 5y ago

Manufacturers already install all sorts of sensors to control the production process. Of course if you produce high end luxury cars there is probably not enough data for a useful model, but if you operate e.g. a hot rolling mill it may be useful.

jpfr 5y ago
Hot rolling is a good example. For the last 30 years neural networks have been used to control the process. There’s a nice NeurIPS paper from 1991 [1]. And even with the non-deep networks from back in the day, people felt compelled to include prior knowledge and even used Bayesian priors to deal with a lack of data. [1] http://papers.neurips.cc/paper/447-neural-control-for-rolling-mills-incorporating-domain-theories-to-overcome-data-deficiency.pdf http://papers.neurips.cc/paper/447-neural-control-for-rollin...