6 ms·

This is brand new technology. It's going to take a few years to reliably productionize - and most of the applied solutions will look nothing like the research.

This is brand new technology. It's going to take a few years to reliably productionize - and most of the applied solutions will look nothing like the research. Many real world problems are going to combine multiple neural nets into systems with specific applications and there's a lot of detail to work out. The hype may collapse in the short term, but that's only because many of the first movers are stereotypical tech startups who overpromise without truly understanding the problem or solution spaces and therefore underdeliver. But, speaking from personal experience, some of the tech has already been proven - one example is massively accelerated modeling as an alternative to slow finite difference/finite element simulation with 99% accuracy, which will in the next 6-12 months totally change the approach to a wide range of modeling problems, and enable a totally new form of work where instead of setting up a model and waiting days or weeks, one may iterate effectively in real time. There are emerging solutions to knowledge management and "intelligent" data harvesting, where ML outputs are being manipulated in a rudimentary form of reasoning. Think specialized industries like petroleum, mechanical engineering, EM engineering - plenty of "layman" related features like recommendation engines are going to flop, but the cat is out of the bag for heavy industrial knowledge work. Just give it some time - we are on the cusp of a monumental leap in R&D across the spectrum of human endeavor. Very exciting times.

semi-extrinsic 7y ago
> one example is massively accelerated modeling as an alternative to slow finite difference/finite element simulation I work in the field of numerical modelling (fluid simulation), and I haven't seen any convincing demos of anything but utterly trivial problems solved this way. Care to share some examples? (There are some works that focus on fluid simulation for CGI. That's neat, but a) only interpolates between real simulation data, and b) doesn't achieve physical accuracy, just looks good.)
- allovernow 7y ago
  >Care to share some examples? Unfortunately, the only example I know of is confidential, with a provisional patent filed. But what I can tell you is that we are already solving non-trivial problems some 3-6 orders of magnitude faster than FDM/FEM. Not quite navier-stokes tier, but still in the realm of PDEs. Sorry I can't be more specific, such is the nature of cutting edge industry, as I'm sure you understand!
  - semi-extrinsic 7y ago
    "Nullius in verba" - I'll believe it when I see it. And not sure I see the point in patenting such a thing. If it's as good as you say, in less than six months a research group in some country where US patents aren't held in very high regard will have duplicated the work and either sell it as a service or publish it in the scientific literature. They might not even mention knowing about the patent at all! Things have been discovered by multiple people almost simultaneously so many times in the history of science that it's almost a natural law; it'd be impossible to prove foul play.
    - jacques_chester 7y ago
      > in less than six months a research group in some country where US patents aren't held in very high regard will have duplicated the work and either sell it as a service or publish it in the scientific literature US patents can be used to prevent the sale or import of violating foreign products into the US market. It's still the largest market and US patents are still the lynchpin of any patent portfolio. Simultaneous discovery does occur and primacy is difficult to prove. Most patent offices had first-to-file as the deciding rule, the USPTO adopted first-to-file a few years back to cut down on endless litigation.
metalens 7y ago
I used to do electromagnetic modeling using finite element methods (though now a product manager for AI software infra) and it would to take me on the order of hours to days or weeks to model wave interaction with real-world objects. A machine learning model trained to understand Maxwell's Equations can in principle be used perform said simulations, resulting in probably an order or more of magnitude increase in simulation speed. Getting this to work well will reduce the time (and cost) it takes to design optical sensors, radar for autonomous vehicles, smartphone antennas, MRI machines, and more. Having said that, it would require a lot of heaving lifting to pull this off to achieve near-physical accuracy for real-world physics problems. A cursory search on Google for "arxiv deep learning electromagnetics" returns results of proofs of concept in this direction.
- allovernow 7y ago
  You're on the right track. A lot of this tech is a potential goldmine and I'm sure there are many players developing in secret and not publishing yet (or ever).
- YeGoblynQueenne 7y ago
  Were would the speedup come from? I don't understand. If I understand your comment correctly, essentially you have a hand-crafted simulator for some physical process and then you train a neural net model to approximate the simulator. Why would the approximated simulator have "an order or more of magnitude increase in simulation speed"? Unless the approximation has massive losses in accuracy, of course. Honestly asking and really interested to know what you mean.
  - allovernow 7y ago
    It's all about precision heuristics, derived from joint probabilities of inputs and outputs. That, by and large, is how I am increasingly coming to understand the power of neural networks. Imagine you are given a picture of a candle, overlaid with a grid, and asked to fill in, with colored pencils, colors for the air surrounding the candle representing relative temperature. Of course a human utilizes intuition to rapidly assign high temperature to the flame and decreasing temperature with increasing distance. A "dumb" finite method would need, even for such a relatively simple problem (for a human), to perform calculations for a series of time steps in each grid until some steady state condition to arrive at a much more precise but still overall similar coloring of the grid cells. You can do the same task much more quickly because you have developed intuition of the physics, which is to say you have learned heuristics which capture the general trends of the problem (air is hot close to a flame and cold far away). Neural nets take the best of both worlds - by effectively learning probability relationships between input and output pixels, they internalize heuristic approaches to produce outputs approaching finite method accuracies at a fraction of the computation. There's a lot of waste that can be optimized out of finite computation by hardcoding rules (heuristics), but doing so for real problems is impractical. Neural nets learn these rules through training - a far simpler task is organizing the data to teach the net the right trends; much like designing lessons for a child to teach a predictive ability.

semi-extrinsic 7y ago

> one example is massively accelerated modeling as an alternative to slow finite difference/finite element simulation I work in the field of numerical modelling (fluid simulation), and I haven't seen any convincing demos of anything but utterly trivial problems solved this way. Care to share some examples? (There are some works that focus on fluid simulation for CGI. That's neat, but a) only interpolates between real simulation data, and b) doesn't achieve physical accuracy, just looks good.)

allovernow 7y ago
>Care to share some examples? Unfortunately, the only example I know of is confidential, with a provisional patent filed. But what I can tell you is that we are already solving non-trivial problems some 3-6 orders of magnitude faster than FDM/FEM. Not quite navier-stokes tier, but still in the realm of PDEs. Sorry I can't be more specific, such is the nature of cutting edge industry, as I'm sure you understand!
- semi-extrinsic 7y ago
  "Nullius in verba" - I'll believe it when I see it. And not sure I see the point in patenting such a thing. If it's as good as you say, in less than six months a research group in some country where US patents aren't held in very high regard will have duplicated the work and either sell it as a service or publish it in the scientific literature. They might not even mention knowing about the patent at all! Things have been discovered by multiple people almost simultaneously so many times in the history of science that it's almost a natural law; it'd be impossible to prove foul play.
  - jacques_chester 7y ago
    > in less than six months a research group in some country where US patents aren't held in very high regard will have duplicated the work and either sell it as a service or publish it in the scientific literature US patents can be used to prevent the sale or import of violating foreign products into the US market. It's still the largest market and US patents are still the lynchpin of any patent portfolio. Simultaneous discovery does occur and primacy is difficult to prove. Most patent offices had first-to-file as the deciding rule, the USPTO adopted first-to-file a few years back to cut down on endless litigation.

metalens 7y ago

I used to do electromagnetic modeling using finite element methods (though now a product manager for AI software infra) and it would to take me on the order of hours to days or weeks to model wave interaction with real-world objects. A machine learning model trained to understand Maxwell's Equations can in principle be used perform said simulations, resulting in probably an order or more of magnitude increase in simulation speed. Getting this to work well will reduce the time (and cost) it takes to design optical sensors, radar for autonomous vehicles, smartphone antennas, MRI machines, and more. Having said that, it would require a lot of heaving lifting to pull this off to achieve near-physical accuracy for real-world physics problems. A cursory search on Google for "arxiv deep learning electromagnetics" returns results of proofs of concept in this direction.

allovernow 7y ago
You're on the right track. A lot of this tech is a potential goldmine and I'm sure there are many players developing in secret and not publishing yet (or ever).
YeGoblynQueenne 7y ago
Were would the speedup come from? I don't understand. If I understand your comment correctly, essentially you have a hand-crafted simulator for some physical process and then you train a neural net model to approximate the simulator. Why would the approximated simulator have "an order or more of magnitude increase in simulation speed"? Unless the approximation has massive losses in accuracy, of course. Honestly asking and really interested to know what you mean.
- allovernow 7y ago
  It's all about precision heuristics, derived from joint probabilities of inputs and outputs. That, by and large, is how I am increasingly coming to understand the power of neural networks. Imagine you are given a picture of a candle, overlaid with a grid, and asked to fill in, with colored pencils, colors for the air surrounding the candle representing relative temperature. Of course a human utilizes intuition to rapidly assign high temperature to the flame and decreasing temperature with increasing distance. A "dumb" finite method would need, even for such a relatively simple problem (for a human), to perform calculations for a series of time steps in each grid until some steady state condition to arrive at a much more precise but still overall similar coloring of the grid cells. You can do the same task much more quickly because you have developed intuition of the physics, which is to say you have learned heuristics which capture the general trends of the problem (air is hot close to a flame and cold far away). Neural nets take the best of both worlds - by effectively learning probability relationships between input and output pixels, they internalize heuristic approaches to produce outputs approaching finite method accuracies at a fraction of the computation. There's a lot of waste that can be optimized out of finite computation by hardcoding rules (heuristics), but doing so for real problems is impractical. Neural nets learn these rules through training - a far simpler task is organizing the data to teach the net the right trends; much like designing lessons for a child to teach a predictive ability.