6 ms·

I've observed an increasing trend with modern AI research that A) It's no longer practical to incorporate into real products given the hardware costs for train

by lumost 6y ago

I've observed an increasing trend with modern AI research that A) It's no longer practical to incorporate into real products given the hardware costs for training and inference. In NLP, large transformer based models can easily occupy a 5 thousand dollar GPU at a paltry 50-60 words per second processing pace. B) The Research often demonstrates things that existing non-ML toolchains were already reasonable at doing with a better result, and a faster iteration time. This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place! While this is fascinating from the technical and academic perspective, it's hard to be convinced that any gains we see in the field won't be offset by increased hardware/latency costs from a product standpoint.

lostmsu 6y ago
A) is just a sign, that further engineering, both model and framework, is required. Also, 50-60 words per second does not sound too bad. It amounts to 4M words per day. I imagine most people only type a thousand words per day. So with a single GPU you can handle 4000 users (assuming batch-style processing). B) This specific example actually shows ML toolchains are orders of magnitude better, than existing ones. Imagine you wanted to make a football game from scratch, and this technique would let you do that by just feeding a lot of football game recordings to a program, as opposed to hiring software devs and graphics designers, and iterating for a year.
- lumost 6y ago
  On A: Using a p3.2xl for that task leaves you with a minimum ec2 cost of goods sold of 50 cents per user per month for the target task. To make a reasonable SaaS business out of this you would need to be charging a minimum of $5.00 per user month for the service that this GPU is providing, assuming all other tasks have a negligible impact on Cost of Goods Sold, and that you are able to spread the load optimally throughout the day - and thus don't have to purchase higher capacity during peak. This isn't too bad of a situation if this GPU provides 100% of your value proposition. but, if you needed your product to perform 5 such tasks - or competition/customer demand forces you to move to a bigger model with a 10 words per second inference rate then you may find it hard to balance customer value vs. price vs. margin.
  - chx 6y ago
    > On A: Using a p3.2xl for that task leaves you with a minimum ec2 cost of goods sold of 50 cents per user per month for the target task https://twitter.com/br_/status/979442438254166016 https://twitter.com/br_/status/979442438254166016 > "selling AWS at a loss" is crisp shorthand for a lot of startups' business models!
YeGoblynQueenne 6y ago
Or, to paraphrase: great paper, no application.
- SQueeeeeL 6y ago
  It really is a great paper, the problem is when your entire field has few real applications but everyone is making too much money to admit that. Cool tech demos like this is awesome as a blog post, but not sure why it's academic...
  - YeGoblynQueenne 6y ago
    Well, the paper has an experiment where they train an agent based on their system to play two of the three games (PacMan, PacManMaze and VizDoom) and compare its result to two other systems (an LSTM-based one and a model-based RL one, WorldModel). Their system gets the best score in Pacman but unfortunately the version that does better than WorldModel is the one that doesn't have the memory module whose usefulness is a central claim of their paper. WorldModel outperforms their system in VizDoom by ~300 points, though their system without the memory module "solves" VizDoom (i.e. scores just a bit over than 750; that is 765) and they claim that it's the first Gan-based system that achieves this. A bit thin of the ground (especially when it comes to supporting the claim about the usefulness of the memory module) but the way neural nets research and especially RL research is going (from what I've seen, not my area of expertise) that's the kind of result that you can expect will get your paper published these days.
  - esoterica 6y ago
    I don't know where you got the idea that the purpose of an academic paper is to produce the schematics for a working commercial product. The fact that it's novel and interesting is sufficient grounds for producing a paper about it. Why do you think projects like these are solely the domain of blog posts?
    - YeGoblynQueenne 6y ago
      That is not true for academic papers in computer science and particularly in machine learning were contributions are expected to have some sort of practical application that is at least possible to foresee. Indeed, a common criticism against papers in machine learning is "what is a real-world application of this approach?". If you can't answer that convincingly then it's unlikely your paper gets published, particularly if it has rather mediocre results otherwise, as is the case with this paper (see my earlier comment about the comparison to the two other systems which is a bit of an anticlimax). In fact, the paper (well, the preprint version) makes a clear attempt to motivate the work with a practical application, saying that it aims to "learn a simulator [for robotic agents] by simply watching an agent interact with an environment".
esoterica 6y ago
Do you really think nVidia is seriously advocating for using neural networks as the best way to write a PacMan clone? It's obviously just a proof of concept to demonstrate that GANs are sophisticated enough to learn certain categories of models.
- robotresearcher 6y ago
  Doing these cute demos is a double-edged sword. A fun and charming demo attracts attention by non-experts who can't see the wood for the trees. In my field we make robots do a thing, say folding laundry, as an example of dextrous manipulation[1], and the popular conversation is about whether your laundry robot makes commercial sense and not about your novel grasping strategy and nifty motion planner that makes this sorta-possible for the first time. But everyone sees your video. Pros and cons. [1] laundry folding at Berkeley (not me) https://youtu.be/gy5g33S0Gzo https://youtu.be/gy5g33S0Gzo
empath75 6y ago
The ai has created a functional internal model of an external system based on limited sensory input. That is essentially a kind of consciousness.
- eckza 6y ago
  > The ai has created a functional internal model of an external system based on limited sensory input. Sure. > That is essentially a kind of consciousness. That is a flying leap.
- 101404 6y ago
  Not at all. It just means some people poked a pile of a bazillion if-else statements long enough, until the desired result came out. I believe everyone here knows the xkcd I am referring to. Still, I think it's an interesting result. But, as someone on Fridman's podcast said the other day: recent advances in AI have all been thanks to advances in computing power, not any new ideas. The future if AI seems to depend on Moore's Law, more than anything.
  - plutonorm 6y ago
    What makes you think the pile of if-else statements isn't conscious?
    - 101404 6y ago
      Because it's a pile of if-else statements shuffled until it was able to perform a single specialized thing. Any housefly would by a thousand times as "conscious".
      - bobly_today 6y ago
        What makes you so certain a house fly doesn’t have a form of rudimentary consciousness? It might be a spectrum.
        6y ago
SkyBelow 6y ago
It reminds me of older electric cars. They worked. They could get you around. But they did so at higher price, slower speeds, or other factors that a consumer wasn't willing to pay for. Thus they were regulated to novelty items that some work campuses would have and concept cars that would be shown from time to time. But eventually we reached the point where they are not only competitive, they may be positioned to fully overtake combustion cars. I wonder how many discoveries on electric cars ended up being fully dead ends along the path that took us to the point we are now?
- rootusrootus 6y ago
  It seems like there is not a lot of novel technology in an EV; what made them viable was steady improvements in battery technology so that the intersection between range, price, and weight is in a place that is financially feasible. Combined with continued improvements in battery, the already huge gain in fuel efficiency (for a typical car at least) will be what causes them to overtake ICEVs.
- YeGoblynQueenne 6y ago
  Electric cars of the era you describe could at least go. The approach described in this paper doesn't "go". Its goal is to "learn a simulator [to train a robotic agent] simply by watching an agent interacting with the environment". The learning part is done by training on the output of an existing simulator. Obviously, if you already have the output of a simulator, you don't need to learn a simulator so the only real use of this approach is to train a simulator by watching a human agent interacting with the environment. So the idea is to eventually apply this thing to training robots to interact in the real world by watching how humans do it (this is not clearly stated in the paper; it's my charitable interpretation of the very poor motivation of learning a simulator from the output of an existing simulator). The only problem is that creating an accurate simulation of a human's interaction with the real world (let alone an accurate simulation of the real world) is prohibitively expensive. Thus, for the time being, this kind of idea works only in very, very simple environments like PacMan. So we don't yet have an expensive prototype electic car that nevertheless can do everything an er other kind of car can do. We have... not sure what. The description of an idea with an example of how such a thing could be achieved in the far future, when we can train robots to act like humans by watching us.
fpgaminer 6y ago
> It's no longer practical to incorporate into real products given the hardware costs for training and inference I don't see that trend. Bleeding edge research has always been beyond what's practical, by its very nature. The focus is on pushing the boundaries, not to build something efficient/commercial. It's the followup work that makes it practical. See for example the evolution of WaveNet. It was breakthrough research, but yeah it wasn't practical. Now a couple years later and we're generating speech on mobile devices of similar quality (see the recent Facebook research), and able to teach these machines using orders of magnitude less data (see 15.ai). Not to mention the plethora of research that is explicitly about making neural networks more efficient and easier to train (e.g. EfficientNet; 1cycle; etc). Bleeding edge stuff like AlphaStar, BigGAN, GPT-2, etc are just that: bleeding edge. They'll get more efficient and practical every year, just like everything else in machine learning.
- kevincox 6y ago
  Exactly. It is sort of like saying the first transistor was too expensive to be practical. Development always starts with some novel technology that has major flaws, but some good nuggets. Then it is further developed to be practical.
- ekelsen 6y ago
  One minor nit: The recent FB blogpost wasn't really "research" - it was mostly a reimplementation of this paper: https://arxiv.org/abs/1802.08435 https://arxiv.org/abs/1802.08435
mysterEFrank 6y ago
This is obviously not meant to be a real game engine. It shows that GANs can learn not only images but transitions between images given some action, which means they can be used as simulators. See world-models which used VAEs https://worldmodels.github.io/ https://worldmodels.github.io/
srnvs123 6y ago
I don’t think that’s the point of bleeding edge research. Pushing boundries != commercialization potential
- monadic2 6y ago
  Is there research in this field that is not commercially funded? That seems like a silly distinction in America.
DeonPenny 6y ago
Researchers are not entrepreneurs. It's for the entrepreneurs and business to find a way to make this random game into something profitable.
eeZah7Ux 6y ago
> 3-4 orders of magnitude higher resource consumption Just 10000x? More like 6 orders of magnitude
anigbrowl 6y ago
Your comment reminds me of snide dismissals of personal computing in the 1980s - looks worse than [existing thing], very expensive, requires an army of technicians, probably explodes unpredictably, overall just a terrible investment.
- angel_j 6y ago
  Followed by decades long monopoly over expensive new technology, likely crushing waves of innovation and usage, in exchange for slow, buggy, not-even-automated spreadsheets; plus the hemming of governments and organizations into endless software rental contracts and platforms, that maybe don't add value as fast as Moore's Law.
  - lumost 6y ago
    that's a fair criticism, however it's not a given that any product category will end up with the same growth curve of PCs, the web, cloud, and mobile. thinking back on the PC, Web, cloud, and mobile growth periods, each came with a transformative change in the capabilities, cost, and availability of computing power. We're only seeing two of these aspects in AI today - with the cost side heading in the wrong direction.
    - angel_j 6y ago
      AI supercharged the importance of computing power to arms-race levels. We're still just brute forcing it with Deep Learning, but the capabilities are too awesome to ignore. Therefore, compute power will be bought-up and hoarded expensively, and used on the highest bidder. Will the market save us, and simply over-clock production? Not happening yet. Corporate appetite should be quite large. I do A.I. R&D and am just watching the mountain of power ascend before me and aged gaming laptop. My next level is yet the foothills. Yeah, I'd say growth or not, A.I. is going where PC, web, cloud, everything, goes, to the industrial monopolists and the money spenders.
      - lumost 6y ago
m463 6y ago
Lots of new stuff sucks. The english longbow was vastly superior to the first guns that came out, which were slow to load, unreliable to shoot, didn't work in the rain, and were extremely inaccurate. 3d printed stuff sucks. It's not very strong, it's not very pretty and takes a long time to create. Meanwhile, take a look at this ML photo processing technique: https://www.youtube.com/watch?v=bcZFQ3f26pA https://www.youtube.com/watch?v=bcZFQ3f26pA
collyw 6y ago
"This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place!" True, though my first thought was that this would be interesting from a reverse engineering point of view. Then again Pacman is pretty simple concept. Something worth reverse engineering is probably a lot more complex.

by lumost 6y ago

lostmsu 6y ago

A) is just a sign, that further engineering, both model and framework, is required. Also, 50-60 words per second does not sound too bad. It amounts to 4M words per day. I imagine most people only type a thousand words per day. So with a single GPU you can handle 4000 users (assuming batch-style processing). B) This specific example actually shows ML toolchains are orders of magnitude better, than existing ones. Imagine you wanted to make a football game from scratch, and this technique would let you do that by just feeding a lot of football game recordings to a program, as opposed to hiring software devs and graphics designers, and iterating for a year.

lumost 6y ago
On A: Using a p3.2xl for that task leaves you with a minimum ec2 cost of goods sold of 50 cents per user per month for the target task. To make a reasonable SaaS business out of this you would need to be charging a minimum of $5.00 per user month for the service that this GPU is providing, assuming all other tasks have a negligible impact on Cost of Goods Sold, and that you are able to spread the load optimally throughout the day - and thus don't have to purchase higher capacity during peak. This isn't too bad of a situation if this GPU provides 100% of your value proposition. but, if you needed your product to perform 5 such tasks - or competition/customer demand forces you to move to a bigger model with a 10 words per second inference rate then you may find it hard to balance customer value vs. price vs. margin.
- chx 6y ago
  > On A: Using a p3.2xl for that task leaves you with a minimum ec2 cost of goods sold of 50 cents per user per month for the target task https://twitter.com/br_/status/979442438254166016 https://twitter.com/br_/status/979442438254166016 > "selling AWS at a loss" is crisp shorthand for a lot of startups' business models!

YeGoblynQueenne 6y ago

Or, to paraphrase: great paper, no application.

SQueeeeeL 6y ago
It really is a great paper, the problem is when your entire field has few real applications but everyone is making too much money to admit that. Cool tech demos like this is awesome as a blog post, but not sure why it's academic...
- YeGoblynQueenne 6y ago
  Well, the paper has an experiment where they train an agent based on their system to play two of the three games (PacMan, PacManMaze and VizDoom) and compare its result to two other systems (an LSTM-based one and a model-based RL one, WorldModel). Their system gets the best score in Pacman but unfortunately the version that does better than WorldModel is the one that doesn't have the memory module whose usefulness is a central claim of their paper. WorldModel outperforms their system in VizDoom by ~300 points, though their system without the memory module "solves" VizDoom (i.e. scores just a bit over than 750; that is 765) and they claim that it's the first Gan-based system that achieves this. A bit thin of the ground (especially when it comes to supporting the claim about the usefulness of the memory module) but the way neural nets research and especially RL research is going (from what I've seen, not my area of expertise) that's the kind of result that you can expect will get your paper published these days.
- esoterica 6y ago
  I don't know where you got the idea that the purpose of an academic paper is to produce the schematics for a working commercial product. The fact that it's novel and interesting is sufficient grounds for producing a paper about it. Why do you think projects like these are solely the domain of blog posts?
  - YeGoblynQueenne 6y ago
    That is not true for academic papers in computer science and particularly in machine learning were contributions are expected to have some sort of practical application that is at least possible to foresee. Indeed, a common criticism against papers in machine learning is "what is a real-world application of this approach?". If you can't answer that convincingly then it's unlikely your paper gets published, particularly if it has rather mediocre results otherwise, as is the case with this paper (see my earlier comment about the comparison to the two other systems which is a bit of an anticlimax). In fact, the paper (well, the preprint version) makes a clear attempt to motivate the work with a practical application, saying that it aims to "learn a simulator [for robotic agents] by simply watching an agent interact with an environment".

esoterica 6y ago

Do you really think nVidia is seriously advocating for using neural networks as the best way to write a PacMan clone? It's obviously just a proof of concept to demonstrate that GANs are sophisticated enough to learn certain categories of models.

robotresearcher 6y ago
Doing these cute demos is a double-edged sword. A fun and charming demo attracts attention by non-experts who can't see the wood for the trees. In my field we make robots do a thing, say folding laundry, as an example of dextrous manipulation[1], and the popular conversation is about whether your laundry robot makes commercial sense and not about your novel grasping strategy and nifty motion planner that makes this sorta-possible for the first time. But everyone sees your video. Pros and cons. [1] laundry folding at Berkeley (not me) https://youtu.be/gy5g33S0Gzo https://youtu.be/gy5g33S0Gzo

empath75 6y ago

The ai has created a functional internal model of an external system based on limited sensory input. That is essentially a kind of consciousness.

eckza 6y ago
> The ai has created a functional internal model of an external system based on limited sensory input. Sure. > That is essentially a kind of consciousness. That is a flying leap.
101404 6y ago
Not at all. It just means some people poked a pile of a bazillion if-else statements long enough, until the desired result came out. I believe everyone here knows the xkcd I am referring to. Still, I think it's an interesting result. But, as someone on Fridman's podcast said the other day: recent advances in AI have all been thanks to advances in computing power, not any new ideas. The future if AI seems to depend on Moore's Law, more than anything.
- plutonorm 6y ago
  What makes you think the pile of if-else statements isn't conscious?
  - 101404 6y ago
    Because it's a pile of if-else statements shuffled until it was able to perform a single specialized thing. Any housefly would by a thousand times as "conscious".
    - bobly_today 6y ago
      What makes you so certain a house fly doesn’t have a form of rudimentary consciousness? It might be a spectrum.
      - 6y ago

SkyBelow 6y ago

It reminds me of older electric cars. They worked. They could get you around. But they did so at higher price, slower speeds, or other factors that a consumer wasn't willing to pay for. Thus they were regulated to novelty items that some work campuses would have and concept cars that would be shown from time to time. But eventually we reached the point where they are not only competitive, they may be positioned to fully overtake combustion cars. I wonder how many discoveries on electric cars ended up being fully dead ends along the path that took us to the point we are now?

rootusrootus 6y ago
It seems like there is not a lot of novel technology in an EV; what made them viable was steady improvements in battery technology so that the intersection between range, price, and weight is in a place that is financially feasible. Combined with continued improvements in battery, the already huge gain in fuel efficiency (for a typical car at least) will be what causes them to overtake ICEVs.
YeGoblynQueenne 6y ago
Electric cars of the era you describe could at least go. The approach described in this paper doesn't "go". Its goal is to "learn a simulator [to train a robotic agent] simply by watching an agent interacting with the environment". The learning part is done by training on the output of an existing simulator. Obviously, if you already have the output of a simulator, you don't need to learn a simulator so the only real use of this approach is to train a simulator by watching a human agent interacting with the environment. So the idea is to eventually apply this thing to training robots to interact in the real world by watching how humans do it (this is not clearly stated in the paper; it's my charitable interpretation of the very poor motivation of learning a simulator from the output of an existing simulator). The only problem is that creating an accurate simulation of a human's interaction with the real world (let alone an accurate simulation of the real world) is prohibitively expensive. Thus, for the time being, this kind of idea works only in very, very simple environments like PacMan. So we don't yet have an expensive prototype electic car that nevertheless can do everything an er other kind of car can do. We have... not sure what. The description of an idea with an example of how such a thing could be achieved in the far future, when we can train robots to act like humans by watching us.

fpgaminer 6y ago

> It's no longer practical to incorporate into real products given the hardware costs for training and inference I don't see that trend. Bleeding edge research has always been beyond what's practical, by its very nature. The focus is on pushing the boundaries, not to build something efficient/commercial. It's the followup work that makes it practical. See for example the evolution of WaveNet. It was breakthrough research, but yeah it wasn't practical. Now a couple years later and we're generating speech on mobile devices of similar quality (see the recent Facebook research), and able to teach these machines using orders of magnitude less data (see 15.ai). Not to mention the plethora of research that is explicitly about making neural networks more efficient and easier to train (e.g. EfficientNet; 1cycle; etc). Bleeding edge stuff like AlphaStar, BigGAN, GPT-2, etc are just that: bleeding edge. They'll get more efficient and practical every year, just like everything else in machine learning.

kevincox 6y ago
Exactly. It is sort of like saying the first transistor was too expensive to be practical. Development always starts with some novel technology that has major flaws, but some good nuggets. Then it is further developed to be practical.
ekelsen 6y ago
One minor nit: The recent FB blogpost wasn't really "research" - it was mostly a reimplementation of this paper: https://arxiv.org/abs/1802.08435 https://arxiv.org/abs/1802.08435

mysterEFrank 6y ago

This is obviously not meant to be a real game engine. It shows that GANs can learn not only images but transitions between images given some action, which means they can be used as simulators. See world-models which used VAEs https://worldmodels.github.io/ https://worldmodels.github.io/

srnvs123 6y ago

I don’t think that’s the point of bleeding edge research. Pushing boundries != commercialization potential

monadic2 6y ago
Is there research in this field that is not commercially funded? That seems like a silly distinction in America.

DeonPenny 6y ago

Researchers are not entrepreneurs. It's for the entrepreneurs and business to find a way to make this random game into something profitable.

eeZah7Ux 6y ago

> 3-4 orders of magnitude higher resource consumption Just 10000x? More like 6 orders of magnitude

anigbrowl 6y ago

Your comment reminds me of snide dismissals of personal computing in the 1980s - looks worse than [existing thing], very expensive, requires an army of technicians, probably explodes unpredictably, overall just a terrible investment.

angel_j 6y ago
Followed by decades long monopoly over expensive new technology, likely crushing waves of innovation and usage, in exchange for slow, buggy, not-even-automated spreadsheets; plus the hemming of governments and organizations into endless software rental contracts and platforms, that maybe don't add value as fast as Moore's Law.
- lumost 6y ago
  that's a fair criticism, however it's not a given that any product category will end up with the same growth curve of PCs, the web, cloud, and mobile. thinking back on the PC, Web, cloud, and mobile growth periods, each came with a transformative change in the capabilities, cost, and availability of computing power. We're only seeing two of these aspects in AI today - with the cost side heading in the wrong direction.
  - angel_j 6y ago
    AI supercharged the importance of computing power to arms-race levels. We're still just brute forcing it with Deep Learning, but the capabilities are too awesome to ignore. Therefore, compute power will be bought-up and hoarded expensively, and used on the highest bidder. Will the market save us, and simply over-clock production? Not happening yet. Corporate appetite should be quite large. I do A.I. R&D and am just watching the mountain of power ascend before me and aged gaming laptop. My next level is yet the foothills. Yeah, I'd say growth or not, A.I. is going where PC, web, cloud, everything, goes, to the industrial monopolists and the money spenders.
    - lumost 6y ago

m463 6y ago

Lots of new stuff sucks. The english longbow was vastly superior to the first guns that came out, which were slow to load, unreliable to shoot, didn't work in the rain, and were extremely inaccurate. 3d printed stuff sucks. It's not very strong, it's not very pretty and takes a long time to create. Meanwhile, take a look at this ML photo processing technique: https://www.youtube.com/watch?v=bcZFQ3f26pA https://www.youtube.com/watch?v=bcZFQ3f26pA

collyw 6y ago

"This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place!" True, though my first thought was that this would be interesting from a reverse engineering point of view. Then again Pacman is pretty simple concept. Something worth reverse engineering is probably a lot more complex.