11 ms·

I've revised the naivety out of their 3 points at the beginning of the article -------- In particular, many AI companies have: 1. Lower gross margins due to

I've revised the naivety out of their 3 points at the beginning of the article -------- In particular, many AI companies have: 1. Lower gross margins due to heavy cloud infrastructure usage and ongoing human support ---> Their product is actually an army of expensive humans 2. Scaling challenges due to the thorny problem of edge cases ---> Their products don't actually work 3. Weaker defensive moats due to the commoditization of AI models and challenges with data network effects ---> No truely valuable technology or systems

deleted 7y ago
[deleted]
ec109685 7y ago
#1 isn’t saying that at all. Cloud bills are expensive for an early stage startup in the AI space. They later touch on the human angle in the post, but not fair to diminish their cloud bill point. 2. No, they don’t work in all cases. Think Tesla autopilot versus a Level 5 car without a steering wheel. 3. True. I wonder if more ai innovation will start happening behind close doors as a result?
- mkolodny 7y ago
  Regarding #3, we still have so far to go to get to human-level intelligence that I think companies benefit more from open research than closing their doors. When a company publishes their research, others can find and publish improvements to their research. Then, the company can use the published improvements to improve their product.
  - zitterbewegung 7y ago
    Have you ever tried to reproduce research from academics and or companies ? Some papers take more than a day of work to reproduce . Not only that if you publish the research without the dataset it’s practically impossible or extremely expensive . Doing GPT-2 from scratch is around $50k or compute time and data collection. Even a simple deep learning paper will require you to have at least a NVIDIA 1080ti if you are lucky and for NLP I needed to buy a RTX Titan ($2500 graphics card).
    - nl 7y ago
      Some papers take more than a day of work to reproduce A large part of my job is this. Generally reproducing a paper with no code is months of work. Doing GPT-2 from scratch is around $50k or compute time and data collection. It's extraordinarily rare you need to do this though. I know a few who have (mostly for foreign languages) and all have been able to access TPU grants from Google or multi-node GPU clusters (which are pretty easy to find if you work in the field - plenty of vendors want someone to test their "supercomputer" for a a week) Even a simple deep learning paper will require you to have at least a NVIDIA 1080ti if you are lucky and for NLP I needed to buy a RTX Titan ($2500 graphics card Most of my work is in NLP and I do a large amount of it on a 1070.
      - zitterbewegung 7y ago
        Yea GPT2 requires a bit more ram than on the 10 or 20 aeries TI cards . My point I think was unclear was the issue of reproducing the paper from scratch (I guess the model is enough ?) not sure how those papers are peer reviewed unless they also send the dataset to the reviewers ? I also was unclear was when I said reproducing I mean to just get a model that has already been pretrained. I agree with your points .
        nl 7y ago
        Peer review almost never means reproducing the model, but that isn't because of the dataset (which is usually available to the reviewer) but because that's not what a peer review is! A peer review isn't an adversarial process where you think that the person has done something wrong. Instead, it's extra eyes on it to make sure they have thought of everything, and to say what additional tests might be needed and why.
    - itsmefaz 7y ago
      This is a very interesting point! If this is true then it would mean that independent researchers will have a very tough time producing quality research without sufficient funding.
hn_throwaway_99 7y ago
Another way I'd put it: "SaaS products are technically really quite easy. AI is actually hard".
- joe_the_user 7y ago
  It's relatively easy to put together a neural network that works as advertised with test data and sort-of works with a good amount of real world data. That's not the problem. The actual problem is the basically what GP actually said; Making a working neural network doesn't mean it provides any value in the real world.
ssivark 7y ago
It’s important to keep in mind that most of the AI “innovation”/hype is coming from a handful of companies looking to make money from cloud computing or whose moat is built on data. The good old story of making money in a gold rush by selling tools... I would venture that the most hyped stuff (deep learning) doesn’t really work very well in practice, and the stuff that works reliably well is boring enough to barely register on the hype train as “AI”. To be fair, that’s probably an exaggerated caricature, but that is my reading between the lines of SOTA AI results. Very few people (even among ML researchers) appreciate the fundamental limitations of associational reasoning (rather than causal reasoning). It’s going to be an interesting time when this message actually sinks in...
- 2sk21 7y ago
  I really like your point of this being "associational learning". There is a lot more to intelligence than just learning associations
- redisman 7y ago
  Hah too true. Amazon, Nvidia, Coursera, etc. are making most of the money and the "AI startups" along with their investors are paying it.
zelly 7y ago
The biggest money in AI is starting online courses.
- redisman 7y ago
  Or selling GPU instances by the hour

deleted 7y ago

[deleted]

ec109685 7y ago

#1 isn’t saying that at all. Cloud bills are expensive for an early stage startup in the AI space. They later touch on the human angle in the post, but not fair to diminish their cloud bill point. 2. No, they don’t work in all cases. Think Tesla autopilot versus a Level 5 car without a steering wheel. 3. True. I wonder if more ai innovation will start happening behind close doors as a result?

mkolodny 7y ago
Regarding #3, we still have so far to go to get to human-level intelligence that I think companies benefit more from open research than closing their doors. When a company publishes their research, others can find and publish improvements to their research. Then, the company can use the published improvements to improve their product.
- zitterbewegung 7y ago
  Have you ever tried to reproduce research from academics and or companies ? Some papers take more than a day of work to reproduce . Not only that if you publish the research without the dataset it’s practically impossible or extremely expensive . Doing GPT-2 from scratch is around $50k or compute time and data collection. Even a simple deep learning paper will require you to have at least a NVIDIA 1080ti if you are lucky and for NLP I needed to buy a RTX Titan ($2500 graphics card).
  - nl 7y ago
    Some papers take more than a day of work to reproduce A large part of my job is this. Generally reproducing a paper with no code is months of work. Doing GPT-2 from scratch is around $50k or compute time and data collection. It's extraordinarily rare you need to do this though. I know a few who have (mostly for foreign languages) and all have been able to access TPU grants from Google or multi-node GPU clusters (which are pretty easy to find if you work in the field - plenty of vendors want someone to test their "supercomputer" for a a week) Even a simple deep learning paper will require you to have at least a NVIDIA 1080ti if you are lucky and for NLP I needed to buy a RTX Titan ($2500 graphics card Most of my work is in NLP and I do a large amount of it on a 1070.
    - zitterbewegung 7y ago
      Yea GPT2 requires a bit more ram than on the 10 or 20 aeries TI cards . My point I think was unclear was the issue of reproducing the paper from scratch (I guess the model is enough ?) not sure how those papers are peer reviewed unless they also send the dataset to the reviewers ? I also was unclear was when I said reproducing I mean to just get a model that has already been pretrained. I agree with your points .
      - nl 7y ago
        Peer review almost never means reproducing the model, but that isn't because of the dataset (which is usually available to the reviewer) but because that's not what a peer review is! A peer review isn't an adversarial process where you think that the person has done something wrong. Instead, it's extra eyes on it to make sure they have thought of everything, and to say what additional tests might be needed and why.
  - itsmefaz 7y ago
    This is a very interesting point! If this is true then it would mean that independent researchers will have a very tough time producing quality research without sufficient funding.

hn_throwaway_99 7y ago

Another way I'd put it: "SaaS products are technically really quite easy. AI is actually hard".

joe_the_user 7y ago
It's relatively easy to put together a neural network that works as advertised with test data and sort-of works with a good amount of real world data. That's not the problem. The actual problem is the basically what GP actually said; Making a working neural network doesn't mean it provides any value in the real world.

ssivark 7y ago

It’s important to keep in mind that most of the AI “innovation”/hype is coming from a handful of companies looking to make money from cloud computing or whose moat is built on data. The good old story of making money in a gold rush by selling tools... I would venture that the most hyped stuff (deep learning) doesn’t really work very well in practice, and the stuff that works reliably well is boring enough to barely register on the hype train as “AI”. To be fair, that’s probably an exaggerated caricature, but that is my reading between the lines of SOTA AI results. Very few people (even among ML researchers) appreciate the fundamental limitations of associational reasoning (rather than causal reasoning). It’s going to be an interesting time when this message actually sinks in...

2sk21 7y ago
I really like your point of this being "associational learning". There is a lot more to intelligence than just learning associations
redisman 7y ago
Hah too true. Amazon, Nvidia, Coursera, etc. are making most of the money and the "AI startups" along with their investors are paying it.

zelly 7y ago

The biggest money in AI is starting online courses.

redisman 7y ago
Or selling GPU instances by the hour