8 ms·

Reminds me of this article from two years ago [0] and my HN comment on it. Yet another AI startup on the general trajectory of: 1) Someone runs into an interes

by jcalx 2y ago

Reminds me of this article from two years ago [0] and my HN comment on it. Yet another AI startup on the general trajectory of: 1) Someone runs into an interesting problem that can potentially be solved with ML/AI. They try to solve it for themselves. 2) "Hey! The model is kind of working. It's useful enough that I bet other people would pay for it." 3) They launch a paid API, SaaS startup, etc. and get a few paying customers. 4) Turns out their ML/AI method doesn't generalize so well. Reputation is everything at this level, so they hire some human workers to catch and fix the edge cases that end up badly. They tell themselves that they can also use it to train and improve the model. 5) Uh-oh, the model is underperforming, and the human worker pipeline is now some significant part of the full workflow. 6) Then someone writes an article about them using cheap human labor. [0] https://news.ycombinator.com/item?id=37405450 https://news.ycombinator.com/item?id=37405450

deleted 2y ago
[deleted]
Digory 2y ago
I'd think ambiguous statements about the scope of your AI would make it hard to prove fraud, if you were being careful at all. "Involving AI" could mean 1% AI. So it's doubly surprising to me the government chose (criminal) wire fraud, not (civil) securities fraud, which would have a lower burden of proof. Government lawyers almost never try to make their job harder than it has to be.
- tbrownaw 2y ago
  If you click through to the doj press release, they're saying the statements were pretty explicit.
  - pseudo0 2y ago
    Yeah, specifying an automation rate of 93-97% to investors when it's "effectively 0%" per your own executives... That's pretty egregious.
    - torginus 2y ago
      How do you define that? If I write a 'Hello World' program in C++, you could argue that the hard part of compiling, linking, and generating assembly code was done by a computer, so programming is 90% automated, even though most people would understand the automation level to be 0%. You might argue this is a flawed example, but we've automated huge workflows at work that turned major time-consuming PITAs into something it wouldn't occur to most people that a human has anything to do with it.
      - hugh-avherald 2y ago
        If "most people would understand the automation level to be 0%" then you can't represent that the automation level is something else, unless you're explicit about deviating from the commonly understood meaning of 'automation'.
        torginus 2y ago
        The problem with intuition is that you have to be familiar with the domain to have it. You and I have zero intuition on what needs to be or can be done by humans and what can be handed off to machines in this financial domain.
      - mandevil 2y ago
        Law does not work like engineering does. Lawyers, judges and juries understand the intent of the law, and are not bound like we software engineers are to the exact commands in front of them. You could try to convince a jury of this argument, sure. Do you think it will work? And if you do go with that argument then are you actually convincing the jury of your guilty conscience- often an important part of a white collar crime where state of mind of the defendant is very important?
        eigen 2y ago
        > Lawyers, judges and juries understand the intent of the law, and are not bound like we software engineers are to the exact commands in front of them. a good example is O'Connor v. Oakhurst Dairy, No. 16-1901, also known as the Maine Dairy oxford comma case. the District Court followed the intent but the Appeals court followed the law as written. https://www.smithsonianmag.com/smart-news/missing-oxford-comma-just-changed-course-court-case-180962551/ https://www.smithsonianmag.com/smart-news/missing-oxford-com... from the Appeals Court ruling > The District Court concluded that, despite the absent comma, the Maine legislature unambiguously intended for the last term in the exemption's list of activities to identify an exempt activity in its own right. The District Court thus granted summary judgment to the dairy company, as there is no dispute that the drivers do perform that activity. But, we conclude that the exemption's scope is actually not so clear in this regard. https://cases.justia.com/federal/appellate-courts/ca1/16-1901/16-1901-2017-03-13.pdf?ts=1489437006 https://cases.justia.com/federal/appellate-courts/ca1/16-190...
A4ET8a8uTh0_v2 2y ago
To be perfectly honest, I am more amazed that it was a valid business model and people were willing not just invest in it, but offer their rather personal information to an unaffiliated third party.
mvdtnz 2y ago
I think you're being excessively generous. According to the linked article, > But despite Nate acquiring some AI technology and hiring data scientists, its app’s actual automation rate was effectively 0%, the DOJ claims. Sometimes people are just dishonest. And when those people use their dishonestly to fleece real people, they belong in prison.
Lerc 2y ago
Going back earlier, a similar thing in 2017 was done. https://thespinoff.co.nz/the-best-of/06-03-2018/the-mystery-of-zach-new-zealands-all-too-miraculous-medical-ai https://thespinoff.co.nz/the-best-of/06-03-2018/the-mystery-... Interestingly this was a task that could probably be done well enough by AI now. Not that these guys knew how close to reality they turned out to be. I assume they just had no idea of the problem they were attempting and assumed that it was at the geotaging a photo end of the scale when it was at the 'is it a bird' end. Maybe I'm being overly optimistic in assuming people who do this are honestly attempting to solve the problem and fudging it to buy time. In general they seem more deluded about their abilities than planning a con from start to finish.
petesergeant 2y ago
Honestly I think the only real problem here is if you then raise further money claiming you've solved the problem when you haven't, which is also where this particular startup comes unstuck
jjmarr 2y ago
> Reputation is everything at this level, so they hire some human workers to catch and fix the edge cases that end up badly. The most important part of your reputation is admitting fault. Sometimes your product isn't perfect. Lying to your investors about automation rates is far worse for your reputation than just taking the L.
- chii 2y ago
  The expectation is that the startup lies until they make it. It isn't too dissimilar to theranos.
  - Ancalagon 2y ago
    Or Uber. Or Tesla. Or Amazon Go.
    - nyclounge 2y ago
      They are just heavily subsidized!!! We ought to just view and treat them as defense contractors and 3 letter agencies!
  - 52-6F-62 2y ago
    What is making it, in these cases? Monopoly? IPO? Exit and leave the bags with someone else? This is bloody absurd
- siva7 2y ago
  Literally every founder story disproves your theory
  - johnnyanmac 2y ago
    Okay slight correction: |lying to shareholders is okay until they start losing money. Then it's worse than admitting fault.
    - 2y ago
palmotea 2y ago
> 5) Uh-oh, the model is underperforming, and the human worker pipeline is now some significant part of the full workflow. AI stands for "Actually, Indians."
- siva7 2y ago
  [flagged]
  - johnisgood 2y ago
    It is reality. Reality cannot be racist. :P
    - rightbyte 2y ago
      Uhm.. Do you include human social structures in reality? Even of you replace 'reality' with 'true' amd 'truth' the logic doesn't quite work out.
      - johnisgood 2y ago
        Others have already mentioned why it is grounded in reality. Call your ISP, Indian picks up. You are being scammed? Probably an Indian. It is not racist, I have nothing against Indians in particular, but the trend is there and it is quite obvious, hence, reality. If I replace "reality" with "truth", it does not work out because it makes no sense: "truth cannot be racist" makes no sense whatsoever. In relation or correspondence to what? It does work with reality, however.
        rightbyte 2y ago
        I am trying to argue that something existing in reality, does not make it not rascist. Like say apartheid or whatever. As a comment on "Reality cannot be racist".
        johnisgood 2y ago
aucisson_masque 2y ago
> its app’s actual automation rate was effectively 0%, the DOJ claims. In that case, I believe it's a scam. 0% isn't some edge case.
ohgr 2y ago
This is what we did internally. Someone said we could use LLMs for helping engineering teams solve production issues. Turned out it was just a useless tar pit. End game is we outsourced it. Neither of these solved the problem that our stack is a pile of cat shit and needs some maintenance from people who know what the hell they are doing. It’s not solving a problem. It’s adding another layer of cat shit.
baxtr 2y ago
tbh I don’t think any one except for investors care how you deliver a service as long as quality and price are right.
belter 2y ago
> Uh-oh, the model is underperforming, and the human worker pipeline is now some significant part of the full workflow. Tesla robots and Taxis enter the room...
claiir 2y ago
In this case it's a little bit worse; the "nate" app had a literally "0% automation rate," despite representations to investors of an "AI" automation rate of "93-97%" powered by "LSTMs, NLP, and RL." No ML model ever existed! [1] See: > As SANIGER knew, at the time nate was claiming to use AI to automate online purchases, the app’s actual automation rate was effectively 0%. SANIGER concealed that reality from investors and most nate employees: he told employees to keep nate’s automation rate secret; he restricted access to nate’s “automation rate dashboard,” which displayed automation metrics; and he provided false explanations for his secrecy, such as the automation data was a “trade secret.” > SANIGER claimed that nate's "deep learning models" were "custom built" and use a "mix of long short-term memory, natural language processing, and reinforcement learning." > When, on the eve of making an investment, an employee of Investment Firm-1 asked SANIGER about nate's automation rate, that is, the percentage of transactions successfully completed with nate's AI technology, SANIGER claimed that internal testing showed that "success ranges from 93% to 97%." (from [1]) [1]: https://www.justice.gov/usao-sdny/media/1396131/dl?inline https://www.justice.gov/usao-sdny/media/1396131/dl?inline
mvkel 1y ago
> Turns out their ML/AI method doesn't generalize so well. I'd argue the opposite. AI typically generalizes very well. What it can't do well is specifics. It can't do the same thing over and over and follow every detail. That's what's surprised me about so many of these startups. They're looking at it from the bottom-up, something ai is uniquely bad at.

by jcalx 2y ago

deleted 2y ago

[deleted]

Digory 2y ago

I'd think ambiguous statements about the scope of your AI would make it hard to prove fraud, if you were being careful at all. "Involving AI" could mean 1% AI. So it's doubly surprising to me the government chose (criminal) wire fraud, not (civil) securities fraud, which would have a lower burden of proof. Government lawyers almost never try to make their job harder than it has to be.

tbrownaw 2y ago
If you click through to the doj press release, they're saying the statements were pretty explicit.
- pseudo0 2y ago
  Yeah, specifying an automation rate of 93-97% to investors when it's "effectively 0%" per your own executives... That's pretty egregious.
  - torginus 2y ago
    How do you define that? If I write a 'Hello World' program in C++, you could argue that the hard part of compiling, linking, and generating assembly code was done by a computer, so programming is 90% automated, even though most people would understand the automation level to be 0%. You might argue this is a flawed example, but we've automated huge workflows at work that turned major time-consuming PITAs into something it wouldn't occur to most people that a human has anything to do with it.
    - hugh-avherald 2y ago
      If "most people would understand the automation level to be 0%" then you can't represent that the automation level is something else, unless you're explicit about deviating from the commonly understood meaning of 'automation'.
      - torginus 2y ago
        The problem with intuition is that you have to be familiar with the domain to have it. You and I have zero intuition on what needs to be or can be done by humans and what can be handed off to machines in this financial domain.
    - mandevil 2y ago
      Law does not work like engineering does. Lawyers, judges and juries understand the intent of the law, and are not bound like we software engineers are to the exact commands in front of them. You could try to convince a jury of this argument, sure. Do you think it will work? And if you do go with that argument then are you actually convincing the jury of your guilty conscience- often an important part of a white collar crime where state of mind of the defendant is very important?
      - eigen 2y ago
        > Lawyers, judges and juries understand the intent of the law, and are not bound like we software engineers are to the exact commands in front of them. a good example is O'Connor v. Oakhurst Dairy, No. 16-1901, also known as the Maine Dairy oxford comma case. the District Court followed the intent but the Appeals court followed the law as written. https://www.smithsonianmag.com/smart-news/missing-oxford-comma-just-changed-course-court-case-180962551/ https://www.smithsonianmag.com/smart-news/missing-oxford-com... from the Appeals Court ruling > The District Court concluded that, despite the absent comma, the Maine legislature unambiguously intended for the last term in the exemption's list of activities to identify an exempt activity in its own right. The District Court thus granted summary judgment to the dairy company, as there is no dispute that the drivers do perform that activity. But, we conclude that the exemption's scope is actually not so clear in this regard. https://cases.justia.com/federal/appellate-courts/ca1/16-1901/16-1901-2017-03-13.pdf?ts=1489437006 https://cases.justia.com/federal/appellate-courts/ca1/16-190...

A4ET8a8uTh0_v2 2y ago

To be perfectly honest, I am more amazed that it was a valid business model and people were willing not just invest in it, but offer their rather personal information to an unaffiliated third party.

mvdtnz 2y ago

I think you're being excessively generous. According to the linked article, > But despite Nate acquiring some AI technology and hiring data scientists, its app’s actual automation rate was effectively 0%, the DOJ claims. Sometimes people are just dishonest. And when those people use their dishonestly to fleece real people, they belong in prison.

Lerc 2y ago

Going back earlier, a similar thing in 2017 was done. https://thespinoff.co.nz/the-best-of/06-03-2018/the-mystery-of-zach-new-zealands-all-too-miraculous-medical-ai https://thespinoff.co.nz/the-best-of/06-03-2018/the-mystery-... Interestingly this was a task that could probably be done well enough by AI now. Not that these guys knew how close to reality they turned out to be. I assume they just had no idea of the problem they were attempting and assumed that it was at the geotaging a photo end of the scale when it was at the 'is it a bird' end. Maybe I'm being overly optimistic in assuming people who do this are honestly attempting to solve the problem and fudging it to buy time. In general they seem more deluded about their abilities than planning a con from start to finish.

petesergeant 2y ago

Honestly I think the only real problem here is if you then raise further money claiming you've solved the problem when you haven't, which is also where this particular startup comes unstuck

jjmarr 2y ago

> Reputation is everything at this level, so they hire some human workers to catch and fix the edge cases that end up badly. The most important part of your reputation is admitting fault. Sometimes your product isn't perfect. Lying to your investors about automation rates is far worse for your reputation than just taking the L.

chii 2y ago
The expectation is that the startup lies until they make it. It isn't too dissimilar to theranos.
- Ancalagon 2y ago
  Or Uber. Or Tesla. Or Amazon Go.
  - nyclounge 2y ago
    They are just heavily subsidized!!! We ought to just view and treat them as defense contractors and 3 letter agencies!
- 52-6F-62 2y ago
  What is making it, in these cases? Monopoly? IPO? Exit and leave the bags with someone else? This is bloody absurd
siva7 2y ago
Literally every founder story disproves your theory
- johnnyanmac 2y ago
  Okay slight correction: |lying to shareholders is okay until they start losing money. Then it's worse than admitting fault.
  - 2y ago

palmotea 2y ago

> 5) Uh-oh, the model is underperforming, and the human worker pipeline is now some significant part of the full workflow. AI stands for "Actually, Indians."

siva7 2y ago
[flagged]
- johnisgood 2y ago
  It is reality. Reality cannot be racist. :P
  - rightbyte 2y ago
    Uhm.. Do you include human social structures in reality? Even of you replace 'reality' with 'true' amd 'truth' the logic doesn't quite work out.
    - johnisgood 2y ago
      Others have already mentioned why it is grounded in reality. Call your ISP, Indian picks up. You are being scammed? Probably an Indian. It is not racist, I have nothing against Indians in particular, but the trend is there and it is quite obvious, hence, reality. If I replace "reality" with "truth", it does not work out because it makes no sense: "truth cannot be racist" makes no sense whatsoever. In relation or correspondence to what? It does work with reality, however.
      - rightbyte 2y ago
        I am trying to argue that something existing in reality, does not make it not rascist. Like say apartheid or whatever. As a comment on "Reality cannot be racist".
        johnisgood 2y ago

aucisson_masque 2y ago

> its app’s actual automation rate was effectively 0%, the DOJ claims. In that case, I believe it's a scam. 0% isn't some edge case.

ohgr 2y ago

This is what we did internally. Someone said we could use LLMs for helping engineering teams solve production issues. Turned out it was just a useless tar pit. End game is we outsourced it. Neither of these solved the problem that our stack is a pile of cat shit and needs some maintenance from people who know what the hell they are doing. It’s not solving a problem. It’s adding another layer of cat shit.

baxtr 2y ago

tbh I don’t think any one except for investors care how you deliver a service as long as quality and price are right.

belter 2y ago

> Uh-oh, the model is underperforming, and the human worker pipeline is now some significant part of the full workflow. Tesla robots and Taxis enter the room...

claiir 2y ago

In this case it's a little bit worse; the "nate" app had a literally "0% automation rate," despite representations to investors of an "AI" automation rate of "93-97%" powered by "LSTMs, NLP, and RL." No ML model ever existed! [1] See: > As SANIGER knew, at the time nate was claiming to use AI to automate online purchases, the app’s actual automation rate was effectively 0%. SANIGER concealed that reality from investors and most nate employees: he told employees to keep nate’s automation rate secret; he restricted access to nate’s “automation rate dashboard,” which displayed automation metrics; and he provided false explanations for his secrecy, such as the automation data was a “trade secret.” > SANIGER claimed that nate's "deep learning models" were "custom built" and use a "mix of long short-term memory, natural language processing, and reinforcement learning." > When, on the eve of making an investment, an employee of Investment Firm-1 asked SANIGER about nate's automation rate, that is, the percentage of transactions successfully completed with nate's AI technology, SANIGER claimed that internal testing showed that "success ranges from 93% to 97%." (from [1]) [1]: https://www.justice.gov/usao-sdny/media/1396131/dl?inline https://www.justice.gov/usao-sdny/media/1396131/dl?inline

mvkel 1y ago

> Turns out their ML/AI method doesn't generalize so well. I'd argue the opposite. AI typically generalizes very well. What it can't do well is specifics. It can't do the same thing over and over and follow every detail. That's what's surprised me about so many of these startups. They're looking at it from the bottom-up, something ai is uniquely bad at.