3 ms·

> The post has counts for everything that went right and nothing for what could go wrong. That's AI for you. At Amazon we have many forums to share our AI win

> The post has counts for everything that went right and nothing for what could go wrong. That's AI for you. At Amazon we have many forums to share our AI wins, but none to share AI failures or disappoinments. No wonder execs make bad decisions regarding AI, they only hear completely one-sided stories.

Cthulhu_ 3mo ago
At my company there's a lot of discussion about AI and complaints that people run out of tokens within a day, but zero results are shown. No measurable (or measured) gains. Or nothing that people are willing to talk about, in any case.
- KptMarchewa 3mo ago
  At my company the performance improvements channels has been exploding, with people claiming giant improvements in latency, throughput, and decreased cost of the services. The cost decreases itself is order of magnitude more (at annualized run rate) more than we pay for tokens. YMMV.
  - jpc0 3mo ago
    I hope to see more of this across the industry. I definitely make use of AI but in my experience I almost always could have done it better myself, the places where I threw AI at the problem I didn't care about the results being good, only good enough. When we see memory and compute requirements for version x+1 of software decrease instead of increase I will happily say AI is the oracle people proclaim it to be.
- dgellow 3mo ago
  I’ve been asking since months on HN for proofs that companies using AI see positive ROI from it. So far I didn’t get a single concrete example
inigyou 3mo ago
"It can't be that stupid—you must be prompting it wrong." - Ed Zitron
frizlab 3mo ago
In $COMPANY, for the mid-yearly review, the employees were asked whether their AI usage was 1/ efficient, 2/ adoptive (integrated in the way they work) or 3/ transformative. Saying “never used it” or “I tried and it was useless” was literally not possible.
- tmp10423288442 3mo ago
  You'd have to be a pretty stubborn software engineer to not find AI useful for anything at this point, unless they're making you use Github Copilot or something far behind the frontier. You don't have repetitive tests to write? You never need to write a script to run something?
  - fwip 3mo ago
    And you'd have to be a pretty pig-headed exec to assume that the only experience an employee could possibly have with LLMs is positive.
  - frizlab 3mo ago
    Currently the only thing I personally find LLMs useful for are code reviews.
  - xigoi 3mo ago
    I don’t use AI for ethical reasons. I don’t want to support companies that are destroying the planet and inflating hardware prices.
rpdillon 3mo ago
This is kind of a weird take. If you were the CEO of Amazon, would you be setting up channels for people to talk about their AI failures? The general way technology is deployed is that we try to find ways to make it work, because those are the most interesting. We're not as interested in all the ways it doesn't work. From my perspective, some people are trying to use AI in the same way somebody might use a laptop to paddle a canoe. Sure, you can do it, but it's not a good idea. The fact that it doesn't work well is not particularly interesting. If I steelman your position, I guess the ideal repository would be a set of cases where it's known to work well and a set of cases where it's known to not work well. A little bit like ProtonDB or SteamDB, perhaps.
- iLoveOncall 3mo ago
  > If you were the CEO of Amazon, would you be setting up channels for people to talk about their AI failures? I would set up channels to talk about AI, encouraging both successes and failures, with proofs required for both, and punishing people who intentionally misreport on either.
  - rpdillon 3mo ago
    Wow. Setting up a forum for discussion where people could potentially be punished for what they say is not going to encourage discussion.
    - iLoveOncall 3mo ago
      That's fine, lies are more harmful than anecdotes are beneficial. There's no forum to discuss the best IDE and people still manage, in reality there's no need for a forum around AI.
      - rpdillon 3mo ago
        Well, you're at Amazon. Is the Dev Improvement discussion group still around? I feel like that was a pretty good place to learn about neat stuff (in my case, Emacs-related). I enjoyed the discussion.
warkdarrior 3mo ago
> At Amazon we have many forums [...], but none to share AI failures or disappoinments. I come to HN to read about AI failures and disappointments.

Cthulhu_ 3mo ago

At my company there's a lot of discussion about AI and complaints that people run out of tokens within a day, but zero results are shown. No measurable (or measured) gains. Or nothing that people are willing to talk about, in any case.

KptMarchewa 3mo ago
At my company the performance improvements channels has been exploding, with people claiming giant improvements in latency, throughput, and decreased cost of the services. The cost decreases itself is order of magnitude more (at annualized run rate) more than we pay for tokens. YMMV.
- jpc0 3mo ago
  I hope to see more of this across the industry. I definitely make use of AI but in my experience I almost always could have done it better myself, the places where I threw AI at the problem I didn't care about the results being good, only good enough. When we see memory and compute requirements for version x+1 of software decrease instead of increase I will happily say AI is the oracle people proclaim it to be.
dgellow 3mo ago
I’ve been asking since months on HN for proofs that companies using AI see positive ROI from it. So far I didn’t get a single concrete example

inigyou 3mo ago

"It can't be that stupid—you must be prompting it wrong." - Ed Zitron

frizlab 3mo ago

In $COMPANY, for the mid-yearly review, the employees were asked whether their AI usage was 1/ efficient, 2/ adoptive (integrated in the way they work) or 3/ transformative. Saying “never used it” or “I tried and it was useless” was literally not possible.

tmp10423288442 3mo ago
You'd have to be a pretty stubborn software engineer to not find AI useful for anything at this point, unless they're making you use Github Copilot or something far behind the frontier. You don't have repetitive tests to write? You never need to write a script to run something?
- fwip 3mo ago
  And you'd have to be a pretty pig-headed exec to assume that the only experience an employee could possibly have with LLMs is positive.
- frizlab 3mo ago
  Currently the only thing I personally find LLMs useful for are code reviews.
- xigoi 3mo ago
  I don’t use AI for ethical reasons. I don’t want to support companies that are destroying the planet and inflating hardware prices.

rpdillon 3mo ago

This is kind of a weird take. If you were the CEO of Amazon, would you be setting up channels for people to talk about their AI failures? The general way technology is deployed is that we try to find ways to make it work, because those are the most interesting. We're not as interested in all the ways it doesn't work. From my perspective, some people are trying to use AI in the same way somebody might use a laptop to paddle a canoe. Sure, you can do it, but it's not a good idea. The fact that it doesn't work well is not particularly interesting. If I steelman your position, I guess the ideal repository would be a set of cases where it's known to work well and a set of cases where it's known to not work well. A little bit like ProtonDB or SteamDB, perhaps.

iLoveOncall 3mo ago
> If you were the CEO of Amazon, would you be setting up channels for people to talk about their AI failures? I would set up channels to talk about AI, encouraging both successes and failures, with proofs required for both, and punishing people who intentionally misreport on either.
- rpdillon 3mo ago
  Wow. Setting up a forum for discussion where people could potentially be punished for what they say is not going to encourage discussion.
  - iLoveOncall 3mo ago
    That's fine, lies are more harmful than anecdotes are beneficial. There's no forum to discuss the best IDE and people still manage, in reality there's no need for a forum around AI.
    - rpdillon 3mo ago
      Well, you're at Amazon. Is the Dev Improvement discussion group still around? I feel like that was a pretty good place to learn about neat stuff (in my case, Emacs-related). I enjoyed the discussion.

warkdarrior 3mo ago

> At Amazon we have many forums [...], but none to share AI failures or disappoinments. I come to HN to read about AI failures and disappointments.