3 ms·

I've spent 10 years implementing new technology into businesses from all over the world and the thing that has killed the most projects at the proof of concept

I've spent 10 years implementing new technology into businesses from all over the world and the thing that has killed the most projects at the proof of concept phase has not been whether it works. It's whether the cost of going to production from PoC produces a significant ROI. AI is absolutely changing the economics there. Humans were simply nowhere near answering all the useful questions they had which software could answer. The problem was that software was expensive. Now, software is cheaper and so people who couldn't afford to answer questions they've long wondered about can afford to answer them.

HumblyTossed 27d ago
> Now, software is cheaper and so people who couldn't afford to answer questions they've long wondered about can afford to answer them. Temporarily. Token price will rise sharply when the bills finally come due.
- Havoc 27d ago
  No longer convinced that’s true. Some of the indebted providers might go under but there is nothing preventing someone from just setting up a new provider and serving tokens debt free. GLM or whatever. That provides continuous downward pressure on token prices even if it isn’t Astra level
  - scott_weber 27d ago
    Agree model providers can't take much margin, a few percent plus maybe a bit extra from those willing to pay for a "better" model. Upstream is more concentrated: TSMC, NVIDIA, AMD could maybe raise prices and capture more of the value, which would affect open model providers.
- jvwww 27d ago
  Do you have any evidence to suggest that this true? And if so, why can't I just use any of the very good open source Chinese models (and eventually American once reflection, thinking machines, etc catch up).
  - HumblyTossed 27d ago
    Look around? It's rape and pillage time for corporations.
  - DubiousPusher 23d ago
    Exactly. You can buy on server grade mobo and CPU, stock it with ddr4 and run qwen and basically get gpt 3+ level quality now. A year from now, I imagine we'll see even better efficiency.
ProllyInfamous 27d ago
>people who couldn't afford to answer questions they've long wondered about can afford to answer them. If the software isn't downright free – today, you can slap a $349 RTX 5060 [†] into any POS surplus computer and have an offline assistant, running Llama or Ornith, via Ollama in linux (i.e. the LLM and OS are FREE open-source software) [∑] I have a working demo of this that has been shown/leant to several friends, and they're all surprised that it "all works so well, without being online, for only a few hundred dollars." When I've demonstrated Mistral-small on my 5070Ti... that has been a real jaw dropper. I haven't shown anybody qwen3.8:27b, yet... but dammmmm, what a month of learning it's been. ---- A friend that was going through wifecancer confided in me that "you can ask it anything, without feeling embarassed" – and that has stuck with me (that so many smart people are afraid to ask simple questions [*], out of perceptional worries). ---- [†] (8GB DDR7) brand new from Wal-Mart [∑] My first linux/LLM machine was built with setup help from Perplexity.ai (with a dozen "assists" - I am bluecollar, non-coder). This used an obsolete i5 (and $200 used VEGA64 GPU) to create a decent Llama3.1 LLM machine (~100wpm typeback, perhaps 70 tokens/s). [*] even to their own detriment, of shame, when simple solutions often do exist

HumblyTossed 27d ago

> Now, software is cheaper and so people who couldn't afford to answer questions they've long wondered about can afford to answer them. Temporarily. Token price will rise sharply when the bills finally come due.

Havoc 27d ago
No longer convinced that’s true. Some of the indebted providers might go under but there is nothing preventing someone from just setting up a new provider and serving tokens debt free. GLM or whatever. That provides continuous downward pressure on token prices even if it isn’t Astra level
- scott_weber 27d ago
  Agree model providers can't take much margin, a few percent plus maybe a bit extra from those willing to pay for a "better" model. Upstream is more concentrated: TSMC, NVIDIA, AMD could maybe raise prices and capture more of the value, which would affect open model providers.
jvwww 27d ago
Do you have any evidence to suggest that this true? And if so, why can't I just use any of the very good open source Chinese models (and eventually American once reflection, thinking machines, etc catch up).
- HumblyTossed 27d ago
  Look around? It's rape and pillage time for corporations.
- DubiousPusher 23d ago
  Exactly. You can buy on server grade mobo and CPU, stock it with ddr4 and run qwen and basically get gpt 3+ level quality now. A year from now, I imagine we'll see even better efficiency.

ProllyInfamous 27d ago

>people who couldn't afford to answer questions they've long wondered about can afford to answer them. If the software isn't downright free – today, you can slap a $349 RTX 5060 [†] into any POS surplus computer and have an offline assistant, running Llama or Ornith, via Ollama in linux (i.e. the LLM and OS are FREE open-source software) [∑] I have a working demo of this that has been shown/leant to several friends, and they're all surprised that it "all works so well, without being online, for only a few hundred dollars." When I've demonstrated Mistral-small on my 5070Ti... that has been a real jaw dropper. I haven't shown anybody qwen3.8:27b, yet... but dammmmm, what a month of learning it's been. ---- A friend that was going through wifecancer confided in me that "you can ask it anything, without feeling embarassed" – and that has stuck with me (that so many smart people are afraid to ask simple questions [*], out of perceptional worries). ---- [†] (8GB DDR7) brand new from Wal-Mart [∑] My first linux/LLM machine was built with setup help from Perplexity.ai (with a dozen "assists" - I am bluecollar, non-coder). This used an obsolete i5 (and $200 used VEGA64 GPU) to create a decent Llama3.1 LLM machine (~100wpm typeback, perhaps 70 tokens/s). [*] even to their own detriment, of shame, when simple solutions often do exist