5 ms·

I think this is a good example of the VC mindset, but I think it is also flawed on their part. LLMs are a lot more like a generalized processor than people are

I think this is a good example of the VC mindset, but I think it is also flawed on their part. LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added.

LordDragonfang 3y ago
>when you learn how to program it I don't know if you mean, like, LoRAs and similar (actual substantive changes), but the vast majority of "learning how to program" LLMs (accounting for the majority of startup pitches as well) is "prompt engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of English prose, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
- tsunamifury 3y ago
  This is the error of the thinking. It would be like saying software doesn't have a moat because thats just clever talking to a processor. But no one would say that now, thats ridiculous. There is a sufficient degree of prompt engineering that is already defensible, I'm already doing it myself IMO. You'll see very sophisticated hybrid programming/prompting systems being developed in the next year that will prove out the case. For example 30 parallel prompts that then amalgamate into a decision and an audit, with 10 simulation level prompts running chained afterwards to clean the output. These types of atomic configurations will become sufficiently complex to not be just for 'anybody'.
  - __loam 3y ago
    The output of that many chained prompts is probably so unreliable that it's useless.
    - tsunamifury 3y ago
      Again, this is a misunderstanding of what LLMs are capable of. These aren't chained, you can run parallel prompts of 15 personas with X diversity of perspective, that reason on a singular request, string, input or variable, they provide output plus audit explanation. You then run an amalgamation or committee decision (sort of like mixture of experts) on it to output variable or string. Then run parallel simulation or reflection prompts based on X different context personas to double check their application to outside cases, unconsidered context, etc. It's pretty effective on complex problems like Spam, Trust and Safety, etc. And the applications of these sort of reasoning atomic configurations I think are unlimited. It's not just 'talking fancy' to an AI, its building processes that systematically improve reasoning to different very hard applied problems.
      - ironborn123 3y ago
        Brave are those who have set out to make prompt engg an entire industry, with gpt-5 and gemini lurking on the horizon.
        tsunamifury 3y ago
        Sam has specifically stated it's unlikely there will be a GPT5, and likely GPT4 is just a deeply prompt optimized and multimode version of GPT3. But overall, hasn't that theme been true for like... all tech ever? You have to set up and build your own innovation path at some point.
        matmulbro 3y ago
        [flagged]
      - dlkf 3y ago
        > the applications of these sort of reasoning atomic configurations I think are unlimited They are limited to applications in which the latency slo is O(seconds), knowledge of 2021-present doesn’t matter, and you’re allowed to make things up when you don’t know the answer. There are, to be fair, many such applications. But it’s not unlimited.
- svnt 3y ago
  > I don't know if you mean, like, <extensions> and similar (actual substantive changes), but the vast majority of "learning how to program" <computers> (accounting for the majority of startup pitches as well) is "<punchcard> engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of <clever pointer dereferences>, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
  - tsunamifury 3y ago
    Ha I think you get it. The only moat is either user engagement, sales or sufficient sophistication of otherwise known techniques. There is not dark arts magic moat.
- treprinum 3y ago
  There is a lot of moat for intelligent people who leverage emerging properties of huge LLMs. I am not going to give you a hint but I already use those LLMs in unexpected ways that weren't possible before.
alfalfasprout 3y ago
> LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added. So... prompt engineering? They're an extremely inefficient processor though and very prone to error (despite what synthetic benchmarks may show).
- tsunamifury 3y ago
  As with any truly promising technology, out of the box performance should not be mistaken for long term potential. I think this is very much like the CCD sensor, that Kodak couldn't envision using because it was "so expensive, slow and low resolution."

LordDragonfang 3y ago

>when you learn how to program it I don't know if you mean, like, LoRAs and similar (actual substantive changes), but the vast majority of "learning how to program" LLMs (accounting for the majority of startup pitches as well) is "prompt engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of English prose, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.

tsunamifury 3y ago
This is the error of the thinking. It would be like saying software doesn't have a moat because thats just clever talking to a processor. But no one would say that now, thats ridiculous. There is a sufficient degree of prompt engineering that is already defensible, I'm already doing it myself IMO. You'll see very sophisticated hybrid programming/prompting systems being developed in the next year that will prove out the case. For example 30 parallel prompts that then amalgamate into a decision and an audit, with 10 simulation level prompts running chained afterwards to clean the output. These types of atomic configurations will become sufficiently complex to not be just for 'anybody'.
- __loam 3y ago
  The output of that many chained prompts is probably so unreliable that it's useless.
  - tsunamifury 3y ago
    Again, this is a misunderstanding of what LLMs are capable of. These aren't chained, you can run parallel prompts of 15 personas with X diversity of perspective, that reason on a singular request, string, input or variable, they provide output plus audit explanation. You then run an amalgamation or committee decision (sort of like mixture of experts) on it to output variable or string. Then run parallel simulation or reflection prompts based on X different context personas to double check their application to outside cases, unconsidered context, etc. It's pretty effective on complex problems like Spam, Trust and Safety, etc. And the applications of these sort of reasoning atomic configurations I think are unlimited. It's not just 'talking fancy' to an AI, its building processes that systematically improve reasoning to different very hard applied problems.
    - ironborn123 3y ago
      Brave are those who have set out to make prompt engg an entire industry, with gpt-5 and gemini lurking on the horizon.
      - tsunamifury 3y ago
        Sam has specifically stated it's unlikely there will be a GPT5, and likely GPT4 is just a deeply prompt optimized and multimode version of GPT3. But overall, hasn't that theme been true for like... all tech ever? You have to set up and build your own innovation path at some point.
        matmulbro 3y ago
        [flagged]
    - dlkf 3y ago
      > the applications of these sort of reasoning atomic configurations I think are unlimited They are limited to applications in which the latency slo is O(seconds), knowledge of 2021-present doesn’t matter, and you’re allowed to make things up when you don’t know the answer. There are, to be fair, many such applications. But it’s not unlimited.
svnt 3y ago
> I don't know if you mean, like, <extensions> and similar (actual substantive changes), but the vast majority of "learning how to program" <computers> (accounting for the majority of startup pitches as well) is "<punchcard> engineering" - which, as the meme goes, isn't a moat. There's a skill to it, yes, but if your singular advantage boils down to a few lines of <clever pointer dereferences>, your product isn't able to control a market - and VCs are (rightly) not interested unless you have the possibility to be a near-monopoly.
- tsunamifury 3y ago
  Ha I think you get it. The only moat is either user engagement, sales or sufficient sophistication of otherwise known techniques. There is not dark arts magic moat.
treprinum 3y ago
There is a lot of moat for intelligent people who leverage emerging properties of huge LLMs. I am not going to give you a hint but I already use those LLMs in unexpected ways that weren't possible before.

alfalfasprout 3y ago

> LLMs are a lot more like a generalized processor than people are admitting right now. Granted you can talk to it, but it becomes significantly more capable when you learn how to program it -- and thats where the value will be added. So... prompt engineering? They're an extremely inefficient processor though and very prone to error (despite what synthetic benchmarks may show).

tsunamifury 3y ago
As with any truly promising technology, out of the box performance should not be mistaken for long term potential. I think this is very much like the CCD sensor, that Kodak couldn't envision using because it was "so expensive, slow and low resolution."