8 ms·

The future of generative AI is niche, not generalized

107 points by gumby 4y ago

amelius 4y ago
Only if we let companies get away with disclaimers like: "our bot can say things that are incorrect, and it doesn't give you any rights".
- echelon 4y ago
  GenAI porn is going to be huge. GenAI interactive literotica is going to be huge. A couple of companies are already executing in these areas and have amassed loyal followings.
  - LewisVerstappen 4y ago
    > couple of companies are already executing in these areas and have amassed loyal followings What companies are there in the interactive literotica space? I was curious about building something like this myself (before people dismiss this as a dumb idea, they should check out what the best selling books on Kindle are).
    - KaoruAoiShiho 4y ago
      Fiction.live is going to be huge in this area, they can leverage their existing content and writers to get incremental results. Since the technology is in its early days the writers can manually improve the generated results to get it to be useful quality, and this will tide the whole system over until perfect completely-AI results are here, guessing sometime towards the end of the year.
    - yi_xuan 4y ago
      seems Microsoft was trying: https://www.nytimes.com/2015/08/04/science/for-sympathetic-ear-more-chinese-turn-to-smartphone-program.html https://www.nytimes.com/2015/08/04/science/for-sympathetic-e...
    - anonylizard 4y ago
      Well, the simplest of 'literotica' is just an uncensored, fiction-writing friendly LLM (Fine tuned on fiction/RP, rather than truthful and friendly responses). NovelAI has been around for a long time, after AI dungeon died in flames. They sell exactly that. They also did the first powerful fine-tune of SD. Now they've got a H100 cluster, and are committed to training a GPT3.5 equivalent. These companies have difficulties getting investor funding, the good side is however, AI products are so useful, users have an extremely high intention to pay. So these no-investor companies can pure bootstrap themselves with high profitability and high growth.
      - echelon 4y ago
        Are investors disinterested in NovelAI, or are they just choosing to bootstrap?
  - Loveaway 4y ago
    Porn already is huge? In terms of reach, but money compared to other industries is rather small. Video Games for example are 20x-50x bigger then Porn. If anything GenAI means there's gonna be even less money in it. Ok so, but GenAI interactive literotica companies with loyal followings you say? When's the IPO going to be exactly?
    - Mistletoe 4y ago
      When will we get good AI quests and characters in video games? To me, this is one of the most exciting aspects of AI. I can imagine a very cool future where the dialog isn’t written, each character is just given a “this is your motivation” prompt like an actor might get. The replayability factor will be huge.
      - TigeriusKirk 4y ago
        Companies are of course experimenting with this. And the mod communities are working on projects too. A Fallout 4 mod is using AI to voice human written options in the same voices as existing characters. And there's a couple Skyrim projects working on dialogue generation. Personally I expect this to be an area where amateurs/hobbyists contribute quite a bit of research.
    - f6v 4y ago
      > Video Games for example are 20x-50x bigger then Porn. I wonder how these two can be compared. Adult entertainment often resides in the dark economy.
      - yurishimo 4y ago
        This got me curious. Currently the global market cap for video games is somewhere around $188B. From a cursory Google search, the revenue from alcohol sales worldwide is 10x that of video games. Seems like a crazy gap for adult entertainment!
      - Loveaway 4y ago
        It's an estimate. You can come up with different values, but I think no one would really argue porn is that huge of a slice of the pie. Clean money is also a lot more valueable than dark money. Generative porn will be a big thing, but won't magically lead to more money flowing into the industry. If anything, less. What's gonna happen is, synthetic porn will replace real porn over time, that's pretty much it. All things considered, biggest beneficiary gotta be the women currently working in the industry, no doubt.
  - anonymous_sorry 4y ago
    Based purely on interactions with ChatGPT, interactive fiction isn't really there yet. ChatGPT is not able to tell a cohesive interactive story, because it doesn't understand what's going on. Just for one example, it forgets who is where, and has characters who are not present ("wait in the car") participate in dialogue. I'm not sure if this is something that can be iterated on, or if it's a fundamental characteristic of large language models.
    - TeMPOraL 4y ago
      I don't know; AI Dungeon was quite good for interactive fiction and role-play, and that was in GPT-2 and early GPT-3 era.
    - tudorw 4y ago
      this is being addressed as a priority, it's a lack of long term memory, you can improve results by repeating key facts during the conversation, stating the same request in different ways and by asking it to summarise unique elements of the conversation so far, then start a new conversation with the 'compressed' conversation, this helps preserve memory, or context might be a better word, imagine blowing a balloon in an nth dimensional topological manifold, your balloon is inside the big balloon that is AI, at the edges of the balloon are 'understandings' that have been established, you have to keep blowing...
jsemrau 4y ago
Interesting article yet not as thought provoking as it could have been. >A related trend we’ve seen is domain-specific language models. One can argue that we have a general level of understanding up to the end of high-school and then specialise when going to universities or trade schools. Hence seeing the same in artificial agents does make sense for me. Further, calling it "niche" is a strange word to use in combination with domain-specific models. While from the origin of the word, "niche" can refer to a defined market segment, we use "niche" in common english as a modifier more akin to "relating to or aimed at a small specialized group or market" . Which might or might not be true depending on the size of the market.
- xyzzy123 4y ago
  I think we might see specialisation, but driven more by proprietary datasets, fine tuning or specific supporting software. That is, a sort of protectionism and/or moat application, rather than limits on how many fields the LLM can be proficient in.
scyzoryk_xyz 4y ago
I can imagine AI being module-based. Like with plug-ins or apps, call it what you will. So if you’re, say working in a hospital in a specific role, you will be advised by an AI module for that specific system. Or if you are working in an OR with a specific surgical procedure every day, you will have an AI module active all day to augment your knowledge and deliver relevant information where it’s needed. It’s going to make some types of work immensely satisfying.
- CuriouslyC 4y ago
  Based on how things are unfolding in stable diffusion land, we're going to see general purpose base models that are then fine tuned for specific tasks using LoRAs that can be quickly loaded/unloaded as needed.
  - orbital-decay 4y ago
    >LoRAs that can be quickly loaded/unloaded as needed No need to shuffle them around, you can combine different finetunes in one composition.
    - anotherhue 4y ago
      Kindof? But then, for high-risk cases like this you'd need to validate their combinatorial effect.
      - deleted 4y ago
        [deleted]
- sebastianconcpt 4y ago
  Correct, as long as they become ROI positive/business backed, one module per profession. In the future, this extends to robots, of course.
- ricketycricket 4y ago
  I work in a field related to surgeon education. Until the results of AI output can be guaranteed to be deterministic and proven to always be correct, the legal ramifications of an AI providing incorrect information or promoting off-label usage is likely insurmountable. There are dozens or hundreds of eyes that see this information before it is approved for use and weeks can be spent arguing over a single word.
janmo 4y ago
I follow a bunch of indie devs on Twitter, most of them started an AI product often based on ChatGPT and Stable diffusion. Many went viral in a very short time frame, but now they are experiencing a huge churn rate, meaning people canceling their membership, also an alarming high refund and dispute rate. I believe AI startups will loose 90% of their customers soon, as those are only people who where hyped into trying it out but have actually no use case for it. AI is currently what Internet was in 1999.
- lfkdev 4y ago
  The Internet literally took over the whole world
  - animaomnium 4y ago
    ...but only after the dot-com bubble burst
  - Janicc 4y ago
    Exactly
  - janmo 4y ago
    Sure, but the AI hype bubble will pop in between, and then it will be a slow and gradual process. It will take decades for AI to be fully integrated in our society.
    - GreedClarifies 4y ago
      Your statement hinges on the definition of "fully". I'll simply counter by saying that the value created by these AI technologies is immense and given that value it will likely be intergraded into society very quickly.
    - gumby 4y ago
      The ai bubble has popped in the past and shall again.
- 4y ago
cubefox 4y ago
> once we’ve realized it doesn’t know everything — and never will — that will be when it starts to become really useful. I disagree with this. Once it becomes superhumanly smart, that will be when it starts to become really useful. Of course superhuman intelligence might be dismissed as outlandish science fiction technology which is far away. But three years ago ChatGPT-4 or Midjourney V5 were also outlandish science fiction technology.
- RandomLensman 4y ago
  LLMs cannot reason or use mathematics - in a way, they don't know what they are talking about. Why would such technology lead to superhuman smarts?
  - circuit10 4y ago
    What is your definition of “reasoning” here? They are clearly able to do many things that we would call reasoning if a human did them
    - RandomLensman 4y ago
      They seem to not be able to use concepts they can "explain" (beyond having answers sufficiently dense in the training set, it seems).
      - circuit10 4y ago
        Humans also take a while to learn how to use concepts that they might be able to remember an answer to, though. Also keep in mind that LLMs have a limited time to produce each token and can’t “stop to think” like a human could. Obviously LLMs aren’t as good as reasoning as humans but they clearly do some (possibly quite limited) form of reasoning
        RandomLensman 4y ago
        Do you have good examples of an LLM reasoning, i.e., using a concept it explains and giving an answer outside of its training data on something that has a clearly defined correct answer?
p-e-w 4y ago
These speculative articles are always amusing to read. Nobody knows how even the current generation of LLMs work (at the plumbing layer, sure, but the high-level behavior is a complete mystery). Yet there is an endless stream of opinion pieces telling us what future LLMs won't be able to do. Here's the most truthful 'article' on AI you will ever read: "We have no idea where we are, and we have no idea where we are going." I struggle to understand why this is so incredibly hard to admit for some people.
- Ensorceled 4y ago
  Everyone has this same feeling that we don’t know what the hell is going on or what is going to happen. In troubled times there’s a buck to be made by confidently asserting you know what’s going on and what’s coming next.
  - p-e-w 4y ago
    One of the problems is that people treat experts on AI technology as if they were also experts on AI philosophy, which leads to poorly edited thought salads being published in a respectable context. Just because someone understands variational autoencoders doesn't mean they have a clue about how the field of AI will look like 5 years from now, and it certainly doesn't mean they can anticipate the societal and political impacts of those technologies any better than the average (intelligent) Joe.
    - _a_a_a_ 4y ago
      I don't know what 'AI philosophy' is, if it even exists, but if it does exist then poor quality articles might reflect the state of AI philosophy, or it might be just crappy articles. I'm not comfortable with the idea that philosophy is intrinsically nebulous and poor quality, any more than observing amateur footballers being not very good would lead you to assume that there can be no such thing as a good footballer; of course there are, but there are a damn sight rarer than your local kids having a kick around for fun.
      - 4y ago
Taek 4y ago
This reads a lot like "The future of the Internet is niche (fax machines), not generalized)
mensetmanusman 4y ago
Unless there is a breakthrough that simultaneously delivers: -low cost -high quality -low latency -large token count interactions over all of human knowledge in every language, it’s almost a tautology to suggest we will focus AI systems to be subject matter based over some subset to solve the cost/latency factor. This is why the Y just invested $20M in a legal specific AI.
vrglvrglvrgl 4y ago
[dead]
Kalanos 4y ago
what about a general agent that has access to niche actions, and the ability to train its own niche models?

107 points by gumby 4y ago

Only if we let companies get away with disclaimers like: "our bot can say things that are incorrect, and it doesn't give you any rights".

echelon 4y ago
GenAI porn is going to be huge. GenAI interactive literotica is going to be huge. A couple of companies are already executing in these areas and have amassed loyal followings.
- LewisVerstappen 4y ago
  > couple of companies are already executing in these areas and have amassed loyal followings What companies are there in the interactive literotica space? I was curious about building something like this myself (before people dismiss this as a dumb idea, they should check out what the best selling books on Kindle are).
  - KaoruAoiShiho 4y ago
    Fiction.live is going to be huge in this area, they can leverage their existing content and writers to get incremental results. Since the technology is in its early days the writers can manually improve the generated results to get it to be useful quality, and this will tide the whole system over until perfect completely-AI results are here, guessing sometime towards the end of the year.
  - yi_xuan 4y ago
    seems Microsoft was trying: https://www.nytimes.com/2015/08/04/science/for-sympathetic-ear-more-chinese-turn-to-smartphone-program.html https://www.nytimes.com/2015/08/04/science/for-sympathetic-e...
  - anonylizard 4y ago
    Well, the simplest of 'literotica' is just an uncensored, fiction-writing friendly LLM (Fine tuned on fiction/RP, rather than truthful and friendly responses). NovelAI has been around for a long time, after AI dungeon died in flames. They sell exactly that. They also did the first powerful fine-tune of SD. Now they've got a H100 cluster, and are committed to training a GPT3.5 equivalent. These companies have difficulties getting investor funding, the good side is however, AI products are so useful, users have an extremely high intention to pay. So these no-investor companies can pure bootstrap themselves with high profitability and high growth.
    - echelon 4y ago
      Are investors disinterested in NovelAI, or are they just choosing to bootstrap?
- Loveaway 4y ago
  Porn already is huge? In terms of reach, but money compared to other industries is rather small. Video Games for example are 20x-50x bigger then Porn. If anything GenAI means there's gonna be even less money in it. Ok so, but GenAI interactive literotica companies with loyal followings you say? When's the IPO going to be exactly?
  - Mistletoe 4y ago
    When will we get good AI quests and characters in video games? To me, this is one of the most exciting aspects of AI. I can imagine a very cool future where the dialog isn’t written, each character is just given a “this is your motivation” prompt like an actor might get. The replayability factor will be huge.
    - TigeriusKirk 4y ago
      Companies are of course experimenting with this. And the mod communities are working on projects too. A Fallout 4 mod is using AI to voice human written options in the same voices as existing characters. And there's a couple Skyrim projects working on dialogue generation. Personally I expect this to be an area where amateurs/hobbyists contribute quite a bit of research.
  - f6v 4y ago
    > Video Games for example are 20x-50x bigger then Porn. I wonder how these two can be compared. Adult entertainment often resides in the dark economy.
    - yurishimo 4y ago
      This got me curious. Currently the global market cap for video games is somewhere around $188B. From a cursory Google search, the revenue from alcohol sales worldwide is 10x that of video games. Seems like a crazy gap for adult entertainment!
    - Loveaway 4y ago
      It's an estimate. You can come up with different values, but I think no one would really argue porn is that huge of a slice of the pie. Clean money is also a lot more valueable than dark money. Generative porn will be a big thing, but won't magically lead to more money flowing into the industry. If anything, less. What's gonna happen is, synthetic porn will replace real porn over time, that's pretty much it. All things considered, biggest beneficiary gotta be the women currently working in the industry, no doubt.
- anonymous_sorry 4y ago
  Based purely on interactions with ChatGPT, interactive fiction isn't really there yet. ChatGPT is not able to tell a cohesive interactive story, because it doesn't understand what's going on. Just for one example, it forgets who is where, and has characters who are not present ("wait in the car") participate in dialogue. I'm not sure if this is something that can be iterated on, or if it's a fundamental characteristic of large language models.
  - TeMPOraL 4y ago
    I don't know; AI Dungeon was quite good for interactive fiction and role-play, and that was in GPT-2 and early GPT-3 era.
  - tudorw 4y ago
    this is being addressed as a priority, it's a lack of long term memory, you can improve results by repeating key facts during the conversation, stating the same request in different ways and by asking it to summarise unique elements of the conversation so far, then start a new conversation with the 'compressed' conversation, this helps preserve memory, or context might be a better word, imagine blowing a balloon in an nth dimensional topological manifold, your balloon is inside the big balloon that is AI, at the edges of the balloon are 'understandings' that have been established, you have to keep blowing...

jsemrau 4y ago

Interesting article yet not as thought provoking as it could have been. >A related trend we’ve seen is domain-specific language models. One can argue that we have a general level of understanding up to the end of high-school and then specialise when going to universities or trade schools. Hence seeing the same in artificial agents does make sense for me. Further, calling it "niche" is a strange word to use in combination with domain-specific models. While from the origin of the word, "niche" can refer to a defined market segment, we use "niche" in common english as a modifier more akin to "relating to or aimed at a small specialized group or market" . Which might or might not be true depending on the size of the market.

xyzzy123 4y ago
I think we might see specialisation, but driven more by proprietary datasets, fine tuning or specific supporting software. That is, a sort of protectionism and/or moat application, rather than limits on how many fields the LLM can be proficient in.

scyzoryk_xyz 4y ago

I can imagine AI being module-based. Like with plug-ins or apps, call it what you will. So if you’re, say working in a hospital in a specific role, you will be advised by an AI module for that specific system. Or if you are working in an OR with a specific surgical procedure every day, you will have an AI module active all day to augment your knowledge and deliver relevant information where it’s needed. It’s going to make some types of work immensely satisfying.

CuriouslyC 4y ago
Based on how things are unfolding in stable diffusion land, we're going to see general purpose base models that are then fine tuned for specific tasks using LoRAs that can be quickly loaded/unloaded as needed.
- orbital-decay 4y ago
  >LoRAs that can be quickly loaded/unloaded as needed No need to shuffle them around, you can combine different finetunes in one composition.
  - anotherhue 4y ago
    Kindof? But then, for high-risk cases like this you'd need to validate their combinatorial effect.
    - deleted 4y ago
      [deleted]
sebastianconcpt 4y ago
Correct, as long as they become ROI positive/business backed, one module per profession. In the future, this extends to robots, of course.
ricketycricket 4y ago
I work in a field related to surgeon education. Until the results of AI output can be guaranteed to be deterministic and proven to always be correct, the legal ramifications of an AI providing incorrect information or promoting off-label usage is likely insurmountable. There are dozens or hundreds of eyes that see this information before it is approved for use and weeks can be spent arguing over a single word.

janmo 4y ago

I follow a bunch of indie devs on Twitter, most of them started an AI product often based on ChatGPT and Stable diffusion. Many went viral in a very short time frame, but now they are experiencing a huge churn rate, meaning people canceling their membership, also an alarming high refund and dispute rate. I believe AI startups will loose 90% of their customers soon, as those are only people who where hyped into trying it out but have actually no use case for it. AI is currently what Internet was in 1999.

lfkdev 4y ago
The Internet literally took over the whole world
- animaomnium 4y ago
  ...but only after the dot-com bubble burst
- Janicc 4y ago
  Exactly
- janmo 4y ago
  Sure, but the AI hype bubble will pop in between, and then it will be a slow and gradual process. It will take decades for AI to be fully integrated in our society.
  - GreedClarifies 4y ago
    Your statement hinges on the definition of "fully". I'll simply counter by saying that the value created by these AI technologies is immense and given that value it will likely be intergraded into society very quickly.
  - gumby 4y ago
    The ai bubble has popped in the past and shall again.
4y ago

cubefox 4y ago

> once we’ve realized it doesn’t know everything — and never will — that will be when it starts to become really useful. I disagree with this. Once it becomes superhumanly smart, that will be when it starts to become really useful. Of course superhuman intelligence might be dismissed as outlandish science fiction technology which is far away. But three years ago ChatGPT-4 or Midjourney V5 were also outlandish science fiction technology.

RandomLensman 4y ago
LLMs cannot reason or use mathematics - in a way, they don't know what they are talking about. Why would such technology lead to superhuman smarts?
- circuit10 4y ago
  What is your definition of “reasoning” here? They are clearly able to do many things that we would call reasoning if a human did them
  - RandomLensman 4y ago
    They seem to not be able to use concepts they can "explain" (beyond having answers sufficiently dense in the training set, it seems).
    - circuit10 4y ago
      Humans also take a while to learn how to use concepts that they might be able to remember an answer to, though. Also keep in mind that LLMs have a limited time to produce each token and can’t “stop to think” like a human could. Obviously LLMs aren’t as good as reasoning as humans but they clearly do some (possibly quite limited) form of reasoning
      - RandomLensman 4y ago
        Do you have good examples of an LLM reasoning, i.e., using a concept it explains and giving an answer outside of its training data on something that has a clearly defined correct answer?

p-e-w 4y ago

These speculative articles are always amusing to read. Nobody knows how even the current generation of LLMs work (at the plumbing layer, sure, but the high-level behavior is a complete mystery). Yet there is an endless stream of opinion pieces telling us what future LLMs won't be able to do. Here's the most truthful 'article' on AI you will ever read: "We have no idea where we are, and we have no idea where we are going." I struggle to understand why this is so incredibly hard to admit for some people.

Ensorceled 4y ago
Everyone has this same feeling that we don’t know what the hell is going on or what is going to happen. In troubled times there’s a buck to be made by confidently asserting you know what’s going on and what’s coming next.
- p-e-w 4y ago
  One of the problems is that people treat experts on AI technology as if they were also experts on AI philosophy, which leads to poorly edited thought salads being published in a respectable context. Just because someone understands variational autoencoders doesn't mean they have a clue about how the field of AI will look like 5 years from now, and it certainly doesn't mean they can anticipate the societal and political impacts of those technologies any better than the average (intelligent) Joe.
  - _a_a_a_ 4y ago
    I don't know what 'AI philosophy' is, if it even exists, but if it does exist then poor quality articles might reflect the state of AI philosophy, or it might be just crappy articles. I'm not comfortable with the idea that philosophy is intrinsically nebulous and poor quality, any more than observing amateur footballers being not very good would lead you to assume that there can be no such thing as a good footballer; of course there are, but there are a damn sight rarer than your local kids having a kick around for fun.
    - 4y ago

Taek 4y ago

This reads a lot like "The future of the Internet is niche (fax machines), not generalized)

mensetmanusman 4y ago

Unless there is a breakthrough that simultaneously delivers: -low cost -high quality -low latency -large token count interactions over all of human knowledge in every language, it’s almost a tautology to suggest we will focus AI systems to be subject matter based over some subset to solve the cost/latency factor. This is why the Y just invested $20M in a legal specific AI.

vrglvrglvrgl 4y ago

[dead]

Kalanos 4y ago

what about a general agent that has access to niche actions, and the ability to train its own niche models?