5 ms·

As a (former) ChatGPT plugins developer, our business has absolutely tanked due to GPTs. Discoverability is nonexistent because search is just buried by spam. A

As a (former) ChatGPT plugins developer, our business has absolutely tanked due to GPTs. Discoverability is nonexistent because search is just buried by spam. At least with plugins, there were only a few hundred to sort through, and most had some unique API they would plug into. For context, we built ChatOCR- an OCR tool that lets users extract text from photos and PDFs. We made roughly $20k from 39,000 users over 6 months on the plugins catalog

danpalmer 3y ago
Do end users have to pay you each time or buy access or credits or something, or do you just get a cut of ChatGPT paid subscriptions when a paying user uses your plugin?
- Areibman 3y ago
  We use a third party plugins manager called pluginlab.ai. It manages auth and subscriptions for plugins users by prompting them to sign in when they hit a paywall.
jbverschoor 3y ago
[flagged]
- csallen 3y ago
  Nobody in this thread said they need OpenAI to provide them discoverability.
- refulgentis 3y ago
  Not sure what you mean: OpenAI's App Store doesn't charge and has discoverability. (hence the viability of the spam, hence TFA) OP is likely affected by the change from plugins to GPTs Now, users have to manually select an app. Whereas before, you'd have a list of, say, 5 apps and the AI would attempt to intelligently determine which app to use. OpenAI said plugins didn't find product-market fit, and that's why they moved on. It's worth opining there's an air of rushing around doing nothing with their product development. Like instead of "what should it be?", and building it, and sustaining it, it's a pastiche of every startup shibboleth you've ever heard - sprints, if 10-20% of the user base / 10^7 users aren't using it within 3-5 months, it's time to Pivot (throw away the working solution we invested in) That makes sense when you're at 100 users in an uncertain market but at their scale, it reminds me more of how Google ended up with the reputation it has. * forgive me for not air-quoting app and app store, these aren't __apps__ per se, but it was immensely distracting, with an air of condescention, when I air-quoted everything
  - chrsig 3y ago
    > Not sure what you mean: OpenAI's App Store doesn't charge and has discoverability. (hence the viability of the spam, hence TFA) I think the point was that the app store charge acts as an effective filter for spam, enough so as to determine the sustainability of the app store ecosystem. I could very well be mistaken though, that was just how I interpreted it.
- matthewmacleod 3y ago
  It’s almost like different people can think differently about different things!
- xandrius 3y ago
  Wrong thread.
barfbagginus 3y ago
There's a conflict in value proposition here, which is more drastic than the discovery issue. That might help you better contextualize the failure, and avoid similar risks in the future. Namely, GPT is now a multimodal trained llm capable of doing OCR over PDFs and images. Given the accessibility of that feature, we expect that users do not try to discover OCR plugins anymore- they feel no need. Out of curiosity, what was the contingency plan in the case that OpenAI did this? What rationales did you use to estimate the likelihood and severity of that risk? Were there good reasons to discount that risk?
- Areibman 3y ago
  We knew the time would come, but we built ChatOCR in a week. If we overthought the time horizon problem, we'd have 0 users and $0. But also, GPT-4-Vision is multi-modal but does not specifically use OCR. Our tool is used mostly to extract text from documents and load it into context, and we still saw growth after OpenAI built this feature into ChatGPT.
  - nextworddev 3y ago
    $20k for a week’s work seems more than good, no?
    - sangnoir 3y ago
      Whose week's work? One person, a small team (<5), or a large organization? At some point on that axis, the RoI becomes negative.
      - guiambros 3y ago
        You can't get a large organization to move fast and ship something in a week. So this was either a solo developer en devour, or at max low single digits.
  - barfbagginus 3y ago
    Thank you for sharing your risk thinking here. I like that you are risk hungry, not prone to hesitation, and you achieved revenue on a shoe string budget. I'm curious about the idea that your OCR-generated context might give better recall than whatever GPT uses to parse PDFs into context. That would support the idea that it is discoverability, and not feature parity, that is killing the product. Can we discuss benchmarks that show conclusively that your solution does better on QA accuracy with respect to a PDF, as compared to GPT4's own PDF mode? Also I'm curious about the trajectory of active users after the feature dropped from OAI. Was there an inflection point where new active users per month went negative? How long did it take before that inflection happened? Do you still have enough users to keep the service up and gathering revenue, or do you plan to mothball? Do you have any plans to pivot the offering to other markets?
codethatwerks 3y ago
Is there a name for this rugpull that is a recurring theme in tech, when you rely solely on another service bringing attention to your service. E.g. when youtube cancels you or your adwords account is blocked so you no longer make money
- jewel 3y ago
  I've heard it called "Platform risk". Also "Playing in someone else's walled garden", or something along those lines. I realize that's not a term for the inevitable rugpull, but that's the closest I can think of.
  - d13 3y ago
    You live on that platform, you die on that platform.
  - amelius 3y ago
    Building your castle in someone else's kingdom.
    - theolivenbaum 3y ago
      And obviously the Germans have a word and a law for it: Erbpacht (https://de.m.wikipedia.org/wiki/Erbpacht https://de.m.wikipedia.org/wiki/Erbpacht)
      - mr_toad 3y ago
        Translates as “emphyteusis”, a word I’d never heard of before. https://en.wiktionary.org/wiki/emphyteusis https://en.wiktionary.org/wiki/emphyteusis
    - oldsecondhand 3y ago
      Building your mobile home on rented land.
- snazz 3y ago
deleted 3y ago
[deleted]
irthomasthomas 3y ago
One would think that an company creating cutting edge AI tools, would dog-food that AI biggly. Like, using it for discoverability, here, or having a chat-with-docs system, like groq does. But that doesn't seem to be the case. Though they do use it for prompt generation. I wonder.... The prompt generators can't really go wrong. I mean, they can look better or worse, subjectively. But a QnA bot answering questions about docs can be objectively wrong. Is this calculated to improve the optics, and avoid a lot of hallucination complaints?
- nextworddev 3y ago
  It’s concerning that OpenAI doesn’t dog food LLMs for stuff outside of their core business

danpalmer 3y ago

Do end users have to pay you each time or buy access or credits or something, or do you just get a cut of ChatGPT paid subscriptions when a paying user uses your plugin?

Areibman 3y ago
We use a third party plugins manager called pluginlab.ai. It manages auth and subscriptions for plugins users by prompting them to sign in when they hit a paywall.

jbverschoor 3y ago

[flagged]

csallen 3y ago
Nobody in this thread said they need OpenAI to provide them discoverability.
refulgentis 3y ago
Not sure what you mean: OpenAI's App Store doesn't charge and has discoverability. (hence the viability of the spam, hence TFA) OP is likely affected by the change from plugins to GPTs Now, users have to manually select an app. Whereas before, you'd have a list of, say, 5 apps and the AI would attempt to intelligently determine which app to use. OpenAI said plugins didn't find product-market fit, and that's why they moved on. It's worth opining there's an air of rushing around doing nothing with their product development. Like instead of "what should it be?", and building it, and sustaining it, it's a pastiche of every startup shibboleth you've ever heard - sprints, if 10-20% of the user base / 10^7 users aren't using it within 3-5 months, it's time to Pivot (throw away the working solution we invested in) That makes sense when you're at 100 users in an uncertain market but at their scale, it reminds me more of how Google ended up with the reputation it has. * forgive me for not air-quoting app and app store, these aren't __apps__ per se, but it was immensely distracting, with an air of condescention, when I air-quoted everything
- chrsig 3y ago
  > Not sure what you mean: OpenAI's App Store doesn't charge and has discoverability. (hence the viability of the spam, hence TFA) I think the point was that the app store charge acts as an effective filter for spam, enough so as to determine the sustainability of the app store ecosystem. I could very well be mistaken though, that was just how I interpreted it.
matthewmacleod 3y ago
It’s almost like different people can think differently about different things!
xandrius 3y ago
Wrong thread.

barfbagginus 3y ago

There's a conflict in value proposition here, which is more drastic than the discovery issue. That might help you better contextualize the failure, and avoid similar risks in the future. Namely, GPT is now a multimodal trained llm capable of doing OCR over PDFs and images. Given the accessibility of that feature, we expect that users do not try to discover OCR plugins anymore- they feel no need. Out of curiosity, what was the contingency plan in the case that OpenAI did this? What rationales did you use to estimate the likelihood and severity of that risk? Were there good reasons to discount that risk?

Areibman 3y ago
We knew the time would come, but we built ChatOCR in a week. If we overthought the time horizon problem, we'd have 0 users and $0. But also, GPT-4-Vision is multi-modal but does not specifically use OCR. Our tool is used mostly to extract text from documents and load it into context, and we still saw growth after OpenAI built this feature into ChatGPT.
- nextworddev 3y ago
  $20k for a week’s work seems more than good, no?
  - sangnoir 3y ago
    Whose week's work? One person, a small team (<5), or a large organization? At some point on that axis, the RoI becomes negative.
    - guiambros 3y ago
      You can't get a large organization to move fast and ship something in a week. So this was either a solo developer en devour, or at max low single digits.
- barfbagginus 3y ago
  Thank you for sharing your risk thinking here. I like that you are risk hungry, not prone to hesitation, and you achieved revenue on a shoe string budget. I'm curious about the idea that your OCR-generated context might give better recall than whatever GPT uses to parse PDFs into context. That would support the idea that it is discoverability, and not feature parity, that is killing the product. Can we discuss benchmarks that show conclusively that your solution does better on QA accuracy with respect to a PDF, as compared to GPT4's own PDF mode? Also I'm curious about the trajectory of active users after the feature dropped from OAI. Was there an inflection point where new active users per month went negative? How long did it take before that inflection happened? Do you still have enough users to keep the service up and gathering revenue, or do you plan to mothball? Do you have any plans to pivot the offering to other markets?

codethatwerks 3y ago

Is there a name for this rugpull that is a recurring theme in tech, when you rely solely on another service bringing attention to your service. E.g. when youtube cancels you or your adwords account is blocked so you no longer make money

jewel 3y ago
I've heard it called "Platform risk". Also "Playing in someone else's walled garden", or something along those lines. I realize that's not a term for the inevitable rugpull, but that's the closest I can think of.
- d13 3y ago
  You live on that platform, you die on that platform.
- amelius 3y ago
  Building your castle in someone else's kingdom.
  - theolivenbaum 3y ago
    And obviously the Germans have a word and a law for it: Erbpacht (https://de.m.wikipedia.org/wiki/Erbpacht https://de.m.wikipedia.org/wiki/Erbpacht)
    - mr_toad 3y ago
      Translates as “emphyteusis”, a word I’d never heard of before. https://en.wiktionary.org/wiki/emphyteusis https://en.wiktionary.org/wiki/emphyteusis
  - oldsecondhand 3y ago
    Building your mobile home on rented land.
snazz 3y ago

deleted 3y ago

[deleted]

irthomasthomas 3y ago

One would think that an company creating cutting edge AI tools, would dog-food that AI biggly. Like, using it for discoverability, here, or having a chat-with-docs system, like groq does. But that doesn't seem to be the case. Though they do use it for prompt generation. I wonder.... The prompt generators can't really go wrong. I mean, they can look better or worse, subjectively. But a QnA bot answering questions about docs can be objectively wrong. Is this calculated to improve the optics, and avoid a lot of hallucination complaints?

nextworddev 3y ago
It’s concerning that OpenAI doesn’t dog food LLMs for stuff outside of their core business