4 ms·

The market for completely anonymous and uncensored models via API must be huge. How come nothing exists yet?

by alchemist1e9 3y ago

AbrahamParangi 3y ago
https://replicate.com https://replicate.com does in fact already exist.
- alchemist1e9 3y ago
  Can’t this bot developer just switch to them then? as replicate can’t know what it is being used for I would assume. Perhaps there was fine tuning or RAG being used that still needs a private and anonymous service solution.
- lolinder 3y ago
  They have a list of prohibited uses which, though not has extensive as OpenAI's, includes this gem at the end (emphasis added): > To engage in any other conduct that restricts or inhibits anyone’s use or enjoyment of Service, or which, as determined by us, may harm or offend Company or users of Service or expose them to liability. Edit: Also, their privacy policy makes it clear that "completely anonymous" is not a thing they offer— they collect emails and IP addresses, transmit them to the United States, and will deliver them to law enforcement upon request. https://replicate.com/privacy https://replicate.com/privacy
  - alchemist1e9 3y ago
    So it’s not anonymous then because they see what the customer is generating? Otherwise how can they know what it’s being used for?
    - lolinder 3y ago
      Yes, it's definitely not anonymous. I added to my comment right as you replied—they collect IP addresses and usage data and explicitly say they will share that with law enforcement upon request. It's a US-based company, so they kind of have to.
      - alchemist1e9 3y ago
        Let’s say there exists some “illegal” or “criminal” output of an LLM, doesn’t that mean by definition the government has criminalized speech and thought? since the output of an LLM is by definition tokens of language. Now image generation I guess we can see as different since we do agree some images or videos have content society sees as illegal. The whole topic is fascinating to me. Censorship and regulation around generative AIs.
moffkalast 3y ago
Why trust someone else if you want it to be anonymous? Buy a proper GPU, run a local model and that's it handled. Basically all open source inference engines support OpenAI's API format and act as drop in replacements.
- bongripper 3y ago
  "just spend $1000 on a new GPU"
  - moffkalast 3y ago
    How much do you think those API calls will cost you that require someone else to buy the GPU, maintain the server architecture, and also take a cut in the process? It only takes a 3-4 months to amortize if you want the instance to be up 24/7. And in the end you can still resell the GPU.
  - openquery 3y ago
    I rent spot instances on GCP with big GPUs on demand. It works quite well and since I'm just playing around I pay a very small amount per hour and kill it after I'm done.
    - HeatrayEnjoyer 3y ago
      Does this work on-demand? Could I build something I can query from my phone as needed?
      - 0xedd 3y ago
        Depends on your budget. If it's the absolute minimum, it won't be real time responses. As provisioning infra isn't instantaneous.
  - beej71 3y ago
    Just want to point out that $1000 is nothing to a lot of potential bad actors. $1 million is nothing.
yieldcrv 3y ago
Plenty exists LM Studio lets you load in models from a built in HuggingFace browser, and sets up a server that accepts OpenAi 3.0 API requests, so everyone that built an app around ChatGPT API plugs in exactly the same
foxhop 3y ago
I'm working on this problem in my spare time, it requires a GPU (or two) RTX 3090 or 4090 because of the need for vRAM memory but I could also forsee APIs in front that allow worker nodes to run the quantized models for cheaper. the problem is model switching is hard so you would have to pick which model to run upfront for inference openchat in particular ships with an inference server compatible with openai follow me here: https://www.youtube.com/@fxhp1 https://www.youtube.com/@fxhp1
lolinder 3y ago
Completely anonymous and uncensored anything is hard to come by. You really can't legally provide a service like that in most Western countries. Every service that has tried has ended up realizing why it doesn't exist. Even if you tried to base your business outside of the West, you're left figuring out how to collect payments for it without getting blocked. LLMs are far more expensive to provide than plain text communication or a VPN. Eventually it'll probably happen, but there are a lot of logistics to sort out first, and in the meantime the open source models still have a ways to go before they're good enough at arbitrary problems out of the box to be worth paying for.
vunderba 3y ago
The issue with this is the same one that exists with commercial VNP offerings. It is impossible for the user to verify claims of any company that guarantees anonymity/doesn't track/doesn't keep logs/etc.

The market for completely anonymous and uncensored models via API must be huge. How come nothing exists yet?

AbrahamParangi 3y ago

https://replicate.com https://replicate.com does in fact already exist.

alchemist1e9 3y ago
Can’t this bot developer just switch to them then? as replicate can’t know what it is being used for I would assume. Perhaps there was fine tuning or RAG being used that still needs a private and anonymous service solution.
lolinder 3y ago
They have a list of prohibited uses which, though not has extensive as OpenAI's, includes this gem at the end (emphasis added): > To engage in any other conduct that restricts or inhibits anyone’s use or enjoyment of Service, or which, as determined by us, may harm or offend Company or users of Service or expose them to liability. Edit: Also, their privacy policy makes it clear that "completely anonymous" is not a thing they offer— they collect emails and IP addresses, transmit them to the United States, and will deliver them to law enforcement upon request. https://replicate.com/privacy https://replicate.com/privacy
- alchemist1e9 3y ago
  So it’s not anonymous then because they see what the customer is generating? Otherwise how can they know what it’s being used for?
  - lolinder 3y ago
    Yes, it's definitely not anonymous. I added to my comment right as you replied—they collect IP addresses and usage data and explicitly say they will share that with law enforcement upon request. It's a US-based company, so they kind of have to.
    - alchemist1e9 3y ago
      Let’s say there exists some “illegal” or “criminal” output of an LLM, doesn’t that mean by definition the government has criminalized speech and thought? since the output of an LLM is by definition tokens of language. Now image generation I guess we can see as different since we do agree some images or videos have content society sees as illegal. The whole topic is fascinating to me. Censorship and regulation around generative AIs.

moffkalast 3y ago

Why trust someone else if you want it to be anonymous? Buy a proper GPU, run a local model and that's it handled. Basically all open source inference engines support OpenAI's API format and act as drop in replacements.

bongripper 3y ago
"just spend $1000 on a new GPU"
- moffkalast 3y ago
  How much do you think those API calls will cost you that require someone else to buy the GPU, maintain the server architecture, and also take a cut in the process? It only takes a 3-4 months to amortize if you want the instance to be up 24/7. And in the end you can still resell the GPU.
- openquery 3y ago
  I rent spot instances on GCP with big GPUs on demand. It works quite well and since I'm just playing around I pay a very small amount per hour and kill it after I'm done.
  - HeatrayEnjoyer 3y ago
    Does this work on-demand? Could I build something I can query from my phone as needed?
    - 0xedd 3y ago
      Depends on your budget. If it's the absolute minimum, it won't be real time responses. As provisioning infra isn't instantaneous.
- beej71 3y ago
  Just want to point out that $1000 is nothing to a lot of potential bad actors. $1 million is nothing.

yieldcrv 3y ago

Plenty exists LM Studio lets you load in models from a built in HuggingFace browser, and sets up a server that accepts OpenAi 3.0 API requests, so everyone that built an app around ChatGPT API plugs in exactly the same

foxhop 3y ago

I'm working on this problem in my spare time, it requires a GPU (or two) RTX 3090 or 4090 because of the need for vRAM memory but I could also forsee APIs in front that allow worker nodes to run the quantized models for cheaper. the problem is model switching is hard so you would have to pick which model to run upfront for inference openchat in particular ships with an inference server compatible with openai follow me here: https://www.youtube.com/@fxhp1 https://www.youtube.com/@fxhp1

lolinder 3y ago

Completely anonymous and uncensored anything is hard to come by. You really can't legally provide a service like that in most Western countries. Every service that has tried has ended up realizing why it doesn't exist. Even if you tried to base your business outside of the West, you're left figuring out how to collect payments for it without getting blocked. LLMs are far more expensive to provide than plain text communication or a VPN. Eventually it'll probably happen, but there are a lot of logistics to sort out first, and in the meantime the open source models still have a ways to go before they're good enough at arbitrary problems out of the box to be worth paying for.

vunderba 3y ago

The issue with this is the same one that exists with commercial VNP offerings. It is impossible for the user to verify claims of any company that guarantees anonymity/doesn't track/doesn't keep logs/etc.