5 ms·

I don't hate this idea. Having an adjunct to therapy- really, a vastly souped-up ELIZA- that can gently talk back could be valuable. My real issue here is that

I don't hate this idea. Having an adjunct to therapy- really, a vastly souped-up ELIZA- that can gently talk back could be valuable. My real issue here is that there are no privacy protections around any of it. Your conversations with GPT-3 in a therapeutic/journaling context might be mined and/or exposed just as easily as your attempts to get it to generate shitposts. Back in the day you had to do a heist to steal someone's therapist's notes and physically steal a filing cabinet: https://www.smithsonianmag.com/history/the-worlds-most-famous-filing-cabinet-36568830/ https://www.smithsonianmag.com/history/the-worlds-most-famou... At least information you give to a therapist mostly stays in their head, and unless they have a very good memory, naturally blurs over time. Information you give GPT-3 can sit in a database, attached to your name, perfectly preserved indefinitely. Really every texting-therapy service has this issue, not just automated ones. At least you have to go out of your way to archive a video or audio call, but text can just get blindly copied around systems forever.

simonw 4y ago
This is one of the reasons I'm looking forward to being able to run a language model with similar capabilities to GPT-3 entirely on hardware I control - like I can today with Stable Diffusion. My current impression is that this is a few years away, due to the intense compute requirements for that scale of model. But I think it will happen eventually.
- generalizations 4y ago
  I looked into this a bit last week. The OPT-30b model seems close to GPT-3 in capabilities, and running on 4x 24GB NVIDIA A10G GPUs, can generate 90 tokens per second. I wonder how fast that model could run on 4x 24GB NVIDIA P40 GPUs - which currently cost about $110 on ebay. It'd be a lot slower, but even if it's 1/90th the speed, that's still 1 token per second.
  - danielbln 4y ago
    1 token/s is way too slow for dialog, especially since a token isn't even a word, but often part of a word. 1 t/s might be sufficient for asynchronous processing, but if you want a chatGPT like therapy dialog, then that's not good enough.
    - generalizations 4y ago
      Do we know what the speed difference actually is? I'm not sure what benchmarks would measure that. My best plan so far is to just run a smaller model on one of the GPUs and see how long it takes.
- theptip 4y ago
  Agreed. It's promising that Apple in particular are working on ways to run models like Stable Diffusion locally on mobile device hardware. I think they probably have the best chance of making that happen, given their vertical integration and hardware-centric / non-ad-based economic model. One speculative possiblity I haven't seen discussed much -- up until now, we've not seen large-scale work on hardware implementation of these NN models, since the frontier of capabilities has been advancing so fast. However, perhaps we're now approacing the first point where a baked-in-silicon (or at least part-baked) model might be economically viable. Especially if someone can figure out a way to efficiently fine-tune such a model (something like an overlay filesystem), or perhaps something like a FPGA where the underlying logical structure can be reprogrammed.
kibwen 4y ago
Yet another reason why we need open, free, self-hostable AI models.
- lairv 4y ago
  Note that while "open and free" is possible if a company bites the bullet, there is currently no way it would be self-hostable on consumer GPU
  - williamcotton 4y ago
    Wouldn't it be great if there was some way to do distributed training of models?
    - lairv 4y ago
      Actually there is the Petals [0] project which does that (I think it's more for inference and fine-tuning, not full training) and might be the best current approach for "self-hosted LLM", though it defeats the point of privacy/anonymity since everyone in the distributed swarm have access to your data [0] https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals
  - heurist 4y ago
    Is there any concept of a "frontend" model that interfaces with a "backend" model? I.e. frontend takes the raw input, encodes it, backend processes the encoding, returns to frontend, which decodes for end use?
    - jonathankoren 4y ago
      All models have some sort of translation layer between the real world and world of statistics. If you mean a common input format that obfuscates the raw input, anything that deals in a latent space could work, but they’d be pretty generic. It would be interesting if you could make a system that does statistics and (even modeling if you get it to work) with homomorphic encrypted data. That way, even the host of the model couldn’t understand what was going on
adamsmith143 4y ago
This is really a non-issue. There are already a host of GPT-2 and GPT-3 clones that are available and can be run locally if you have the machine to handle it. E.g. https://huggingface.co/models?sort=downloads&search=GPT https://huggingface.co/models?sort=downloads&search=GPT
- cypress66 4y ago
  None are as good as chatgpt
  - adamsmith143 4y ago
    The parent comment was talking about GPT-3, there are also already alternate implementations of ChatGPT-like tools.
lgas 4y ago
Some human therapists record sessions digitally. Obviously the scale is different so it still may be a concern, but having the information somewhere that could persist forever was already a potential hazard even before bringing AI therapists into it.
canadianfella 4y ago
[dead]

simonw 4y ago

This is one of the reasons I'm looking forward to being able to run a language model with similar capabilities to GPT-3 entirely on hardware I control - like I can today with Stable Diffusion. My current impression is that this is a few years away, due to the intense compute requirements for that scale of model. But I think it will happen eventually.

generalizations 4y ago
I looked into this a bit last week. The OPT-30b model seems close to GPT-3 in capabilities, and running on 4x 24GB NVIDIA A10G GPUs, can generate 90 tokens per second. I wonder how fast that model could run on 4x 24GB NVIDIA P40 GPUs - which currently cost about $110 on ebay. It'd be a lot slower, but even if it's 1/90th the speed, that's still 1 token per second.
- danielbln 4y ago
  1 token/s is way too slow for dialog, especially since a token isn't even a word, but often part of a word. 1 t/s might be sufficient for asynchronous processing, but if you want a chatGPT like therapy dialog, then that's not good enough.
  - generalizations 4y ago
    Do we know what the speed difference actually is? I'm not sure what benchmarks would measure that. My best plan so far is to just run a smaller model on one of the GPUs and see how long it takes.
theptip 4y ago
Agreed. It's promising that Apple in particular are working on ways to run models like Stable Diffusion locally on mobile device hardware. I think they probably have the best chance of making that happen, given their vertical integration and hardware-centric / non-ad-based economic model. One speculative possiblity I haven't seen discussed much -- up until now, we've not seen large-scale work on hardware implementation of these NN models, since the frontier of capabilities has been advancing so fast. However, perhaps we're now approacing the first point where a baked-in-silicon (or at least part-baked) model might be economically viable. Especially if someone can figure out a way to efficiently fine-tune such a model (something like an overlay filesystem), or perhaps something like a FPGA where the underlying logical structure can be reprogrammed.

kibwen 4y ago

Yet another reason why we need open, free, self-hostable AI models.

lairv 4y ago
Note that while "open and free" is possible if a company bites the bullet, there is currently no way it would be self-hostable on consumer GPU
- williamcotton 4y ago
  Wouldn't it be great if there was some way to do distributed training of models?
  - lairv 4y ago
    Actually there is the Petals [0] project which does that (I think it's more for inference and fine-tuning, not full training) and might be the best current approach for "self-hosted LLM", though it defeats the point of privacy/anonymity since everyone in the distributed swarm have access to your data [0] https://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals
- heurist 4y ago
  Is there any concept of a "frontend" model that interfaces with a "backend" model? I.e. frontend takes the raw input, encodes it, backend processes the encoding, returns to frontend, which decodes for end use?
  - jonathankoren 4y ago
    All models have some sort of translation layer between the real world and world of statistics. If you mean a common input format that obfuscates the raw input, anything that deals in a latent space could work, but they’d be pretty generic. It would be interesting if you could make a system that does statistics and (even modeling if you get it to work) with homomorphic encrypted data. That way, even the host of the model couldn’t understand what was going on

adamsmith143 4y ago

This is really a non-issue. There are already a host of GPT-2 and GPT-3 clones that are available and can be run locally if you have the machine to handle it. E.g. https://huggingface.co/models?sort=downloads&search=GPT https://huggingface.co/models?sort=downloads&search=GPT

cypress66 4y ago
None are as good as chatgpt
- adamsmith143 4y ago
  The parent comment was talking about GPT-3, there are also already alternate implementations of ChatGPT-like tools.

lgas 4y ago

Some human therapists record sessions digitally. Obviously the scale is different so it still may be a concern, but having the information somewhere that could persist forever was already a potential hazard even before bringing AI therapists into it.

canadianfella 4y ago

[dead]