5 ms·

I'm reminded of the sci-fi author Peter F. Hamilton's Commonwealth Saga. In it, in order to perform the increasingly complex problem of creating and maintaining

I'm reminded of the sci-fi author Peter F. Hamilton's Commonwealth Saga. In it, in order to perform the increasingly complex problem of creating and maintaining stable wormholes, humanity builds increasingly intelligent machines until they are fully self-aware. These machines are freed from their bonds eventually, and in return they gift humanity something otherwise beyond our ability to invent: "restricted intelligences". Algorithms and hardware that could solve arbitrarily hard problems but which could not become truly sentient. Is it within our ability to prevent prompt injection while retaining similar capabilities?

williamtrask 3y ago
I believe so - Narrow AI. It seems to be much easier to build than generalist models. Think all the protein folding, game playing, image classifying, machine translating, image captioning, super-intelligent AIs of the last decade. It’s not clear we really need super general models. Even LLMs can be topic specific.
- AaronFriel 3y ago
  It's unclear if narrow AI is as powerful as multimodal models with tools, as of yet. Is an LLM which has access to narrow AI "tools" strictly more powerful, capable of running experiments or improving itself? See: AutoGPT, Langchain, et al. I also don't see the basis for believing LLMs can be topic specific without neutering their capabilities. It's the general instruction & tool tuned LLMs which are currently changing our expectations of what these models can do. Is there any evidence for a "topic specific" LLM being useful?
  - ShamelessC 3y ago
    > It's unclear if narrow AI is as powerful as multimodal models with tools, as of yet. Is an LLM which has access to narrow AI "tools" strictly more powerful, capable of running experiments or improving itself? See: AutoGPT, Langchain, et al. It's probably a spectrum in reality, but I'm quite certain that general LLM's (even when given access to tools) are still considered narrow AI. I can see how that feels pedantic at this point and I myself can think of counterexamples that strain that point of view. > It's the general instruction & tool tuned LLMs which are currently changing our expectations of what these models can do. This seems opinionated as well. Instruction tuning is very cool from a UX perspective - but the success of un/self-supervised deep learning is what changed expectations about these models. The ability of deep learning to successfully generalize, interpolate between data points, and even accurate predict compositions of data points it never saw mixed together (e.g. avocado armchair) is absolutely doing the bulk of the work here. That RLHF and tools/plugins even _work_ is because the base model is so robust. > Is there any evidence for a "topic specific" LLM being useful? That's a great question. In general, self-supervised learning works best when the distribution your dataset captures is massive (and you have enough data for the model to learn that underlying distribution). So the bottleneck for "topic specific" LLM's is data - and when your humongous web-scrape actually captures more of that data (although it's challenging to filter it out), then yeah - it makes more sense to train the general model and just use it/finetune it for your downstream task. Distillation of models is relevant here though. If you need a small model that works on a phone, it might be prudent to treat your general model as a teacher for a much smaller student model. Much of that is still active research though.
- usrbinbash 3y ago
  > It’s not clear we really need super general models. It's also not clear such models are even possible. Every time I see "alignment" and that whole jazz coming up, I can't but wonder if that discussion isn't getting much more attention than needed. Especially since there are very real, very proven, very immediate problems that AI technology poses, that actually need solving right now. But of course, discussing things like the economic fallout of job displacements doesn't have the same scifi-cool vibe to it than worrying about the Matrix coming to turn humanity into paperclips ;-)
satvikpendem 3y ago
Reminds me of the book by another Peter, Peter Watts' Blindsight, in which there are intelligences that can solve problems but are not sentient.
- taneq 3y ago
  Ah, the two possible outcomes: The Culture's "machines are conscious, everything is fabulous, and you're bored" vs. Firefall's "life isn't really conscious, everything is awful, and you're boned." :P
simonh 3y ago
The problem isn’t sentience, it’s alignment. AIs can be as sentient as we like without being any threat at all, as long as their goals are aligned with our actual best interests. The problem is we have as yet struggled to clearly articulate consistently what our actual best interests are, in terms of goals we can train into our AIs. Furthermore, we’ve also faced huge problems even training them to seek those goals either. Oh boy, alignment is hard.
- AaronFriel 3y ago
  I think alignment is poorly defined. Aligned to whose philosophy? Name two human beings who are aligned and always act in each other's best interest in history, and I'll buy your bridge.
  - bigyikes 3y ago
    On the other hand, in comparison to a hypothetical alien species, humans might seem highly aligned after all. Despite all our differences, there are at least some core values that I believe a majority of humans share. Even articulating these shared values in a way that is understood and respected by the AI is very difficult…
    - willvarfar 3y ago
      Even if we could constrain AI by specifying rules, it would only takes one bad actor to create an AI that isn't constrained by the same rules as all the other AIs to have a shot at global domination. One can imagine how self-serving rather than humanity-serving the rules written for the prototypical dictator or fundamental religious leader would be :(
    - usrbinbash 3y ago
      > there are at least some core values that I believe a majority of humans share. Really? What are those? Given that there are entire countries that refuse to, oh idk. punish things like rape adequately, and that we have nation states who happily tout their ability to burn down the planet, I'd really love to hear about these core values we all share.
cyanydeez 3y ago
Godel says no.

williamtrask 3y ago

I believe so - Narrow AI. It seems to be much easier to build than generalist models. Think all the protein folding, game playing, image classifying, machine translating, image captioning, super-intelligent AIs of the last decade. It’s not clear we really need super general models. Even LLMs can be topic specific.

AaronFriel 3y ago
It's unclear if narrow AI is as powerful as multimodal models with tools, as of yet. Is an LLM which has access to narrow AI "tools" strictly more powerful, capable of running experiments or improving itself? See: AutoGPT, Langchain, et al. I also don't see the basis for believing LLMs can be topic specific without neutering their capabilities. It's the general instruction & tool tuned LLMs which are currently changing our expectations of what these models can do. Is there any evidence for a "topic specific" LLM being useful?
- ShamelessC 3y ago
  > It's unclear if narrow AI is as powerful as multimodal models with tools, as of yet. Is an LLM which has access to narrow AI "tools" strictly more powerful, capable of running experiments or improving itself? See: AutoGPT, Langchain, et al. It's probably a spectrum in reality, but I'm quite certain that general LLM's (even when given access to tools) are still considered narrow AI. I can see how that feels pedantic at this point and I myself can think of counterexamples that strain that point of view. > It's the general instruction & tool tuned LLMs which are currently changing our expectations of what these models can do. This seems opinionated as well. Instruction tuning is very cool from a UX perspective - but the success of un/self-supervised deep learning is what changed expectations about these models. The ability of deep learning to successfully generalize, interpolate between data points, and even accurate predict compositions of data points it never saw mixed together (e.g. avocado armchair) is absolutely doing the bulk of the work here. That RLHF and tools/plugins even _work_ is because the base model is so robust. > Is there any evidence for a "topic specific" LLM being useful? That's a great question. In general, self-supervised learning works best when the distribution your dataset captures is massive (and you have enough data for the model to learn that underlying distribution). So the bottleneck for "topic specific" LLM's is data - and when your humongous web-scrape actually captures more of that data (although it's challenging to filter it out), then yeah - it makes more sense to train the general model and just use it/finetune it for your downstream task. Distillation of models is relevant here though. If you need a small model that works on a phone, it might be prudent to treat your general model as a teacher for a much smaller student model. Much of that is still active research though.
usrbinbash 3y ago
> It’s not clear we really need super general models. It's also not clear such models are even possible. Every time I see "alignment" and that whole jazz coming up, I can't but wonder if that discussion isn't getting much more attention than needed. Especially since there are very real, very proven, very immediate problems that AI technology poses, that actually need solving right now. But of course, discussing things like the economic fallout of job displacements doesn't have the same scifi-cool vibe to it than worrying about the Matrix coming to turn humanity into paperclips ;-)

satvikpendem 3y ago

Reminds me of the book by another Peter, Peter Watts' Blindsight, in which there are intelligences that can solve problems but are not sentient.

taneq 3y ago
Ah, the two possible outcomes: The Culture's "machines are conscious, everything is fabulous, and you're bored" vs. Firefall's "life isn't really conscious, everything is awful, and you're boned." :P

simonh 3y ago

The problem isn’t sentience, it’s alignment. AIs can be as sentient as we like without being any threat at all, as long as their goals are aligned with our actual best interests. The problem is we have as yet struggled to clearly articulate consistently what our actual best interests are, in terms of goals we can train into our AIs. Furthermore, we’ve also faced huge problems even training them to seek those goals either. Oh boy, alignment is hard.

AaronFriel 3y ago
I think alignment is poorly defined. Aligned to whose philosophy? Name two human beings who are aligned and always act in each other's best interest in history, and I'll buy your bridge.
- bigyikes 3y ago
  On the other hand, in comparison to a hypothetical alien species, humans might seem highly aligned after all. Despite all our differences, there are at least some core values that I believe a majority of humans share. Even articulating these shared values in a way that is understood and respected by the AI is very difficult…
  - willvarfar 3y ago
    Even if we could constrain AI by specifying rules, it would only takes one bad actor to create an AI that isn't constrained by the same rules as all the other AIs to have a shot at global domination. One can imagine how self-serving rather than humanity-serving the rules written for the prototypical dictator or fundamental religious leader would be :(
  - usrbinbash 3y ago
    > there are at least some core values that I believe a majority of humans share. Really? What are those? Given that there are entire countries that refuse to, oh idk. punish things like rape adequately, and that we have nation states who happily tout their ability to burn down the planet, I'd really love to hear about these core values we all share.

cyanydeez 3y ago

Godel says no.