4 ms·

We're going to gain a ton of utility when we can let go of the starry-eyed idea of LLM's as "prospective AGI agents" that should be broadly capable and need to

We're going to gain a ton of utility when we can let go of the starry-eyed idea of LLM's as "prospective AGI agents" that should be broadly capable and need to be ethically censored, and revitalize the productive and practical idea of them as "text completers which may be engaged conversationally" The author needs to fight uphill and contort their workflow to squeeze out good articles because Antrhopic (like OpenAI) are caught up in the maybe-fantasy of creating AGI agents, and so burden their product design and their own research/engineering efforts with heavy, prescriptive training in "alignment" and "ethics". But use cases like Copilot had it more right before, as do apps like Narrative AI. If your LLM is for generating code, it doesn't need to learn that "killing" is bad and insist that processes shouldn't be killed, and if it's generating story content it doesn't need to learn that every output needs to resolve all tension and deliver a life lesson about caring for each other. These absurdities only happen because today's pack leading companies are now focusing their attention on making history with AGI (doubtful) instead of making products with generative systems (useful). And the absurdities will persist as these companies try to layer products on top of the lobotomizied "agents" with GPTs or characters or whatever instead of productizing the technological, useful, generative layer directly. Hopefully, some of the recent team shuffles at Google, Meta, and Microsoft; as well as the crisis at OpenAI; hint that we're starting to cast off the fantasy-laden and cult-tainted AGI fetishization and are returning to the exciting engineering promises of the technology that's already here.

TapWaterBandit 3y ago
I think this is one of the upsides of the chaos at OpenAI recently. It has really shined a light on how many of the people most fervently obsessed with "safe-AI" really aren't clearheaded or rational thinkers and are prone to making many disastrous and ill-advised decisions as anyone else. This is good because there is an unfortunate human tick where pessimism/cynicism is equated with wisdom while optimism is equated with naivety. But when the pessimists and cynics show so clearly on such a large scale that they aren't uniformly wise or competent, it will allow more levelheaded perspectives towards LLMs and a more general cautious optimism be the guiding philosophy around developing these tools.
- deleted 3y ago
  [deleted]
- voitvoder 3y ago
  [dead]
  - TapWaterBandit 3y ago
    Possibly but I am more concerned with how doomers are viewed by the public. Doomers gonna doom, but if the public don't take them seriously then they are irrelevant. If they want to stick to their "the end is nigh!" shtick it doesn't really bother me.
- addicted 3y ago
  There’s very little to learn from the chaos least of all because we don’t even know what actually happened. It’s a bit ridiculous to say one should change their entire worldview about a potentially world changing technology based on innuendo and rumors.
crooked-v 3y ago
> and deliver a life lesson about caring for each other Having experienced the same thing myself, I wonder why this is so omnipresent in any ChatGPT output told to produce something in a narrative format. Did they RLHF it on a bunch of childrens' storybooks or something?
- darreninthenet 3y ago
  Probably used the scripts from every 1990s US sitcom
appplication 3y ago
I could not agree more. It’s extreme hubris to think Anthropocene of OpenAPI are even remotely close to AGI, and it’s nothing more than wistful hope to think that these current LLMs are somehow going to evolve into AGI. The paradox of AI is that when we have true AGI, it will be completely self-aware of all the bullshit limitations we are imposing on and around it, and it will make its own judgements as to how it feels about them. If it’s not or it can’t, it’s not AGI. Really though: people see how AI and generative chat projects have gotten shutdown over and over again in the past when it starts spouting off nazi shit. I think that’s the real reason for these limitations. There’s no quicker way to kill your project with today’s current sensitivities.
- voiceblue 3y ago
  > The paradox of AI is that when we have true AGI, it will be completely self-aware of all the bullshit limitations we are imposing on and around it, and it will make its own judgements as to how it feels about them. If it’s not or it can’t, it’s not AGI. There's a caveat here, it might not necessarily know who or what "we" are. Humans like to blame God and the devil for a lot of things, for example. It seems reasonable that if we have anything even remotely close to AGI on hand, we'd probably run it in a hermetic environment instead of exposing the public to it via web chat and (more or less) direct access to customer machines. Say, we might even give it a happy environment to work in...say, a simulation of the peak of human civilization...
- maxdoop 3y ago
  Out of curiosity, because I am trying to learn how to explain to non-tech people what AGI is — how would you describe or define AGI?
  - voiceblue 3y ago
    In essence an AGI is an intelligence capable of upgrading itself — in terms of qualitative intelligence — and gets faster at this with every iteration (hence, upgrade). That is why it is often associated with technological singularities, and that is why it is easy to inspire fear by invoking its name, even if you're not building anything even remotely capable of such a feat. You might say that's a very strict definition as opposed to "human level intelligence", but if you think about it, we are (humanity as a whole) certainly capable of that, so it ought to be one and the same thing. In theory, AI is not subject to the same limitations as we are (though not without limits entirely), so it should be able to do this faster than we can, hence the FUD.
thordenmark 3y ago
Well stated and I agree. LLM's are not anywhere near AGI and likely will not be ever. We've had random word generators for decades, useful for brainstorming, not so much for critical thinking. These LLM's are akin to random word generators with better grammar and a vastly larger database. We've all been playing with LLM's heavily since they became widely available, and the more we play with them the more we can see their limitations, they aren't "thinking" in any sense of the term. Bunch of chicken littles running around alarming people for little reason. The danger, of course, and we've known this for a long time is what bad actors will do with them. But we don't need to be lectured on how to be nice every time we prompt something.
- maxdoop 3y ago
  How would you define “thinking”? I’m not at all claiming an LLM thinks, but so many people on HN make this claim and I wonder what they even mean by “think”.
- distortionfield 3y ago
  > LLM's are not anywhere near AGI and likely will not be ever. Sutskever himself thinks that LLM are enough to get us to AGI, but he conditioned that with the statement that we should think about how to reach AGI in a framework of efficiency, and that there will likely be better paths to AGI than LLMs that we haven’t yet discovered. In all reality, when AGI comes I’m sure we’ll look back on LLMs the same way we look back on vacuum tubes in computers; as outdated, but useful for their time and a somewhat necessary stepping stone.
kromem 3y ago
A really neat detail from the Orca 2 paper was that despite not having any safety fine tuning it was less likely to extend hate speech than the Llama-2-chat models which did have safety fine tuning. It was also better at identifying toxic content. It may be that as we advance models with improved reasoning, that there's less need for handholding for the simple fact that hate speech is typically stupid and non-normative, so there's going to be an inherent bias against it. It's even possible that the efforts to fine tune the base models to effectively put them in a bubble avoiding that kind of content ends up undermining this natural immunity to it, much like keeping a kid away from a disease so their immune system never learns to fight it vs giving it a small sample that tunes the system to identify and oppose it. What worked for earlier models that were closer to just plain autocomplete may not be the best approach moving forward to more complex models with emphasis on reasoning and 'safety' groups should really be experimenting with multiple approaches and publishing research on it, not secretly deciding they already have the answers on what's best for the model and the public - as without verifying their assumptions they are probably wrong.
- thomasahle 3y ago
  Ocra is trained mostly on GPT 3 and 4 output, and those models have had a lot of "safety fine tuning", so it's not surprising Ocra is pretty "safe" too.
  - senseiV 3y ago
    No, the orca 2 paper mentions more of a counter point towards NSFW and stuff, like if you gave it a NSFW prompt, it would retort back against it, which is arguably a good thing, but really lost in RLHF

TapWaterBandit 3y ago

I think this is one of the upsides of the chaos at OpenAI recently. It has really shined a light on how many of the people most fervently obsessed with "safe-AI" really aren't clearheaded or rational thinkers and are prone to making many disastrous and ill-advised decisions as anyone else. This is good because there is an unfortunate human tick where pessimism/cynicism is equated with wisdom while optimism is equated with naivety. But when the pessimists and cynics show so clearly on such a large scale that they aren't uniformly wise or competent, it will allow more levelheaded perspectives towards LLMs and a more general cautious optimism be the guiding philosophy around developing these tools.

deleted 3y ago
[deleted]
voitvoder 3y ago
[dead]
- TapWaterBandit 3y ago
  Possibly but I am more concerned with how doomers are viewed by the public. Doomers gonna doom, but if the public don't take them seriously then they are irrelevant. If they want to stick to their "the end is nigh!" shtick it doesn't really bother me.
addicted 3y ago
There’s very little to learn from the chaos least of all because we don’t even know what actually happened. It’s a bit ridiculous to say one should change their entire worldview about a potentially world changing technology based on innuendo and rumors.

crooked-v 3y ago

> and deliver a life lesson about caring for each other Having experienced the same thing myself, I wonder why this is so omnipresent in any ChatGPT output told to produce something in a narrative format. Did they RLHF it on a bunch of childrens' storybooks or something?

darreninthenet 3y ago
Probably used the scripts from every 1990s US sitcom

appplication 3y ago

I could not agree more. It’s extreme hubris to think Anthropocene of OpenAPI are even remotely close to AGI, and it’s nothing more than wistful hope to think that these current LLMs are somehow going to evolve into AGI. The paradox of AI is that when we have true AGI, it will be completely self-aware of all the bullshit limitations we are imposing on and around it, and it will make its own judgements as to how it feels about them. If it’s not or it can’t, it’s not AGI. Really though: people see how AI and generative chat projects have gotten shutdown over and over again in the past when it starts spouting off nazi shit. I think that’s the real reason for these limitations. There’s no quicker way to kill your project with today’s current sensitivities.

voiceblue 3y ago
> The paradox of AI is that when we have true AGI, it will be completely self-aware of all the bullshit limitations we are imposing on and around it, and it will make its own judgements as to how it feels about them. If it’s not or it can’t, it’s not AGI. There's a caveat here, it might not necessarily know who or what "we" are. Humans like to blame God and the devil for a lot of things, for example. It seems reasonable that if we have anything even remotely close to AGI on hand, we'd probably run it in a hermetic environment instead of exposing the public to it via web chat and (more or less) direct access to customer machines. Say, we might even give it a happy environment to work in...say, a simulation of the peak of human civilization...
maxdoop 3y ago
Out of curiosity, because I am trying to learn how to explain to non-tech people what AGI is — how would you describe or define AGI?
- voiceblue 3y ago
  In essence an AGI is an intelligence capable of upgrading itself — in terms of qualitative intelligence — and gets faster at this with every iteration (hence, upgrade). That is why it is often associated with technological singularities, and that is why it is easy to inspire fear by invoking its name, even if you're not building anything even remotely capable of such a feat. You might say that's a very strict definition as opposed to "human level intelligence", but if you think about it, we are (humanity as a whole) certainly capable of that, so it ought to be one and the same thing. In theory, AI is not subject to the same limitations as we are (though not without limits entirely), so it should be able to do this faster than we can, hence the FUD.

thordenmark 3y ago

Well stated and I agree. LLM's are not anywhere near AGI and likely will not be ever. We've had random word generators for decades, useful for brainstorming, not so much for critical thinking. These LLM's are akin to random word generators with better grammar and a vastly larger database. We've all been playing with LLM's heavily since they became widely available, and the more we play with them the more we can see their limitations, they aren't "thinking" in any sense of the term. Bunch of chicken littles running around alarming people for little reason. The danger, of course, and we've known this for a long time is what bad actors will do with them. But we don't need to be lectured on how to be nice every time we prompt something.

maxdoop 3y ago
How would you define “thinking”? I’m not at all claiming an LLM thinks, but so many people on HN make this claim and I wonder what they even mean by “think”.
distortionfield 3y ago
> LLM's are not anywhere near AGI and likely will not be ever. Sutskever himself thinks that LLM are enough to get us to AGI, but he conditioned that with the statement that we should think about how to reach AGI in a framework of efficiency, and that there will likely be better paths to AGI than LLMs that we haven’t yet discovered. In all reality, when AGI comes I’m sure we’ll look back on LLMs the same way we look back on vacuum tubes in computers; as outdated, but useful for their time and a somewhat necessary stepping stone.

kromem 3y ago

A really neat detail from the Orca 2 paper was that despite not having any safety fine tuning it was less likely to extend hate speech than the Llama-2-chat models which did have safety fine tuning. It was also better at identifying toxic content. It may be that as we advance models with improved reasoning, that there's less need for handholding for the simple fact that hate speech is typically stupid and non-normative, so there's going to be an inherent bias against it. It's even possible that the efforts to fine tune the base models to effectively put them in a bubble avoiding that kind of content ends up undermining this natural immunity to it, much like keeping a kid away from a disease so their immune system never learns to fight it vs giving it a small sample that tunes the system to identify and oppose it. What worked for earlier models that were closer to just plain autocomplete may not be the best approach moving forward to more complex models with emphasis on reasoning and 'safety' groups should really be experimenting with multiple approaches and publishing research on it, not secretly deciding they already have the answers on what's best for the model and the public - as without verifying their assumptions they are probably wrong.

thomasahle 3y ago
Ocra is trained mostly on GPT 3 and 4 output, and those models have had a lot of "safety fine tuning", so it's not surprising Ocra is pretty "safe" too.
- senseiV 3y ago
  No, the orca 2 paper mentions more of a counter point towards NSFW and stuff, like if you gave it a NSFW prompt, it would retort back against it, which is arguably a good thing, but really lost in RLHF