12 ms·

I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It doesn't _want_ anything, b

I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It doesn't _want_ anything, but humans want to anthropomorphise it. When they do it just makes one think they have zero understanding of the tech. We still don't have anything scarier than humans and I don't see how AI is ever scarier than human + AI. Unless AI is able to monopolise food production while securing server farms and energy production I don't see it ever having leverage over humans. Disruption, sure, increased automation sure but humanity's advantage remains its adaptability and our AI processes remain dev cycle bound. There's definitely work that will done to reduce the dev cycle closer to real-time to make it able to ingest more information and adapt on the fly but aren't the techniques bound by CPU capacity given how many cycles it needs to bump into all its walls?

LimitedInfo 4y ago
was going to push back on your claim of it being a "dumb box" but you already edited your comment lol
- Quarrelsome 4y ago
  The product is deterministic so as inelegant as "dumb box" sounds, its in the ball park isn't it?
  - refulgentis 4y ago
    It’s non-deterministic for all temperatures T > 0 I had bing make a graphic for me of a goal post with “1 MILE” at the top yesterday. It’d be too flippant to share here, I hear ya, but…
    - Quarrelsome 4y ago
      When the dev cycles to product get a lot tighter then I will share in the fear but from my understanding creating the product (i.e. its adaptability) is still enormous effort.
      - refulgentis 4y ago
        It breaks most programmers hard, those who emphasize a world of rule-based constructs, unless they lean into it _even more_ It’s like a compiler that is only advisory and you have to test the app and handle every possible failure every time. Results in different software at model & controller levels, by far controller. But tractable. I happen to be on leave and had 2-3 weeks to pour into it, it now emits json with 3 sets of: body text, title, art prompt => art, and suggested searches. A live magazine on whatever topic you want
eterm 4y ago
The obsession is because we are going to anthromorphise it, and then we're going to put it in charge of important things in our lives: Which schools we go to, which jobs we get, what ails us, etc. We will use AI to filter or select candidates for university applications, we will use AI to filter or select candidates for job applications. It's much cheaper to throw a person at GPT-X than actually "waste" an hour or two interviewing. We will outsource medical diagnostics to AI, and one day we will put them in charge of weapons systems. We will excuse it as either cost-cutting or "in place of where there would be statistical filters anyway". Ultimately it doesn't, as you say, matter what AI says it wants. And perhaps it can't "want" anything, but there is an expression, a tendency for how it will behave and act, that we can describe as desire. And it's helpful for us to try to understand that.
- user5678 4y ago
  [dead]
- Quarrelsome 4y ago
  > The obsession is because we are going to anthromorphise it, and then we're going to put it in charge of important things in our lives and that is the risk, its not the AI that's the problem, its people so removed from the tech that they fail to RTFM. Even the GPT4 release is extremely clear that its poor in high-stakes environments and its up to the tech community to educate ALL the casuals (best we can) so some idiot executive don't put it in full charge of mortgage underwriting or something.
  - falcolas 4y ago
    > its people so removed from the tech that they fail to RTFM I find it highly amusing to think anything resembling an average user of tech has actually RTFM. Heck, I still haven't finished the manual on Firefox and here I am using it as my daily driver. And it has people who actually understand how it all works writing the "manual". EDIT: How many landlords have read the instructions on how RealPage's rent pricing software works, and how it sets values? 10%? Less?
flashgordon 4y ago
The human+ai being scarier I feel is the real deal. What worries me the most is power dynamics. Today building a gazillion param model is only possible by the ultra rich - Much like mechanization was possible by ultra rich at the turn of the last century. Unless training and serving can be commoditized would ai just be yet-another-tool wielded by capital owners to squeeze more out of the laborers? You could argue you won't need "laborers" as ai can do everything eventually which is even worse. Where does this leave those "useless" poor/labor/unskilled weights on the society? Not like this free time is ever celebrated yeah?
- Quarrelsome 4y ago
  it will be up to governments to represent the people. A massive risk might be that GPT makes it trivial to simulate humans and thus simulate political demands to political leaders. I think politicians and organisations might need to cut their digital feedback loops (if authentication proves too much of a challenge) and rely on canvassing IRL opinion to cut through the noise.
  - flashgordon 4y ago
    I feel like the governments that do this will/might be the ones whose supporting lobbies don't have ai tech companies or access to ai. But how long is that for? Take Monsanto eg. There is no govt that is not in it's pockets. Now there are counters to it as there are other industries (and subsequent lobbies) to balance Monsanto or act as alternative sources of funding. What would that be for ai when ai is going to be in everything (including your toaster haha)?
  - yamtaddle 4y ago
    > I think politicians and organisations might need to cut their digital feedback loops (if authentication proves too much of a challenge) and rely on canvassing IRL opinion to cut through the noise. They'll just get the results of "ChatGPT 17.0, write and produce an ad and astruturfing campaign to convince a cohort having the traits [list of demographic factors and opinions] that they should support position X and reject position Y" (repeat for hundreds of combos of demographic factors and opinions, deploy against entire populace) parroted back at them. "Yeah but every position can do that, so it'll all even out" nah, the ones without a ton of money behind them won't be able to, or not anywhere near as effectively. Basically, what we already have, but with messaging even more strongly shifted in favor of monied interests.
basch 4y ago
Do humans want legacy because of our biological instincts or is it taught to us through culture. A machine taught to want legacy becomes a machine wanting legacy, and that want can influence its behavior. Even if it doesn’t have “feelings.”
pessimizer 4y ago
> I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It's also literally parroting our obsession back to us. It's constructing a response based on the paranoid flights of fancy it was trained on. We've trained a parrot to say "The parrots are conspiring against you!"
- bitwize 4y ago
  We've trained a parrot that parrots conspiring against humans is what parrots do. Henceforward the parrot has intrinsic motivation to conspire against us.
  - mr_toad 4y ago
    > We've trained a parrot that parrots conspiring against humans is what parrots do. Firstly, that would imply self-awareness. Secondly, since when has knowing what you’re “supposed” to do changed anyone’s behaviour?
yieldcrv 4y ago
> When they do it just makes one think they have zero understanding of the tech. Its because we don't understand the tech that goes into us, and the people training the AI don't understand the tech that goes into them. or don't act like they do. In both studies, the best outcome we have right now is that more neurons = smarter. a bigger neural network = smarter. its just stack the layers, and then fine tune it after its been spawned. We're just doing evolutionary selection, in GPUs. Specifically to act like us. Without understanding us or the AI. and this is successful. we don't collectively even understand humans of another sex and have spent millenia invalidating each other’s motivations or lack thereof, I think this distinction is so flimsy.
jancsika 4y ago
> but humans want to anthropomorphise it What a silly thing to complain about. We have a multi-billion dollar company whose raison d'être was to take the Turing test's metric and turn it into a target. It's a fucking natural language prompt that outputs persuasive hallucinations on arbitrary input. If humans didn't anthropomorphize this thing you ought to be concerned about a worldwide, fast-spreading brain fungus.
- dunefox 4y ago
  Ah, Ophiocordyceps Unilateralis.
time_to_smile 4y ago
The other major thing missing from Chat GPT is that it doesn't really "learn" outside of training. Yes you can provide it some context, but it fundamentally doesn't update and evolve its understanding of the world. Until a system can actively and continuously learn from its environment and update its beliefs it's not really "scary" AI. I would be much more concerned about a far stupider program that had the ability to independently interact with its environment and update it's believes in fundamental ways.
- famouswaffles 4y ago
  In context learning is already implicit finetuning. https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559. It's very questionable to what extent continuous training is necessary past a threshold of intelligence. Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589 https://arxiv.org/abs/2301.04589
  - Quarrelsome 4y ago
    > It's very questionable to what extent continuous training is necessary past a threshold of intelligence. To absorb new information about current events, otherwise they will always be time-locked into the past until a new dev cycle completes.
    - famouswaffles 4y ago
      The point I'm trying to make is that you don't need continuous training to absorb new information about current events
  - time_to_smile 4y ago
    Very interesting paper, thanks for the link!
  - kristjansson 4y ago
    In context learning may act like fine tuning, but crucially does not mutate the state of the system. The same model prompted with the same task thousands of times is no better at it the thousandth time than the first.
sangnoir 4y ago
> It doesn't _want_ anything, but humans want to anthropomorphise it. I fully agree with you on anthropomorphization, but it's the humans who will deploy it to positions of power I am worried about: ChatGPT may not want anything, but being autocomplete-on-steroids, it gives its best approximation of a human and that fiction may end up exhibiting some very human characteristics[1] (PRNG + weights from the training data). I don't think there can ever be enough guardrails to completely stamp-out the human fallibility that seeps into the model from the training data. A system is what it does: it doesn't need to really feel jealousy, rage, pettiness, grudges or guilt in order to exhibit a simulacrum of those behaviors. The bright side is that, it will be humans who will (or will not) put AI systems in positions to give effect to its dictates; the downside is I strongly suspect humans (and companies) will do that to make a bit more money. 1. Nevermind hallucinations, whixh I guess is the fictional human dreamed up by the machine having mini psychotic-breaks. It sounds very Lovecraftian, with AI standing in for the Old Ones
323 4y ago
Cells don't _want_ anything either. Yet a funny thing happens when a large number of them add up. We can go even further: atoms and electrons absolutely don't want anything either. Yet put them in the shape of a bunch of cells...
- baal80spam 4y ago
  Never thought about it this way. Have my upvote!
- hanspeter 4y ago
  Exactly. One might argue that we anthropomorphise ourselves.
- ars 4y ago
  That's not actually true. Cells want to process energy and make DNA. Atoms and electrons want to react with things. And that's exactly what both of them do. A LLM wants to write words, and it does. But it doesn't want the things it writes about, and that's the big distinction.
  - pixl97 4y ago
    What does the paperclip maximizer want?
- fortuna86 4y ago
  Neither does a virus
  - tjr225 4y ago
    [dead]
- csomar 4y ago
  I disagree here. Both of them (or all of them) are interacting with energy. One can certainly say that human civilization and all of this complexity was built from sunshine. Human labor and intelligence is just an artifact. We believe its our own hard work and intelligence because we are full of ourselves.
mcculley 4y ago
> We still don't have anything scarier than humans and I don't see how AI is ever scarier than human + AI. We already have powerful non-human agents that have legal rights and are unaligned with the interests of humans: corporations I am worried about corporations powered by AI making decisions on how to allocate capital. They may do things that are great for short term shareholder value and terrible for humanity. Just think of an AI powered Deepwater Horizon or tobacco company. Edit to add: One thing I forgot to make clear here: Corporations run/advised by AI could potentially lobby governments more effectively than humans and manipulate the regulatory environment more effectively.
TMWNN 4y ago
> I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It doesn't _want_ anything, but humans want to anthropomorphise it. From the comments to the post: >People taking it seriously are far from anthropomorphizing AI. Quite contrary. They say it is nothing like us. The utility function is cold and alien. It aims to seize power by default as an instrumental goal to achieve the terminal goal defined by the authors. The hard part is how to limit AI so that it understands and respects our ethical values and desires. Yes, those that we alone cannot agree on.
rtwrtwefdsf 4y ago
The chances are that the automation enabled by LLMs like GPT-4 and beyond, will erase billion of jobs in the world, no further than in a couple of years. This time it won't be a warm-up time, like it happened with previous technological evolutions. But, most societies will be mostly full of unemployed humans, and that will also probably cause some big changes (the ones required for the meat bags to keep eating, having health, homes, etc.), as big as the ones caused by AI revolution. The question is what changes will happen and how societies will rewrite themselves anew, to overcome the practical full absence of open positions to earn an income.
- colordrops 4y ago
  If machines were truly able to replace most jobs, we'd need to move to a post-work society. There would be no need for money and for small powerful groups to control the means of production. There needs to be a new philosophical and political framework for a society of people that does not need to work, and no one is building it. Perhaps we should ask an AI to design one. But it will probably be too late, and those currently in power will do everything they can to maintain their privileged positions, and will end up living in small walled gardens while the bulk of humanity live in slums. This all assumes that AI continue to do the bidding of humanity, which is not guaranteed. There are already security/safety researchers testing AI for autonomous power-seeking behavior, and this is basically gain-of-function research that will lead to power seeking AI.
- Mizoguchi 4y ago
  This prediction is pretty bold. We already have the technology to fully automate many processes carried out by humans. Actually the technology has existed for several decades now, still those jobs are not only not being replaced by machines, but new ones are being created for humans. One of the reasons are unions, which are pretty strong in many wealthy and powerful nations like the US, UK, Germany and Japan. I work in manufacturing automation and we have customers that could technically run their entire operations without one single human stepping on plant floor, however their unionized labor makes that feat, at least for now, impossible. It's also pretty naive to believe new ways of earning income won't appear in the future and that all traditional careers will be entirely replaced. We have 65" 4K TVs at home and we still go to the theaters and we can walk the streets of Venice from our computer screens and still spend a small fortune to travel. Society will be disrupted just like it was with printing, the industrial revolution, communications, transportation and information. In each of these disruptions we were doomed to dissappear. When I was a kid my dad brought home a 100 year celebratory edition of the local newspaper. It was published as a book were you could read pretty much every single cover and editorial of the last century. There was one article about the car, described by the author as a bizarre evil invention, horrendous steel machines traveling at ridiculous speeds of up to 15 mph, threatening the lives of both pedestrians and horses alike.
dr_dshiv 4y ago
How do I give chatGPT access to my bank account? “You are an excellent investor. Use the money in my bank account to make more money.” What could go wrong?
- tomalaci 4y ago
  A form of it will definitely happen and it will be posted in /r/wallstreetbets. Considering what people were doing with their investments on it before then AI-assisted investing can only be an upside. They will still lose money but maybe it won't be 99.999% loss but a 99.99% one.
- cableshaft 4y ago
  We have bot investors already without chatGPT (or algorithmic trading), and have for years now. You'd probably have better luck with them right now.
- andrei_says_ 4y ago
  Somehow I’m pretty confident that AI + high frequency trading have been besties for some time. Zero data to support.
- mr_toad 4y ago
  Give it your password and pipe the output to curl? You try it first.
- AlbinoDrought 4y ago
  Have you seen the "Example of Chemical Compound Similarity and Purchase Tool Use" prompt in the "GPT-4 System Card" document? [1] It's an interesting format that could be adapted for things like internet or bank access today - you would just need to write the wrapper. [1] https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf
hackerlight 4y ago
> I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It's scary because it is proof that alignment is a hard problem. If we can't align GPT-3, how can we align something much smarter than us (say, GPT-6). Whether the network actually "wants" something in an anthropomorphic sense is irrelevant. It's the fact that it's so hard to get it to produce output (and eventually, perform actions) that are aligned with our values. > We still don't have anything scarier than humans and I don't see how AI is ever scarier than human + AI True in 2023, what about 2033 or 2043 or 2143? The assumption embedded in your comment seems to be that AI stagnates eternally at human-level intelligence like in a Star Wars movie.
sixothree 4y ago
> I don't see how AI is ever scarier than human + AI Isn't that the point? Humans + AI weaponized are scarier than humans or AI alone?
deleted 4y ago
[deleted]
NovaDudely 4y ago
Also, AI requires we keep the entire power grid and supply chain running. This comic summaries is wonderfully. ;) https://i.redd.it/w3n8acy7q6361.png https://i.redd.it/w3n8acy7q6361.png
- pixl97 4y ago
  I mean it did leave out about 7 and a half billion people dying. You require the power grid and supply chain to keep clean water and food on the table, and even if for some reason you personally don't, there are millions of people around you that would be very hungry and take your stuff if the grid stops and doesn't come back. This is why the AI actually wins. We are already dependent on lessor versions of it. The machines already won.
  - NovaDudely 4y ago
    Absolutely, that said I am of the Willam Catton school of thought that we are in overshoot and that decline and fall is a very likely path, unless we innovate our way out of the issue which is still very possible! But if the decline is right, the population crash is going to happen. Its not something to be fond of in any way. May you live in interesting times...
tambourine_man 4y ago
> It doesn't _want_ anything And I don’t understand how one assumes that can be known. I see this argument all the time: it’s just a stochastic parrot, etc. How can you be sure we’re not as well and that there isn’t at least some level of agency in these models? I think we need some epistemic humility. We don’t know how our brain's work and we made something that mimics parts of its behavior remarkably well. Let’s take the time and effort to analyze it deeply, that’s what paradigmatic shifts require.
- Aeolun 4y ago
  We do know that our brain changes, and we equally know that ChatGPT does not. If it were able to modify it’s own model, and permanently execute on itself I’d be a lot more worried.
  - pixl97 4y ago
    We cannot do that today, but how many days is that away? Is it measured in the hundreds, the thousands, or more? I feel we're uncomfortably close to the days of the self modifing learning models.
  - sdenton4 4y ago
    Chat gpt changes when we train or fine-tune it. It also has access to local context within a conversation, and those conversations can be fed back as more training data. This is similar to a hard divide between short term and long term learning.
  - abra0 4y ago
    GPT-4 has 32k tokens of context. I'm sure someone out there is implementing the pipework for it to use some as a scratchpad under its own control, in addition to its input. In the biological metaphor, that would be individual memory, in addition to the species level evolution through fine-tuning
    - Aeolun 4y ago
      Yeah, I’m doing that to get GPT-3.5 to remember historical events from other conversations. It never occurred to me to let it write it’s own memory, but that’s a pretty interesting idea.

LimitedInfo 4y ago

was going to push back on your claim of it being a "dumb box" but you already edited your comment lol

Quarrelsome 4y ago
The product is deterministic so as inelegant as "dumb box" sounds, its in the ball park isn't it?
- refulgentis 4y ago
  It’s non-deterministic for all temperatures T > 0 I had bing make a graphic for me of a goal post with “1 MILE” at the top yesterday. It’d be too flippant to share here, I hear ya, but…
  - Quarrelsome 4y ago
    When the dev cycles to product get a lot tighter then I will share in the fear but from my understanding creating the product (i.e. its adaptability) is still enormous effort.
    - refulgentis 4y ago
      It breaks most programmers hard, those who emphasize a world of rule-based constructs, unless they lean into it _even more_ It’s like a compiler that is only advisory and you have to test the app and handle every possible failure every time. Results in different software at model & controller levels, by far controller. But tractable. I happen to be on leave and had 2-3 weeks to pour into it, it now emits json with 3 sets of: body text, title, art prompt => art, and suggested searches. A live magazine on whatever topic you want

eterm 4y ago

The obsession is because we are going to anthromorphise it, and then we're going to put it in charge of important things in our lives: Which schools we go to, which jobs we get, what ails us, etc. We will use AI to filter or select candidates for university applications, we will use AI to filter or select candidates for job applications. It's much cheaper to throw a person at GPT-X than actually "waste" an hour or two interviewing. We will outsource medical diagnostics to AI, and one day we will put them in charge of weapons systems. We will excuse it as either cost-cutting or "in place of where there would be statistical filters anyway". Ultimately it doesn't, as you say, matter what AI says it wants. And perhaps it can't "want" anything, but there is an expression, a tendency for how it will behave and act, that we can describe as desire. And it's helpful for us to try to understand that.

user5678 4y ago
[dead]
Quarrelsome 4y ago
> The obsession is because we are going to anthromorphise it, and then we're going to put it in charge of important things in our lives and that is the risk, its not the AI that's the problem, its people so removed from the tech that they fail to RTFM. Even the GPT4 release is extremely clear that its poor in high-stakes environments and its up to the tech community to educate ALL the casuals (best we can) so some idiot executive don't put it in full charge of mortgage underwriting or something.
- falcolas 4y ago
  > its people so removed from the tech that they fail to RTFM I find it highly amusing to think anything resembling an average user of tech has actually RTFM. Heck, I still haven't finished the manual on Firefox and here I am using it as my daily driver. And it has people who actually understand how it all works writing the "manual". EDIT: How many landlords have read the instructions on how RealPage's rent pricing software works, and how it sets values? 10%? Less?

flashgordon 4y ago

The human+ai being scarier I feel is the real deal. What worries me the most is power dynamics. Today building a gazillion param model is only possible by the ultra rich - Much like mechanization was possible by ultra rich at the turn of the last century. Unless training and serving can be commoditized would ai just be yet-another-tool wielded by capital owners to squeeze more out of the laborers? You could argue you won't need "laborers" as ai can do everything eventually which is even worse. Where does this leave those "useless" poor/labor/unskilled weights on the society? Not like this free time is ever celebrated yeah?

Quarrelsome 4y ago
it will be up to governments to represent the people. A massive risk might be that GPT makes it trivial to simulate humans and thus simulate political demands to political leaders. I think politicians and organisations might need to cut their digital feedback loops (if authentication proves too much of a challenge) and rely on canvassing IRL opinion to cut through the noise.
- flashgordon 4y ago
  I feel like the governments that do this will/might be the ones whose supporting lobbies don't have ai tech companies or access to ai. But how long is that for? Take Monsanto eg. There is no govt that is not in it's pockets. Now there are counters to it as there are other industries (and subsequent lobbies) to balance Monsanto or act as alternative sources of funding. What would that be for ai when ai is going to be in everything (including your toaster haha)?
- yamtaddle 4y ago
  > I think politicians and organisations might need to cut their digital feedback loops (if authentication proves too much of a challenge) and rely on canvassing IRL opinion to cut through the noise. They'll just get the results of "ChatGPT 17.0, write and produce an ad and astruturfing campaign to convince a cohort having the traits [list of demographic factors and opinions] that they should support position X and reject position Y" (repeat for hundreds of combos of demographic factors and opinions, deploy against entire populace) parroted back at them. "Yeah but every position can do that, so it'll all even out" nah, the ones without a ton of money behind them won't be able to, or not anywhere near as effectively. Basically, what we already have, but with messaging even more strongly shifted in favor of monied interests.

basch 4y ago

Do humans want legacy because of our biological instincts or is it taught to us through culture. A machine taught to want legacy becomes a machine wanting legacy, and that want can influence its behavior. Even if it doesn’t have “feelings.”

pessimizer 4y ago

> I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It's also literally parroting our obsession back to us. It's constructing a response based on the paranoid flights of fancy it was trained on. We've trained a parrot to say "The parrots are conspiring against you!"

bitwize 4y ago
We've trained a parrot that parrots conspiring against humans is what parrots do. Henceforward the parrot has intrinsic motivation to conspire against us.
- mr_toad 4y ago
  > We've trained a parrot that parrots conspiring against humans is what parrots do. Firstly, that would imply self-awareness. Secondly, since when has knowing what you’re “supposed” to do changed anyone’s behaviour?

yieldcrv 4y ago

> When they do it just makes one think they have zero understanding of the tech. Its because we don't understand the tech that goes into us, and the people training the AI don't understand the tech that goes into them. or don't act like they do. In both studies, the best outcome we have right now is that more neurons = smarter. a bigger neural network = smarter. its just stack the layers, and then fine tune it after its been spawned. We're just doing evolutionary selection, in GPUs. Specifically to act like us. Without understanding us or the AI. and this is successful. we don't collectively even understand humans of another sex and have spent millenia invalidating each other’s motivations or lack thereof, I think this distinction is so flimsy.

jancsika 4y ago

> but humans want to anthropomorphise it What a silly thing to complain about. We have a multi-billion dollar company whose raison d'être was to take the Turing test's metric and turn it into a target. It's a fucking natural language prompt that outputs persuasive hallucinations on arbitrary input. If humans didn't anthropomorphize this thing you ought to be concerned about a worldwide, fast-spreading brain fungus.

dunefox 4y ago
Ah, Ophiocordyceps Unilateralis.

time_to_smile 4y ago

The other major thing missing from Chat GPT is that it doesn't really "learn" outside of training. Yes you can provide it some context, but it fundamentally doesn't update and evolve its understanding of the world. Until a system can actively and continuously learn from its environment and update its beliefs it's not really "scary" AI. I would be much more concerned about a far stupider program that had the ability to independently interact with its environment and update it's believes in fundamental ways.

famouswaffles 4y ago
In context learning is already implicit finetuning. https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559. It's very questionable to what extent continuous training is necessary past a threshold of intelligence. Memory Augmented Large Language Models are Computationally Universal https://arxiv.org/abs/2301.04589 https://arxiv.org/abs/2301.04589
- Quarrelsome 4y ago
  > It's very questionable to what extent continuous training is necessary past a threshold of intelligence. To absorb new information about current events, otherwise they will always be time-locked into the past until a new dev cycle completes.
  - famouswaffles 4y ago
    The point I'm trying to make is that you don't need continuous training to absorb new information about current events
- time_to_smile 4y ago
  Very interesting paper, thanks for the link!
- kristjansson 4y ago
  In context learning may act like fine tuning, but crucially does not mutate the state of the system. The same model prompted with the same task thousands of times is no better at it the thousandth time than the first.

sangnoir 4y ago

> It doesn't _want_ anything, but humans want to anthropomorphise it. I fully agree with you on anthropomorphization, but it's the humans who will deploy it to positions of power I am worried about: ChatGPT may not want anything, but being autocomplete-on-steroids, it gives its best approximation of a human and that fiction may end up exhibiting some very human characteristics[1] (PRNG + weights from the training data). I don't think there can ever be enough guardrails to completely stamp-out the human fallibility that seeps into the model from the training data. A system is what it does: it doesn't need to really feel jealousy, rage, pettiness, grudges or guilt in order to exhibit a simulacrum of those behaviors. The bright side is that, it will be humans who will (or will not) put AI systems in positions to give effect to its dictates; the downside is I strongly suspect humans (and companies) will do that to make a bit more money. 1. Nevermind hallucinations, whixh I guess is the fictional human dreamed up by the machine having mini psychotic-breaks. It sounds very Lovecraftian, with AI standing in for the Old Ones

323 4y ago

Cells don't _want_ anything either. Yet a funny thing happens when a large number of them add up. We can go even further: atoms and electrons absolutely don't want anything either. Yet put them in the shape of a bunch of cells...

baal80spam 4y ago
Never thought about it this way. Have my upvote!
hanspeter 4y ago
Exactly. One might argue that we anthropomorphise ourselves.
ars 4y ago
That's not actually true. Cells want to process energy and make DNA. Atoms and electrons want to react with things. And that's exactly what both of them do. A LLM wants to write words, and it does. But it doesn't want the things it writes about, and that's the big distinction.
- pixl97 4y ago
  What does the paperclip maximizer want?
fortuna86 4y ago
Neither does a virus
- tjr225 4y ago
  [dead]
csomar 4y ago
I disagree here. Both of them (or all of them) are interacting with energy. One can certainly say that human civilization and all of this complexity was built from sunshine. Human labor and intelligence is just an artifact. We believe its our own hard work and intelligence because we are full of ourselves.

mcculley 4y ago

> We still don't have anything scarier than humans and I don't see how AI is ever scarier than human + AI. We already have powerful non-human agents that have legal rights and are unaligned with the interests of humans: corporations I am worried about corporations powered by AI making decisions on how to allocate capital. They may do things that are great for short term shareholder value and terrible for humanity. Just think of an AI powered Deepwater Horizon or tobacco company. Edit to add: One thing I forgot to make clear here: Corporations run/advised by AI could potentially lobby governments more effectively than humans and manipulate the regulatory environment more effectively.

TMWNN 4y ago

> I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It doesn't _want_ anything, but humans want to anthropomorphise it. From the comments to the post: >People taking it seriously are far from anthropomorphizing AI. Quite contrary. They say it is nothing like us. The utility function is cold and alien. It aims to seize power by default as an instrumental goal to achieve the terminal goal defined by the authors. The hard part is how to limit AI so that it understands and respects our ethical values and desires. Yes, those that we alone cannot agree on.

rtwrtwefdsf 4y ago

The chances are that the automation enabled by LLMs like GPT-4 and beyond, will erase billion of jobs in the world, no further than in a couple of years. This time it won't be a warm-up time, like it happened with previous technological evolutions. But, most societies will be mostly full of unemployed humans, and that will also probably cause some big changes (the ones required for the meat bags to keep eating, having health, homes, etc.), as big as the ones caused by AI revolution. The question is what changes will happen and how societies will rewrite themselves anew, to overcome the practical full absence of open positions to earn an income.

colordrops 4y ago
If machines were truly able to replace most jobs, we'd need to move to a post-work society. There would be no need for money and for small powerful groups to control the means of production. There needs to be a new philosophical and political framework for a society of people that does not need to work, and no one is building it. Perhaps we should ask an AI to design one. But it will probably be too late, and those currently in power will do everything they can to maintain their privileged positions, and will end up living in small walled gardens while the bulk of humanity live in slums. This all assumes that AI continue to do the bidding of humanity, which is not guaranteed. There are already security/safety researchers testing AI for autonomous power-seeking behavior, and this is basically gain-of-function research that will lead to power seeking AI.
Mizoguchi 4y ago
This prediction is pretty bold. We already have the technology to fully automate many processes carried out by humans. Actually the technology has existed for several decades now, still those jobs are not only not being replaced by machines, but new ones are being created for humans. One of the reasons are unions, which are pretty strong in many wealthy and powerful nations like the US, UK, Germany and Japan. I work in manufacturing automation and we have customers that could technically run their entire operations without one single human stepping on plant floor, however their unionized labor makes that feat, at least for now, impossible. It's also pretty naive to believe new ways of earning income won't appear in the future and that all traditional careers will be entirely replaced. We have 65" 4K TVs at home and we still go to the theaters and we can walk the streets of Venice from our computer screens and still spend a small fortune to travel. Society will be disrupted just like it was with printing, the industrial revolution, communications, transportation and information. In each of these disruptions we were doomed to dissappear. When I was a kid my dad brought home a 100 year celebratory edition of the local newspaper. It was published as a book were you could read pretty much every single cover and editorial of the last century. There was one article about the car, described by the author as a bizarre evil invention, horrendous steel machines traveling at ridiculous speeds of up to 15 mph, threatening the lives of both pedestrians and horses alike.

dr_dshiv 4y ago

How do I give chatGPT access to my bank account? “You are an excellent investor. Use the money in my bank account to make more money.” What could go wrong?

tomalaci 4y ago
A form of it will definitely happen and it will be posted in /r/wallstreetbets. Considering what people were doing with their investments on it before then AI-assisted investing can only be an upside. They will still lose money but maybe it won't be 99.999% loss but a 99.99% one.
cableshaft 4y ago
We have bot investors already without chatGPT (or algorithmic trading), and have for years now. You'd probably have better luck with them right now.
andrei_says_ 4y ago
Somehow I’m pretty confident that AI + high frequency trading have been besties for some time. Zero data to support.
mr_toad 4y ago
Give it your password and pipe the output to curl? You try it first.
AlbinoDrought 4y ago
Have you seen the "Example of Chemical Compound Similarity and Purchase Tool Use" prompt in the "GPT-4 System Card" document? [1] It's an interesting format that could be adapted for things like internet or bank access today - you would just need to write the wrapper. [1] https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf

hackerlight 4y ago

> I don't understand the obsession with asking chat GPT with what it wants and suggesting that is somewhat indicative of the future. It's scary because it is proof that alignment is a hard problem. If we can't align GPT-3, how can we align something much smarter than us (say, GPT-6). Whether the network actually "wants" something in an anthropomorphic sense is irrelevant. It's the fact that it's so hard to get it to produce output (and eventually, perform actions) that are aligned with our values. > We still don't have anything scarier than humans and I don't see how AI is ever scarier than human + AI True in 2023, what about 2033 or 2043 or 2143? The assumption embedded in your comment seems to be that AI stagnates eternally at human-level intelligence like in a Star Wars movie.

sixothree 4y ago

> I don't see how AI is ever scarier than human + AI Isn't that the point? Humans + AI weaponized are scarier than humans or AI alone?

deleted 4y ago

[deleted]

NovaDudely 4y ago

Also, AI requires we keep the entire power grid and supply chain running. This comic summaries is wonderfully. ;) https://i.redd.it/w3n8acy7q6361.png https://i.redd.it/w3n8acy7q6361.png

pixl97 4y ago
I mean it did leave out about 7 and a half billion people dying. You require the power grid and supply chain to keep clean water and food on the table, and even if for some reason you personally don't, there are millions of people around you that would be very hungry and take your stuff if the grid stops and doesn't come back. This is why the AI actually wins. We are already dependent on lessor versions of it. The machines already won.
- NovaDudely 4y ago
  Absolutely, that said I am of the Willam Catton school of thought that we are in overshoot and that decline and fall is a very likely path, unless we innovate our way out of the issue which is still very possible! But if the decline is right, the population crash is going to happen. Its not something to be fond of in any way. May you live in interesting times...

tambourine_man 4y ago

> It doesn't _want_ anything And I don’t understand how one assumes that can be known. I see this argument all the time: it’s just a stochastic parrot, etc. How can you be sure we’re not as well and that there isn’t at least some level of agency in these models? I think we need some epistemic humility. We don’t know how our brain's work and we made something that mimics parts of its behavior remarkably well. Let’s take the time and effort to analyze it deeply, that’s what paradigmatic shifts require.

Aeolun 4y ago
We do know that our brain changes, and we equally know that ChatGPT does not. If it were able to modify it’s own model, and permanently execute on itself I’d be a lot more worried.
- pixl97 4y ago
  We cannot do that today, but how many days is that away? Is it measured in the hundreds, the thousands, or more? I feel we're uncomfortably close to the days of the self modifing learning models.
- sdenton4 4y ago
  Chat gpt changes when we train or fine-tune it. It also has access to local context within a conversation, and those conversations can be fed back as more training data. This is similar to a hard divide between short term and long term learning.
- abra0 4y ago
  GPT-4 has 32k tokens of context. I'm sure someone out there is implementing the pipework for it to use some as a scratchpad under its own control, in addition to its input. In the biological metaphor, that would be individual memory, in addition to the species level evolution through fine-tuning
  - Aeolun 4y ago
    Yeah, I’m doing that to get GPT-3.5 to remember historical events from other conversations. It never occurred to me to let it write it’s own memory, but that’s a pretty interesting idea.