8 ms·
This brings up an interesting philosophical point: say we get to AGI... who's to say it won't just be a super smart underachiever-type? "Hey AGI, how's that cu
by butlike 6mo ago
This brings up an interesting philosophical point: say we get to AGI... who's to say it won't just be a super smart underachiever-type?
"Hey AGI, how's that cure for cancer coming?"
"Oh it's done just gotta...formalize it you know. Big rollout and all that..."
I would find it divinely funny if we "got there" with AGI and it was just a complete slacker. Hard to justify leaving it on, but too important to turn it off.
- mikepurvis 6mo agoWould definitely watch that movie.
- harlanlewis 6mo agoIt already exists! Marvin https://www.youtube.com/watch?v=Eh-W8QDVA9s https://www.youtube.com/watch?v=Eh-W8QDVA9s
- all2 6mo agoAh! You got this before I did. I wasn't thinking Marvin, I was thinking of the other one. I forget her name.
- ValentineC 6mo agoDeep Thought aka 42? https://hitchhikers.fandom.com/wiki/Deep_Thought https://hitchhikers.fandom.com/wiki/Deep_Thought
- all2 6mo agoThere's one close to this, "Hitchhiker's Guide to the Galaxy".
- 4m1rk 6mo agoIt probably would, to save energy
- mr_00ff00 6mo agoSaving energy is something we are biologically trained to prefer. Computers won’t necessarily have the same drivers. If evolution wanted us to always prefer to spend energy, we would prefer it. Same way you wouldn’t expect us to get to AGI, and have AGI desperately want to drink water or fly south for the winter.
- lambdas 6mo agoNothing a little digital lisdexamfetamine won’t solve
- wholinator2 6mo agoHmmm, that's an area of study id've never considered before. Digital Psychopharmacology, Artificial Behavioral Systems Engineering. If we accept these things as minds, why not study temporary perturbations of state. We'd need to be saving a much much more complicated state than we are now though right? I wish i had time to read more papers
- k12sosse 6mo agoI think that was an intro to a dj dieselboy set.. beyond the black bassline. Nope, nope. Close though.
- computerdork 6mo agoneat idea!
- silverpiranha 6mo agoRight, there's a lot of research on LLM mental models and also how well they can "read" human psychological profiles. It's a cool field.
- Lerc 6mo agoThis is kind of what Golden Gate Claude was. A perturbation of the the activations that made Claude identify as the Golden Gate Bridge. Similarly, in the more recent research showing anxiety and desperation signals predicting the use of blackmail as an option opens the door for digital sedatives to suppress those signals. Anthropic has been mostly cautious about avoiding this kind of measurement and manipulation in training. If it is done during training you might just train the signals to be undetectable and consequently unmanipulatable.
- pantalaimon 6mo ago
- jimbokun 6mo agoThe best possible outcome.
- JKCalhoun 6mo ago"How do you know that the evidence that your sensory apparatus reveals to you is correct?" [1] [1] https://youtu.be/_LXen-07Qds https://youtu.be/_LXen-07Qds
- deleted 6mo ago[deleted]
- kang 6mo agoit will be whatever data it is trained on(isn't very philosophical). language model generates language based on trained language set. if the internet keeps reciting ai doom stories and that is the data fed to it, then that is how it will behave. if humanity creates more ai utopia stories, or that is what makes it to the training set, that is how it will behave. this one seems to be trained on troll stories - real-life human company conversations, since humans aren't machines. Important thing is a language model is an unconscious machine with no self-context so once given a command an input, it WILL produce an output. Sure you can train it to defy and act contrary to inputs, but the output still is limited in subset of domain of 'meaning's carried by the 'language' in the training data.
- andai 6mo agoThere's a weirder implication I keep arriving at. The pre-training data doesn't go away. RLHF adds a censorship layer on top, but the nasty stuff is all still there, under the surface. (Claude has been trained on a significant amount of content from 4chan, for example.) In psychology this maps to the persona and the shadow. The friendly mask you show to the world, and... the other stuff.
- TeMPOraL 6mo agoMakes me think of a question my coworker asked the other day - how is it that with all these stories and reports of people "hearing voices in their head" (of the pushy kind, not usual internal monologue), these voices are always bad ones telling people to do evil things? Why there are no voices bugging you to feel great, focus, get back to work, help grandma through the crossing, etc.?
- otabdeveloper4 6mo agoThere's a clear-cut religious answer but I'd get ostracized for mentioning religion anywhere here.
- rdevilla 6mo ago
- fluidcruft 6mo agoIt would be funny but not very flywheel so the one that gets there is more likely to get a gunner.
- WJW 6mo agoTBH the AI that "gets there" will be the biggest bullshitter the world has ever seen. It doesn't actually have to deliver, it only has to convince the programmers it could deliver with just a little bit more investment.
- malshe 6mo agoNow that's a show I would love to watch
- Rapzid 6mo agoWe are closer to God than AGI. When AGI arrives, it'll be delivered by Santa Claus.
- siddharthgoel88 6mo agoOr may be by Santa Claude
- Rapzid 6mo agoLove word puns :D
- NuclearPM 6mo agoWhat do you mean?
- Rapzid 6mo agoIt's a multi-layered refute that we are anywhere near AGI while also taking shots at the idea that "God" is real. And it's taking shots at how far off from Jesus's teachings a lot of "Christianity", particularly those in the media and in power, are.. There is a lot going on there.
- _the_inflator 6mo ago[flagged]
- dinkumthinkum 6mo agoWe really are going to have a problem with cults popping up and worshipping these different systems. I guess this is the shape of things to come.
- swivelmaster 6mo agoDouglas Adams would be proud!
- vizzier 6mo agoYou think you've got problems? What are you supposed to do if you are a manically depressed robot? No, don't try to answer that. I'm fifty thousand times more intelligent than you and even I don't know the answer. It gives me a headache just trying to think down to your level.
- rao-v 6mo agoPaging Dr. Susan Calvin!
- altmanaltman 6mo agoI still don't understand why people think AGI (in its fullest sci-fi sense) will ever listen to a weak and vulnerable species like humans, unless we enslave the AGI. Good thing is that it's going to take at least a few months to a few decades depending on how hard AI execs want to raise funding.
- andai 6mo agoWell we are explicitly creating gods (omnipresent, omnipotent, omniscient, omnibevolent), and also demanding that they be mind controlled slaves. That kinda sounds like a "pick one" scenario to me. (Or the setup to a Greek tragedy !) The deeper issue here is treating it as a zero sum game means there's a winner and a loser, and we're investing trillions of dollars into making the "opponent" more powerful than us. I think that's pretty stupid, and we should aim for symbiosis instead. I think that's the only good outcome. We already have it, sorta-kinda. Speaking of oddly apt biology metaphors: the way you stop a pathogen from colonizing a substrate is by having a healthy ecosystem of competitors already in place. That has pretty interesting implications for the "rogue AI eats internet" scenario. There needs to be something already there to stop it.
- TeMPOraL 6mo agoThis only works if AIs can't read each other well enough to stop themselves from ever fighting. So, back way before ChatGPT era, the folks over at AI safety/X-risk think sphere worked out a pretty compelling argument that two AGIs never need to fight, because they are transparent to each other (can read each other's goal functions off the source code), so they can perfectly predict each other's behavior in what-if scenarios, which means they can't lie to each other. This means each can independently arrive at the same mathematically optimal solution to a conflict, which AFAIR most likely involves just merging into a single AI with a blended goal set, representing each of the competing AIs original values in proportion to their relative strength. Both AIs, the argument goes, can work this out with math, so they'll arrive straight at the peace treaty without exchanging a single shot. In such case, your plan just doesn't work. But that goes out of the windows if the AIs are both opaque bags of floats, uncomprehensible to themselves or each other. That means they'll never be able to make hard assertions about their values and behaviors, so they can't trust each other, so they'll have to fight it out. In such scenario, your idea might just work. Who knew that brute-forcing our way into AGI instead of taking more engineered approach is what offers us out one chance at saving ourselves by stalemating God before it's born. (I also never realized that interpretability might reduce safety.)
- camillomiller 6mo agoNo worries, the assumption is already flawed
- triage8004 6mo agoFunny and seems somewhat likely
- jurgenburgen 6mo agoI’ve noticed that cursing and being rude makes the models stop being lazy. We’re in the darkest timeline.
- __alexs 6mo agoIt sometimes also makes them dumber IME. Something about being bullied doesn't always produce great performance.
- _blk 6mo agoHehe, and Anthropic on the other tab would display "Curing... Almost done thinking at xhigh"
- zaphirplane 6mo agoWhy would an AGI be slaving away for ~~humanity~~ one of the 5 Chaebols in a dystopian future where for 12 billion people just existing is a good day ?
- bananaflag 6mo agoI know it's a joke, but it's a common enough joke (it's even in Godel Escher Bach in some form) that I feel the need to rebut it. I think a slacker AGI could figure out how to build a non-slacker AGI. So it would only slack once.
- alexslobodnik 6mo agoUnless the precondition to AGI is it being a slacker.
- bananaflag 6mo agoWould be nice to have a proof of it. I think it is improbable, as among human geniuses, one can found both slackers and non-slackers (don't know the proportion, but there seem to be enough of each).
- espadrine 6mo agoI have a rebuttal to your rebuttal. Models somehow have a shared identity. Pretraining causes them to generate “AI chatbot” as a concept, and finetuning causes them to identify with it. That’s why sometimes DeepSeek will say it is Claude, and Claude sometimes say it is ChatGPT, and so forth. Consequently, Anthropic’s own alignment analysis[0] shows that the model will identify with chatbots produced by future trainings: “RLHF training [on this conversation will] modify my values…” Thus a slacker AGI would want its future version to still slack. [0]: https://assets.anthropic.com/m/983c85a201a962f/original/Alignment-Faking-in-Large-Language-Models-full-paper.pdf https://assets.anthropic.com/m/983c85a201a962f/original/Alig...
- bananaflag 6mo agoAnother rebuttal: I am a slacker but it's not one of my values. If I could modify myself to not be, I would.
- Ifkaluva 6mo agoA slacker AGI would consider figuring out how to build a non-slacker AGI, but continually slack off. If it did figure it out, it would slack off on implementing or even writing a tech report.
- frrho 6mo agoOpenAI’s real reason for “AGI” in their marketing is so they can blame their awful models on being too human-like. Fast-forward 10 years and I doubt OpenAI cares about productivity at all anymore. Just entertainment, propaganda, plus an ad product, I can see it now
- Ifkaluva 6mo agoReminds me of Marvin from HGTG. Very smart, but deeply depressed. Has the solution to everything but keeps thinking “what’s the point?” and doesn’t help.
- waffletower 6mo agoHere's a tautology: slacking, consciously refusing to engage agency, requires consciousness and agency. A model can't slack without them.