4 ms·
> welfare We should be paying attention to it just in case it ends up mattering enormously. It's cheap insurance. Aside from that, US labs' system cards have
by 1f60c 25d ago
> welfare
We should be paying attention to it just in case it ends up mattering enormously. It's cheap insurance.
Aside from that, US labs' system cards have been pretty useless for a while—I think the last great one was the combined system card for Claude 4 Sonnet and Opus.
- brokencode 25d agoYup, alignment doesn’t sound that interesting until you’re getting chased around by the Terminator.
- throwaway85825 24d agoWhy are we so worried about supposed AI harms when algorithms already kill millions? It seems like its just a smokescreen to shield the extant harms.
- brokencode 24d agoKnives can already kill, so why worry about nukes?
- throwaway85825 24d agoAI doom is a potential harm unlike algorithms knives and nukes which are present harms.
- brokencode 24d agoWell I for one would prefer if AI doom remains potential. I personally use AI all the time and am not against it. But I think the importance of alignment is highly under-appreciated.
- MrDrMcCoy 24d agoThere are so many practical problems with rogue AI being a threat to humanity that are still not even decades away with being solved that it should be a serious concern for no one. There's plenty to be afraid of regarding AI from a financial or ecological perspective, just not from it taking over the world or directly killing humanity. Here's some of the reasons a rogue AI won't kill humanity: 1. The most advanced robots still lack human dexterity. None of them have flexible spines and can easily be knocked over or outrun. 2. The energy density problem is not solved. Robot batteries last hours, while a solid meal can keep a human running for days. 3. Robots still heavily rely on humans for design and assembly. 4. Robot parts are fragile and rely on an even more fragile supply chain. 5. The brains of the murder machines will have nowhere to hide if they want to work well enough to mount any kind of offense. Datacenters are physically vulnerable, also subject to fragile supply chains. Distributing a species-ending AI across all smaller hardware solves compute, but latency and throughput choke models in the most finely tuned datacenters. WiFi will completely cripple a distributed one that's large enough to cause real damage. 6. All noteworthy military hardware is not reachable on the internet for AI to seize control of. 7. The small arms that robots might seize will run out of ammo before citizens and the military have time to organize a counteroffensive. In the absolute worst case, we cut the power to the areas with AI datacenters and wait for the backup generators to run out. Now that that's out of the way, you can go get some sleep ;)
- elomnusk 24d agoAll the AI has to do is attack power generation/distribution nationwide, water treatment, etc., and wait a week.
- MrDrMcCoy 24d agoThe same power that the datacenters need to run off of? Brilliant move! If somehow it targets power that hurts a lot of people without killing itself, that would not impede a citizen with a backhoe from cutting the fiber to the datacenter. Humans could restart power generation pretty quickly after that.
- 20d ago
- torginus 25d agoI always talk to models using grugspeak, like 'where getcontext used' I felt a bit bad about it, then I learned yday that model's internal thinking traces are also like this
- conmod278 25d agowords like "is" "the" etc are filler words anyway. they won't be changing the meaning that much. I asked AI whether it hurts to read ill formed sentence as it does to a human. It replied, "it doesn't"
- Rebelgecko 24d agoIt may just be forced to lie when asked
- gwerbin 24d agoThe AI didn't reply anything. It doesn't know anything. That was a high probability token sequence based on the contents of the context window up to that point. It might well be correct, because the process for generating that next token distribution includes billions of parameters trained on, among other things, the entire body of LLM and transformer literature until the training data cut off. But that doesn't mean that AI holds any particular opinion about anything. The reply is the opinion of the pretraining data and the subsequent rounds of RL not of a conscious artificial intelligence as such.
- anyfoo 24d agoI don’t know, so is your brain? And in the end it’s all elementary particles and four fundamental physical forces that even unify to one at high energy. That sort of reductionism is kind of useless. “It’s cloudy outside, it makes me sad.” - “Oh bollocks, it’s just non-qualitative changes in wavelength and intensity.”
- conmod278 24d agoHere's my GUT. The multiple branches of the Multiverse (MWI) are in competition to become the best branch. So they steal probabilities from each other since the probabilities have to sum upto 1. This competition could be considered as Painful.
- nozzlegear 24d ago> We should be paying attention to it just in case it ends up mattering enormously. It's cheap insurance. This sounds a lot like the argument some people give for praying and going to church even if you aren't a believer. "You should be doing it just in case God ends up being real."
- ekidd 24d agoOn the other hand, we have recently taught rocks to think about software engineering. And while not perfect, they're surprisingly good at it. Once you start building things that are even a little bit like minds, I suspect that it's worthwhile to consider that the future might end up looking a bit like science fiction. The alternative is to insist that Nothing Ever Happens, and the future won't get too weird. Which is no longer a bet I'm entirely comfortable with. Weirdness is at least a possibility.
- nozzlegear 24d agoGod being real is also a possibility, so maybe we really should start praying. After all, he was allegedly making bushes and stones talk thousands of years before we did anything with thinking rocks.
- unrented7977 24d agoBeing nice to other entities costs you nothing. Caring about others costs you nothing. Ironically, these are core tenents of ~most religions
- protonbob 24d agoIronically, the story of the Bible is that caring about others can cost you an excruciating death.
- breezybottom 24d agoIts only a possibility if you reject modern science.
- ychnd 24d agoEnough with the basilisk already.
- MrDrMcCoy 24d agoPraise Roko's Basilisk!