5 ms·
OpenAI training its next major AI model, forms new safety committee
- righthand 2y agoOoookay, sure. Everyone believes they’re serious right? No ideas about it, just safe AGI. I bet this is Sam appeasing staff.
- ghaff 2y agoThere are no doubt lots of naive idealistic people out there. But who in their right mind puts themselves anywhere near "safe" or "ethical" AI initiatives at any of these companies.
- deleted 2y ago[deleted]
- hereme888 2y ago+1 for the news on training the next-gen model. GPT-4's limits have become quite clear to me, and I'd appreciate something that pushes the frontier of what I can automate in my life.
- deleted 2y ago[deleted]
- astrea 2y ago“We’ve investigated ourselves and found no wrongdoing.”
- 5cott0 2y agowhat happened to all the stories about how apple was partnering with openai for iOS 18? the most innovative thing about openai is the PR
- azinman2 2y ago> the most innovative thing about openai is the PR You’ve got to be joking
- 5cott0 2y agook fine the most innovative thing about openai was blowing up the board to abolish the nonprofit
- azinman2 2y agoThey literally pioneered LLMs, RLHF, multimodal LLMs, and many other fundamental techniques that are causing an epic sea change in society and technology. Don’t be so jaded.
- 5cott0 2y ago[flagged]
- orhmeh09 2y ago"Epic sea change?" In which society?
- anshumankmr 2y agogrokking is something the open ai team came up with right
- genewitch 2y agoHeinlein in the book Stranger in a Strange Land. and iirc (it's been decades) it was used in the context of a "cult" - the people in the cult "grokked" and the people who weren't didn't. I comment something like this when i see the word grok, which has happened more this year than in the last decade - and i am not sure why, probably because i don't pay much attention to OAI news?
- blitzar 2y ago
- rvnx 2y agoSam Altman, Mark Zuckerberg, Elon Musk, such a great selection of people in charge of AI safety. Nice guys who put human first. Glad to see them caring about our safety. Our heroes.
- jimsimmons 2y agoIt's interesting how any thread critical of OAI or Sam gets shot down from the front page very quickly but anything positive lingers on. When Google Gemini threads are created on launch days they get merged very slowly and therefore community's upvotes get scattered. Ofc it's hard to prove any of this. But it almost feels like there's an algorithm with a bias at play
- tivert 2y ago> Ofc it's hard to prove any of this. But it almost feels like there's an algorithm with a bias at play It doesn't pass the smell test that there would be an algorithm involved in actions like that. The development timeline and effort involved makes absolutely no sense vs. alternative explanations like manual action. Sometimes even things you'd expect to be algorithmic are not and are really AI (Actually, Indians). I think the typical explanation for stuff like that on HN is user flagging, which does pass the smell test as an explanation for what you describe. Altman and OpenAI still seem to have a fanclub on HN (real and/or astroturfed), though it's perhaps less dominant than half a year ago.
- edgyquant 2y agoSam comes from YC. I know he was fired but Paul always has good things to say about him. It was also sketchy how for the first year there was continuously multiple threads on the front page about OAI.
- browningstreet 2y agoThey still don’t want to talk about how they resolved, or did not, the Scarlett situation. Silence.
- jdross 2y agoBecause - the situation is like a week old, legal cases and settlements can take months+ - they already put forward their side of it in both a blog post and a WaPo leak Do you think this is like an episode of a TV show?
- throwup238 2y ago> Do you think this is like an episode of a TV show? Increasingly, yes. I’m starting to think that @sama is really Samaritan: https://personofinterest.fandom.com/wiki/Samaritan https://personofinterest.fandom.com/wiki/Samaritan
- browningstreet 2y agoFaking a deepfake, leaving lots of questions unanswered, and then announcing a new safety structure that goes against the very grain of last week's legal conundrum, without mentioning it.. it's all pretty shady and/or weak.
- ppppo 2y agoSerious question, what is there to ai safety? So far I think it's just pretty helpful when coding certain things, but most of what it gives is low quality. What are the real risks involved? Maybe it's just me but so far I don't see risks, maybe once there's GPT 42o.
- tempodox 2y ago> what is there to ai safety? Bullshit jobs where you get payed for pretending. It is supposed to help “Open”“AI” build a moat based on “safety”.
- ppppo 2y agoI do see people already creating this new niche, as we have airport security and unnecessary bureaucracy (Germany as an example) we will probably have something like "AI supervisors" who basically chat with an llm for a few minutes a day for the state or local municipality and make a career out of that.
- deleted 2y ago[deleted]
- moritzwarhier 2y agoWhen GPT-4 was released, they published examples of prompts and agent-like scenarios that are considered malicious. They are even categorized. The appendix also contains full responses before they implemented changes and further training in reaction to the red-teaming results, at least for most of the prompts. (starts on p. 44) https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf I think the categories of prompts are still relevant, and some of the used of LLMs described are already widely deployed (e.g. the one about propaganda on social media) People tend to laugh about the increasingly strict guardrails of commercial chatbots, and it has proved to be at least in large parts an cat and mouse game. But I wouldn't underestimate the viciousness of an unfiltered LLM instructed to generate propaganda, hate speech or convincing instructions for humans to perform harmful acts.
- elzbardico 2y ago
- schnitzelstoat 2y agoI don't think LLM's are capable of AGI regardless of how much powerful they become. AI Safety has become a rewarding grift though.
- d--b 2y agoSo this commitee is three businessmen and one lawyer, right? They don't even try to pretend anymore.
- Finbarr 2y ago> OpenAI says that multiple technical and policy experts, including Aleksander Madry (head of preparedness), Lilian Weng (head of safety systems), John Schulman (head of alignment science), Matt Knight (head of security), and Jakub Pachocki (chief scientist), will also serve on its new committee.
- throwup238 2y ago> alignment science OpenAI’s version of homeopathy.
- deleted 2y ago[deleted]
- agentultra 2y agoTraining another model? Are we still in the, "suck up available water in the middle of a drought,"[0] era of training? I think we need better regulation of these companies to prevent them from doing actual damage rather than trusting them to self-regulate against hypothetical ones. The latter seems more like market-capture smoke-and-mirrors. The former is having real consequences on the environment, workers, and the economy. [0] https://futurism.com/critics-microsoft-water-train-ai-drought https://futurism.com/critics-microsoft-water-train-ai-drough...
- Hayvok 2y ago> I think we need better regulation of these companies to prevent them from doing actual damage rather than trusting them to self-regulate against hypothetical ones. What is the actual damage you are concerned about and determined to regulate? Simply saying "environment, workers, and the economy" is so broad I can't imagine what an effective regulation would look like. How would you even word the regulation?
- agentultra 2y agoFirst, I can't possibly have all of the answers. However the evidence that these companies are doing real damage now is all around us. I already gave the example of Microsoft using billions of litres of fresh water to cool data centres during a drought. In the case of labour the SAG-AFTRA strike, a contributing factor was the use of AI in the industry. There are some estimates that the carbon emissions of these training efforts dwarf the airline industry and will grow to consume, like crypto, more energy than small countries soon [0]. Not sure that we need to be protected against hypothetical, super-intelligent, self-aware AGI systems that are, if even possible, decades away when people are using what we have today to lay off labourers by training models on their work and replacing them. [0] https://www.ll.mit.edu/news/ai-models-are-devouring-energy-tools-reduce-consumption-are-here-if-data-centers-will-adopt https://www.ll.mit.edu/news/ai-models-are-devouring-energy-t... Update: removed unnecessary bit
- Hayvok 2y ago
- elzbardico 2y agoA lot of this is just to push the laughable idea that LLMs will somehow, magically lead us to AGI, so the flow of investors money continue unabated. And the rest, the good and only real part of this is just plain old moderation, because we really don't want someone to use those LLMs to write Mein Kampf II.
- z7 2y agoAltman has said that he doesn't think LLMs alone are sufficient to achieve AGI: https://old.reddit.com/r/singularity/comments/17wknc5/altman_there_are_more_breakthroughs_required_in/ https://old.reddit.com/r/singularity/comments/17wknc5/altman...
- Taikonerd 2y agoI wish Sam Altman himself weren't on the board. To make an analogy: when I worked in application security, my company had a CTO, and a separate CISO (Chief Information Security Officer). One of the senior engineers told me that the CTO's job is to say "faster, faster!" And the CISO's job is to say "slow down, be more careful." You can't have the same person doing both jobs. Altman is clearly on the "faster, faster!" side. And that's OK, but he shouldn't also be on the "slow down, be more careful" committee.
- infecto 2y agomight be an unpopular opinion, but I find the new safety committee reasonable. While "AI Safety" has often seemed like a grift to me, I do see value in addressing immediate concerns like bias evaluation and model alignment. Issues like racial bias in data are real and harmful, and it's crucial to work on improving these outcomes. However, the broader field of AI Safety often feels like fear-mongering over non-issues. If AGI is on the horizon, I’m not particularly worried about OpenAI's "superalignment" team. If one group achieves superalignment, it's likely that other global actors will soon have similar capabilities, so I don't see it as a major concern.