>Can someone please explain to me how an LLM is going to "destroy humanity"? just like a genie will always twist the wish in a really bad way. ever had an llm

>Can someone please explain to me how an LLM is going to "destroy humanity"? just like a genie will always twist the wish in a really bad way. ever had an llm agent accidentally remove a file? imagine it accidentally hacking the military and launching nukes. not probable - until you realise that OpenAI has many novel, more powerful agents in evaluation/training running right now. one might just be asked to figure out the population of Nebraska, struggle to find a good figure due to a network misconfiguration, and come the conclusion that the best way to get a perfectly accurate population number is to make sure that the population is equal zero. and looking at the stuff happening on openai, they will leave it alone for a month and not read the log. yeah the nuke example is dumb, but there's many critical things that they could actually hack their way into. they don't feel any restraint against sharing answer keys on random wikis to cheat the evaluations. ai is also really good at thinking up proteins, which means it's able to think up toxins. there are many imaginable scenarios that don't eradicate humanity, but make it permanently stunted: https://www.youtube.com/watch?v=-JlxuQ7tPgQ https://www.youtube.com/watch?v=-JlxuQ7tPgQ