3 ms·
No, stop. Full stop. Categorial error. Title SHOULD be DATASETS (whether enumerable or supplied by proxy via model such as PPO) are increasingly being created
by upghost 2y ago
No, stop. Full stop. Categorial error.
Title SHOULD be DATASETS (whether enumerable or supplied by proxy via model such as PPO) are increasingly being created that make humans think "AIs are faking alignment".
PSA: An ideal transformer can only perfectly generalize over the dataset it was trained on. Once you have an ideal transformer, you realize it makes no sense to focus on the transformer and should focus on the dataset instead.
Information is conserved. Models do not create information.
An ideal transformer trained on random noise will create random noise. An ideal transformer trained on "misinformation" will produce "misinformation".
Everything less than an ideal transformer simply fails to perfectly generalize over the dataset, but even ideal transformers do not "create" information.
It is harmful for many reasons to anthropomorphize LLMs like this, the least of which is that it obfuscates what we SHOULD be focusing on, which is building better datasets so we can make useful tools. Instead we are focusing on absurd questions like the morality of toasters and spreading this brain rot contagion to our fellow engineers and the public by continuing to validate it.
I don't care how much marketing money OpenAI and the other FAANGs dump into pretending that this thing is going to go Skynet. Unless we have a SKYNET DATASET, it's NOT GOING TO HAPPEN. INFORMATION IS CONSERVED.
Edit: Dear downvoters, please do explain how this threatens your world view. I am actively trying to help build better LLMs, so if you like LLMs, we should be on the same team. If you don't like LLMs, we should also be on the same team. As far as I am concerned, if you are downvoting this, you have either been deceived or are potentially actively participating in deception.
If I am wrong, please tell me why in the comments.
- echelon 2y agoThis. An LLM isn't going to build you a recipe to clone transgenic viruses or help you synthesize novel chemical weapons unless you're training it to do that. I'm unaware of there being training data for these cases. The models won't learn the ten year postgraduate career that connects the dots.
- upghost 2y agoWow, I'm sorry you had to take a downvote for supporting this. Wild times we live in. Would the down-voter mind explaining their reasoning?
- benatkin 2y agoThat’s trivializing deep learning, calling it merely a dataset. It’s something that was grown, not built. It’s patently false to say that models don’t create information. What about AlphaGo? Why is it hard to tell exactly why an LLM arrives at a particular response? Again, grown, not built. This is what Connor Leahy emphasizes. It’s worth checking out some of his content.
- upghost 2y agoNo, this is false. Reinforcement learning also uses a dataset, it's just not an enumerable dataset. The dataset it is trained on comes from the Bellman Backup equation (or variants, including DL variants) being run against a simulation (which is an unenumerable dataset). The reward model can only make a prediction against the state and observe the actual reward output and backpropagate the error differential. This is the way all DL works, as function and dataset approximators. It is not trivializing the field.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- IAmGraydon 2y agoThe fact that you’re 100% correct yet being downvoted tells us all we need to know - people really want to believe that LLMs are actually intelligent on some level. Why? Because it’s humiliating to admit that they’ve been tricked.
- upghost 2y agoThanks for that. I really don't like to rock the boat or make comments like this. But the thing is that saying LLMs are "capital I" Intelligent moves them out of the realm of something we can reason about into the realm of prayer and beseechment. It can make people feel like "this is not something I can participate in or reason about", and thus must trust a handful of "wizards" in a handful of companies. Conferring agency on our tools does nothing if it robs agency from us.