3 ms·
The problem is, can alignment occur before the relevant capabilities have been developed? LLMs, for example, (although very good at impersonating and talking to
by versteegen 4y ago
The problem is, can alignment occur before the relevant capabilities have been developed? LLMs, for example, (although very good at impersonating and talking to humans and having good world models) are particularly poor at structured reasoning and planning which are the capabilities that will actually be so dangerous. I don't believe superintelligence will be a LLM with chain-of-thought reasoning. If it's a different architecture then once again a lot of alignment work won't be relevant.
- jimrandomh 4y agoYes, many angles on the alignment problem can be studied now, and have started making good progress recently. Some things will turn out in retrospect to not have been relevant, due to architectural shifts, but not everything. Some things are specific to LLMs; some things are specific to transformers but not to language-model transformers; some things are conceptual and likely to still apply to quite-different systems; and some things are just field-building and not specific to any architecture at all. Eg in mechanistic interpretability, there are a lot of findings on LLMs that turn out to generalize across a wider set of NN architectures. Eg https://transformer-circuits.pub/2022/solu/index.html https://transformer-circuits.pub/2022/solu/index.html is something that couldn't be done without access to LLMs, but which looks likely to generalize into future architectures.