4 ms·

Is machine learning model "alignment" a serious academic concept? I've only seen this mentioned in pseudo-philosophical Twitter posts before.

BasedAnon 3y ago
yes
mecsred 3y ago
Depends on your definition I guess. Some academics make careers out of pseudo-philosophical twitter posts.
golol 3y ago
It is. I'd argue that alignment is the difference between GPT-3, which was a neat toy for nerds, and ChatGPT.
ftxbro 3y ago
its meaning has shifted to now mean ESG (https://en.wikipedia.org/wiki/Environmental,_social,_and_corporate_governance https://en.wikipedia.org/wiki/Environmental,_social,_and_cor...) and DEI (https://en.wikipedia.org/wiki/Diversity,_equity,_and_inclusion https://en.wikipedia.org/wiki/Diversity,_equity,_and_inclusi...) for corporations that make algorithms that have learned natural language well enough that people are talking with them and taking them seriously. the AI alignment guys don't like that and they have started calling their old AI alignment meaning as 'not-kill-everyoneism' with mixed results (https://www.lesswrong.com/posts/jMzBhCRrr7otmqcvK/notkilleveryoneism-sounds-dumb https://www.lesswrong.com/posts/jMzBhCRrr7otmqcvK/notkilleve...). EDIT: btw i don't have anything against ESG or DEI. I'm not a culture warrior complaining about 'woke ai'. I'm also not a lesswrong guy. I'm answering the OP's question "Is machine learning model "alignment" a serious academic concept? I've only seen this mentioned in pseudo-philosophical Twitter posts before." by saying what they mean now by 'AI alignment'.
- sebzim4500 3y ago
  I don't think anyone uses alignement to refer specifically to DEI bullshit. There are people using the term generally to refer to any kind of finetuning of an LLM, so they would consider what OpenAssistant did to be alignment even though there was no attempt to convince it to not kill humanity or to be politically correct.
Q6T46nT668w6i3m 3y ago
Yes. It’s very important. Think of it as quantifying whether a model understood the objective. This is, as you’d expect, especially important for auto-regressive models.
mkoubaa 3y ago
Academic concepts are only considered serious by convention
sebzim4500 3y ago
Yeah, but like every other term in AI, no two people agree on what the word means so discussion around it is challenging.

BasedAnon 3y ago

yes

mecsred 3y ago

Depends on your definition I guess. Some academics make careers out of pseudo-philosophical twitter posts.

golol 3y ago

It is. I'd argue that alignment is the difference between GPT-3, which was a neat toy for nerds, and ChatGPT.

ftxbro 3y ago

its meaning has shifted to now mean ESG (https://en.wikipedia.org/wiki/Environmental,_social,_and_corporate_governance https://en.wikipedia.org/wiki/Environmental,_social,_and_cor...) and DEI (https://en.wikipedia.org/wiki/Diversity,_equity,_and_inclusion https://en.wikipedia.org/wiki/Diversity,_equity,_and_inclusi...) for corporations that make algorithms that have learned natural language well enough that people are talking with them and taking them seriously. the AI alignment guys don't like that and they have started calling their old AI alignment meaning as 'not-kill-everyoneism' with mixed results (https://www.lesswrong.com/posts/jMzBhCRrr7otmqcvK/notkilleveryoneism-sounds-dumb https://www.lesswrong.com/posts/jMzBhCRrr7otmqcvK/notkilleve...). EDIT: btw i don't have anything against ESG or DEI. I'm not a culture warrior complaining about 'woke ai'. I'm also not a lesswrong guy. I'm answering the OP's question "Is machine learning model "alignment" a serious academic concept? I've only seen this mentioned in pseudo-philosophical Twitter posts before." by saying what they mean now by 'AI alignment'.

sebzim4500 3y ago
I don't think anyone uses alignement to refer specifically to DEI bullshit. There are people using the term generally to refer to any kind of finetuning of an LLM, so they would consider what OpenAssistant did to be alignment even though there was no attempt to convince it to not kill humanity or to be politically correct.

Q6T46nT668w6i3m 3y ago

Yes. It’s very important. Think of it as quantifying whether a model understood the objective. This is, as you’d expect, especially important for auto-regressive models.

mkoubaa 3y ago

Academic concepts are only considered serious by convention

sebzim4500 3y ago

Yeah, but like every other term in AI, no two people agree on what the word means so discussion around it is challenging.