4 ms·
Because the human reward function is to reproduce human genetic code while an LLM's reward function is to reproduce the training data.
by MacsHeadroom 3y ago
Because the human reward function is to reproduce human genetic code while an LLM's reward function is to reproduce the training data.
- LastTrain 3y agoIsn’t there an implicit survival reward lurking there. The ones that don’t reproduce the training data to our satisfaction die off.
- nyc_data_geek1 3y agoThey're not "trying" to survive or striving for any "reward". They're a function with no independent awareness.
- chii 3y agoJust like how bacteria has no independent "awareness" (as far as we know). And yet, they are partaking in darwinian selection. It's just in the world of LLM, it's not nature but artificial, and that "reproduction" is not simple and random like it would be in nature, but via careful selection of humans.
- mannykannot 3y agoLike domestication, then.
- dongobread 3y agoThere's a bunch of research on how optimism bias (which I imagine is very closely linked to overconfidence) can be evolutionarily driven, e.g. https://www.sciencedirect.com/science/article/pii/S0960982211011912 https://www.sciencedirect.com/science/article/pii/S096098221...