I work in the space (doing tech stuff that isn't direct research). The best argument I've seen for alignment by default is something like "morality comes from

I work in the space (doing tech stuff that isn't direct research). The best argument I've seen for alignment by default is something like "morality comes from training data, and therefore the fact that LLM training data sets contain human moral intuitions will mean that an ASI stemming from such a training regime will share enough human values" (Quintin Pope believes something like this, as far as I can tell), which is deeply unconvincing, since it contradicts the evidence we _do_ have from human & animal value formation. Happy to entertain other arguments that alignment-by-default is reasonable; most arguments I've seen are much worse than that one. I haven't seen many people make an active case for alignment-by-default, so much as leave open a whole bunch of swath of uncertainty for unknown unknowns.