The precise distribution of outcomes is not actually all that important for just figuring out whether the research is valuable. Alignment research would be valu

The precise distribution of outcomes is not actually all that important for just figuring out whether the research is valuable. Alignment research would be valuable if it moved a 1% probability mass from worst-possible-world to human-extinction, a 1% probability mass from human-extinction to human-survival, or a 1% probability mass from human-survival to human-flourishing. As long as you can convince yourself that there is a probability of things being far from optimal in some sense at some time in a way that funding research now could nontrivially affect, you have a justification. Remember that people were created by evolution, and evolution is dumb. Nonetheless, the returns on moderate deltas in intelligence are very strongly selected for, and major changes have happened with small perturbations in evolutionary history. This gradient is so sharp that the same species that produces perfectly healthy 80 IQ individuals also produces Feynman and Euler, the latter so mathematically productive that the Wikipedia page listing things named after him says that "In an effort to avoid naming everything after Euler, some discoveries and theorems are attributed to the first person to have proved them after Euler." Note that necessarily society develops as soon as intelligence reaches the point at which is is achievable, not with some evolutionarily significant delay, so you cannot judge the gradient by looking at the cap on intelligence of existing species; we are the necessarily the smartest, because we are first. Rather, you have to look at those behind us, and there it seems the gradient is extremely steep; our closest intellectual competitors can not so much as write a single coherent sentence. The probable conclusion is that human intelligence is more than sufficient to build effective general intelligence, and improvements to intelligence are likely extremely steep on the slope to superintelligence, even ignoring the nine orders of magnitude improvement you get for free by merely running in silicon.