On consideration, maybe "combining them linguistically" is a bit too literal. Of course that is what a language model does, but that ignores that an extremely g

On consideration, maybe "combining them linguistically" is a bit too literal. Of course that is what a language model does, but that ignores that an extremely good language model such as GPT-3+ will have had to have learned a pretty good world model in order to do a good job of next-word prediction, to the extent that regarding this as merely continuation statistics (while true) is a bit too dismissive of what it's actually capable of. So, perhaps the scale upgrade from GPT-3 to GPT-4 will improve it's world model enough that it does a better job of combining sources using a consistent bias/POV rather then happily combining true and false sources as it currently does.