Here's a thought experiment. Say you had more than one RNN trained for the same task (say, generating text). How would you couple them? You could just, charac

Here's a thought experiment. Say you had more than one RNN trained for the same task (say, generating text). How would you couple them? You could just, character by character, have each RNN generate its output on its own, and then take a majority vote or do some weighted average to combine the output, but otherwise let each RNN operate independently. But wouldn't it be strange for those RNNs whose output was not ultimately selected to continue processing along the same lines, uncorrected by what was actually selected? Another possibility is to include a "ultimately selected" input to each RNN so they can be privy to the group's decisions. Yes, your internal state said that "e" should be selected, but the group selected "r"- what will you do now? Yet another possibility would be to perform "retroactive justification". Let us say that one RNN (with an internal state S) selected "e", but the group selected "r". Under retroactive justification, the state used for the next character is a state (maybe the one closest to S) that would have instead produced "r" as output. Split brains are coupled by their shared access to the body and act within the same environment. How do the hemispheres use that coupling?