I had to quit halfway due to time constraints but I'll comment on this: >A more explicit strategy: after producing a new document, take every pair of documents

I had to quit halfway due to time constraints but I'll comment on this: >A more explicit strategy: after producing a new document, take every pair of documents in context and ask the model if they should probably be connected. Ask what future queries could be helped by having those documents connected. Are those queries likely? If so, make the connection. Congratulations! You have invented attention!