Maybe 'new information' can be generated by taking existing data and finding new relationships between that data. If that were the case, the original embedding

Maybe 'new information' can be generated by taking existing data and finding new relationships between that data. If that were the case, the original embedding for that data would change (new relationships = new associations). I agree that infinite knowledge would be difficult to generate this way, but maybe some new knowledge could be. And this process of finding new relationships between existing data might be something we can't automate yet.