2 ms·
I believe this is utter fantasy. That kind of data is usually super messy. LLMs are terrible at disambiguating if something is useful vs harmful information. I
by ZephyrBlu 1y ago
I believe this is utter fantasy. That kind of data is usually super messy. LLMs are terrible at disambiguating if something is useful vs harmful information.
It's also unlikely that context windows will become unbounded to the point where all that data can fit in context, and even if it can it's another question entirely whether the model can actually utilize all the information.
Many, many unknown unknowns would need to be overcome for this to even be in the realm of possibility. Right now it's difficult enough to get simple agents with relatively small context to be reliable and perform well, let alone something like what you're suggesting.