3 ms·
Recently I've been doing some data modeling stuff that essentially boils down to aligning and identifying common concepts between several JSONSchema each of abo
by xkcd-sucks 3y ago
Recently I've been doing some data modeling stuff that essentially boils down to aligning and identifying common concepts between several JSONSchema each of about 30000-50000 characters without whitespace. For example, "This field `$.run.acquisitionTime` in the first schema means the same thing as `$.operation.experiment_timestamp` in the second schema; but generally for more esoteric concepts than datetime.
Naturally, I cannot remember most of the content and use text editors / home built data review tools to augment my very limited working memory. It is slow.
I would love to chuck them all into ChatGPT4 and have that magically align and cross reference them, which is just not possible with limited context. There are perhaps some clever solutions involving progressive abstraction and stuff, which I do not have the time to figure out.
So TL;DR maintaining a context much larger than human working memory is a value prop of computers in general, so it feels like it should be a value prop of LLMs too