> You can see this with even just very large prompts, the larger the context the worse the quality, despite model developers claiming ever larger context window

> You can see this with even just very large prompts, the larger the context the worse the quality, despite model developers claiming ever larger context windows. This is precicely the problem I am attempting to address. The point is that I want to be able to frame the way a particular complex iterative query discovery can be accomplished where I utilized already baked frames of how I want to address the problem, similar to how the Baseball workflow is on txtai - but in a more robust manner. https://github.com/neuml/txtai/blob/master/examples/42_Prompt_driven_search_with_LLMs.ipynb https://github.com/neuml/txtai/blob/master/examples/42_Promp... Where I can build particular query bots that can search properly based on the context where I dont have to spend time constructing the context in my prompt, I tell it to APPLY a context to the following prompt. So it limits the domain, formulates the domain to search. Start looking at how baseball players' Dossier stacks up: start building a context for player: https://i.imgur.com/2xvreAF.png https://i.imgur.com/2xvreAF.png Then I can take that idea, and start applying it to say public figures: https://i.imgur.com/pFODyms.png https://i.imgur.com/pFODyms.png Build a caged set of lenses, think of it like an agent - but when I told it to act like a PHD Chemist specializing in materials science as an expert... I then framed which space in that area to focus the response from - which was as an expert in PFTE. so, when I want to have it lookup aspects of say a bill passing, I can ask it "Looking at the congress-person lattice, what in their lattice is related to these various aspects from that bill/act/contract/bailout where you can see that when something occurred, these properties for this object were affected in such manner. Then you just have say trigger that will say "Whoever has this sheet balance out after this thing occurs" and I have refined how that information will be sought out - for example, it will learn where to grab the best pieces of information over time, as it can be asked "which sources had the best information for [property]" Then you say follow what MLOps was doing [0] for better understanding language used to define the same events. https://mlops.systems/posts/2024-06-25-evaluation-finetuning-manual-dataset.html https://mlops.systems/posts/2024-06-25-evaluation-finetuning... We can then see an article, and then look at whatever lattice we have defined to see who may be connected to that thing - and how. By developing the discernment model for that domain - you can have the ai evaluate the spaces of interest in various ways to then craft the lattice file - which then ideally keeps the FN thing focused so the context window can be more complex without superfluous token/memory cruft. So longer iterative can be made on the same subjects with room to context with less hallucination/forgetting. And the outputs of those can be fed to other workflows in txtai or succinctly wrapped in MagicLoop widgets for great justice.