3 ms·
Not internal consistency exactly, but there are criteria checking how well the chapter plan was followed (which is all the way up at the top of the context wind
by sam-paech 1y ago
Not internal consistency exactly, but there are criteria checking how well the chapter plan was followed (which is all the way up at the top of the context window).
This is done per chapter, and the score trendline is what you see in the "degradation" column.
I would say this could be a reasonable proxy for internal consistency since it is measuring more or less the same ability, i.e. how well it's keeping track of details as context window increases.