3 ms·
> the problem is it has an unstated limit to the amount it’s able to read Not sure which version Bing uses (I’d guess the smaller), but GPT-4 has either an 8K
by dragonwriter 4y ago
> the problem is it has an unstated limit to the amount it’s able to read
Not sure which version Bing uses (I’d guess the smaller), but GPT-4 has either an 8K token or 32K token context (prompt + response) space, some of which is taken up by the hidden system prompt, but that limit is also there using the model through the API.
- carrolldunham 4y agoThe 'creative tone' mode of Bing was upgraded to 'large context size', presumably 32K. By unstated (I've changed my post now) I meant that it doesn't report back "that's too long", it just says "sure here's a summary", which is not representative of the document. I saw another "pdf reading" service (not OP one) that suffered this too.
- dragonwriter 4y agoYeah, GPT itself will do this; you could probably do a separate token count in the app wrapping it to figure out if it was likely to have occurred (or limit the input to a size sufficiently less than the limit to assure that a summary wouldn’t exceed the limit.)
- carrolldunham 4y agoI mean, if you wanted to add no value to GPT, yes. The value of any 'read a pdf' app should be that it traverses the full document as needed for a query, copying into its internal context what may be relevant, re-iterating until the true answer is found regardless of whether it's scattered through the large document. Bing and the OP app both have as author says a "layer" that should do that but currently - Bing isn't using the layer right - it can search the whole internet to orchestrate context for itself but doesn't seem set-up to so search one doc - OP app seems to crash after a while - the third service i forget the name of which has been shown on here (something to do with 'pdf') similarly to Bing doesn't 'search around' in long documents appropriately
- dsubburam 4y agoThe crash was unrelated to tech and a provider billing issue. Fixed. We do traverse the full document, all 2,000+ pages that we support. Give it a go!