3 ms·
Oh, it just uses OpenAI. https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fab04c0c40f305646028dda/app/api/generate/route.ts#L72 https://github.com/ste
by LoganDark 3y ago
Oh, it just uses OpenAI.
https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fab04c0c40f305646028dda/app/api/generate/route.ts#L72 https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fa...
...and has a 50-request limit per day.
https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fab04c0c40f305646028dda/app/api/generate/route.ts#L22 https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fa...
...and trims the content to, um. 500 characters?!
https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fab04c0c40f305646028dda/app/api/generate/route.ts#L46 https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fa...
So past a certain point you'll get the same completion repeatedly / a completion of the start of your document, cut off.
- awayto 3y agoI think the nature/feel of the quick response is likely due to the fact the completion call is streaming "stream: true" [1]. Then, they are parsing it back to text as it streams the response back. [1] https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fab04c0c40f305646028dda/app/api/generate/route.ts#L67 https://github.com/steven-tey/novel/blob/62d32be43dae8aa40fa...
- LoganDark 3y agoI'd have been more surprised if there wasn't streaming. All the LLMs I have ever used, stream tokens in real-time. I'm talking about the time spent per token.
- steventey 3y agoOh yeah good catch – just bumped up the character limit to 5,000 (forgot I just upgraded to gpt-3.5-turbo-16k today) :)