3 ms·
There is also a context window problem where you can only pass in 4k tokens (including the prompt). This is not that large of a window for providing history and
by dbish 4y ago
There is also a context window problem where you can only pass in 4k tokens (including the prompt). This is not that large of a window for providing history and information of the entire workflow. How were you trying to solve that? Also, if you’re building something yourself, you can use davinci-003 directly through their APIs instead of trying to build on top of chatgpt
- jrvarela56 4y agoYou are right, what I described was just a prototype. ChatGPT was just a gateway drug for me to dive into the OpenAI docs and look for other providers and existing solutions. I've been reading up on approaches to 'package' context (example: https://thakkarparth007.github.io/copilot-explorer/posts/copilot-internals.html#how-is-the-prompt-prepared-a-code-walkthrough https://thakkarparth007.github.io/copilot-explorer/posts/cop...). My take for now is that this kind of packaging is very context specific to the point where it's going to make sense for engineering teams to craft their own version of 'gather context before the prompt is created'. I'm testing out approaches using toy examples for now (how would you create a chatbot about a book?; podcasts, etc). I don't have an answer but I'm highly confident we're going to replace a lot of junior-level work in the next year or two.