2 ms·
Are you sure? This is for inference, not training, and I’d do it on blocks of code with less than 4096 tokens. That’s what I have to do with gpt 3.5 turbo via
by alchemist1e9 4y ago
Are you sure?
This is for inference, not training, and I’d do it on blocks of code with less than 4096 tokens. That’s what I have to do with gpt 3.5 turbo via api.
I can do what I want with the API and it is around 1-2 cents per batch of code I run through it, typically 400 lines of average code is around 4k tokens.
- aunty_helen 4y agoIf you can get it within 4k of tokens then your vram needs will be much less. As the other two have pointed out, there's workarounds to this. If you're trying to get codebase level introspection you might need to wait a bit for some of these techs to mature. Exciting space and yea, learning heaps myself day by day.
- alchemist1e9 4y agoThe approach I was going to take is generating english code descriptions/summaries in related chunks of under 4k tokens then those can be combined and also summarized. In my opinion the code descriptions gpt 3.5 turbo has been spitting out for me are good quality and concise. I’d argue they are probably better than what many of the developers themselves would write, especially when english isn’t native for the developer.