3 ms·
This is why Tabnine got e super excited, until I ran across the issue where they think their results are better than what the IDE gives you, which is incredibly
by thejosh 3y ago
This is why Tabnine got e super excited, until I ran across the issue where they think their results are better than what the IDE gives you, which is incredibly annoying. https://github.com/codota/tabnine-intellij/issues/18 https://github.com/codota/tabnine-intellij/issues/18 . I would be happy to pay, but it seems they are convinced their way is best.
I honestly think that if you could have all your private code indexed and accessible, this would be a game changer as it has way better context.
- spenczar5 3y agoI think so too. I’m thinking about writing a little server you would run as a sidecar process. It would tokenize your code and build a local index in a vector DB. Client plugins in IDEs would ask that server for similar code snippets, with a short deadline for response, perhaps 1ms. Then the plugins could assemble a prompt to send to a configurable LLM backend, and do editor-specific code insertion or suggestion or whatever. I use emacs, so I know how I want that to work for me, but it must be different for others. My open questions though are: - does the tokenizer for the index need to be the same as the one used by the LLM? - is running a local LLM feasible, or should I just assume prompts must sent over the network? This would affect latency and privacy a lot. - is fine tuning helpful? Or necessary, even, to ensure the LLM only provides source code, and only in the target language?