4 ms·
Im hacking some really dumb code this week & just tried to get StarCoder.cpp to give me some help, but I don't have any idea how to prompt it to work with code
by rektide 3y ago
Im hacking some really dumb code this week & just tried to get StarCoder.cpp to give me some help, but I don't have any idea how to prompt it to work with code I already have.
I was really surprised that all the HuggingFace stuff needed an account. I didn't have any faith my data would stay local, I didn't understand what that was all for. Which sucks a bit because StarCoder seems to have a fairly friendly vscode extension, Im just too scared to use it.
I think maybe the trick is to just write code comments & ask for help in them? The vscode extension seems to just upload the file, wrapping everything before your cursor in {start token}/* your code here */{end token}.
I'm obviously a total newb here, but new a little tiny bit about LLM, how they are tokenizing systems. It still stuns me a bit seeing that these systems absolutely have the most minimal ability to capture context/hints from the rest of the project, from typescript definitions.
- bavell 3y agoI'm also struggling a bit to get some decent code output (TS) from local LLMs via oobabooga. I've been getting mixed results and I think it's pretty sensitive to the exact combo of model/parameters/prompt. If any one is off or not tuned to work with the others then results aren't great. E.g. some models expect very specific formatting of prompts. Some models give nonsense output at temperature=0.7 while others work great. Some models have been fine-tuned on chatbot-bot like instructions (ala chatgpt, so-called "Instruct" models) while others just autocomplete the given prompt. Very important to use the right combo to get the best results.