2 ms·
> shovels the last N lines to an LLM and asks it for advice. Theoretically, you could set this up locally with llama.cpp (and any terminal that runs a command
by brucethemoose2 3y ago
> shovels the last N lines to an LLM and asks it for advice.
Theoretically, you could set this up locally with llama.cpp (and any terminal that runs a command with a hotkey) right now. A 7B or 13B finetune will start up pretty quick.
Or in a company environment, you could keep it loaded on a server for faster responses.
Also, you could stuff previous outputs into a vector DB and get context from you (or colleague's) previous outputs.