4 ms·

One way to connect llama-2 (cpp) to a node.js app is by using this helper class (stdin) https://gist.github.com/HackyDev/814c6d1c96f259a13dbf5b2dabf98e8f https:

• lgrammel 3y ago
  You can also spin up the llama.cpp server ( https://github.com/ggerganov/llama.cpp/tree/master/examples/server https://github.com/ggerganov/llama.cpp/tree/master/examples/... ) and e.g. use ModelFusion to generate text, stream text, tokenize, etc. : https://modelfusion.dev/integration/model-provider/llamacpp https://modelfusion.dev/integration/model-provider/llamacpp