3 ms·

Had a quick look and couldn't find anything obvious - is it easy to point to a llama.cpp server on a different machine pretending to be openai?

• elijahbenizzy 3y ago
  So yeah, shouldn't be too hard, but Burr does not replace LLM API calls -- you can bring whatever frameworks you want. You could do this by writing a simple function: @action(reads=["prompt"], writes=["response"]) def my_action(state: State) -> Tuple[dict, State]: prompt = state["prompt"] # you stick in whatever library you want for the call llm_response = whatever_library(prompt) state = state.update(response=llm_response) return {"response" : llm_response}, state.update(response=llm_response) Does this help?
  • exe34 3y ago
    Makes sense, thanks!