4 ms·

There's an online demo of Vicuna-13b where you can test its efficiency: https://chat.lmsys.org/ https://chat.lmsys.org/

• csdvrx 3y ago
  Yes, but can you replicate that functionality using llama.cpp? If so, what did you run with main? I haven't been able to get an answer, while for the question above, I can get 'I chose the word "laptop"' with mlc-llm
  • elorant 3y ago
    For the tasks I need it the efficiency is similar to the online model. Only slower. I don't care for conversational functionality.
• techload 3y ago
  After two prompts I was astounded by the innacuracies present in the answers. An they were pretty easy questions.