4 ms·

Show HN: ChatGPT Alternative with LLaMA Models

20 points by namanski 3y ago

Hey folks, we created a ChatGPT alternative for anyone to try out LLaMA models and benchmark responses across OpenAI's models and other open source models. Would love some feedback! No data is being used for retraining models.

gardenhedge 3y ago
seems to be better than gpt-3.5
- anshuman_01_ 3y ago
  I am definitely seeing results as good as 3.5 as well
he11ow 3y ago
This looks great! Can I ask, are you using a HuggingFace endpoint to hit the LLaMA models, or deploying it yourself? Still new to this and trying to understand how putting the large models in production works...
- namanski 3y ago
  Hey hey! We have deployed this on our cloud. It’s running on 2 A10Gs on AWS in the background. We had the tech from our MLOps platform NimbleBox.ai that let us setup a managed service on all major cloud providers so we just frankenstein-ed it to work for LLMs as well :) The prompt engineering, specially for web search, is powered by our open-source tool ChainFury (https://chainfury.nbox.ai/ https://chainfury.nbox.ai/)
  - he11ow 3y ago
    Thanks! Will check out these services, MLOps is definitely where the biggest pain points are right now.
muzani 3y ago
Nice, llama is much better than I expected. It matches ChatGPT quality for everything I'd use it for, though I haven't tested it on code yet because of data issues and such. Is the censorship placed on top of the llama model or is it a part of it?
- namanski 3y ago
  Yeah - It's at par, if not better than GPT 3.5! This is the base model (13B) with no fine-tuning or censorship. Feel free to give it a spin for code as well! We're just the infrastructure layer here so we don't use any data for retraining these models. LLaMA 2 70B coming soon! :D
prabhatsharma 3y ago
this looks awesome. when is LLaMA 2 70B coming?
- cshubhamrao 3y ago
  Thanks :) LLaMA 2 70B is live now on https://chat.nbox.ai https://chat.nbox.ai
replwoacause 3y ago
First question I asked about Trump to see if it really is uncensored and it said “I cannot provide personal opinions or beliefs about individuals. “

20 points by namanski 3y ago

gardenhedge 3y ago

seems to be better than gpt-3.5

anshuman_01_ 3y ago
I am definitely seeing results as good as 3.5 as well

he11ow 3y ago

This looks great! Can I ask, are you using a HuggingFace endpoint to hit the LLaMA models, or deploying it yourself? Still new to this and trying to understand how putting the large models in production works...

namanski 3y ago
Hey hey! We have deployed this on our cloud. It’s running on 2 A10Gs on AWS in the background. We had the tech from our MLOps platform NimbleBox.ai that let us setup a managed service on all major cloud providers so we just frankenstein-ed it to work for LLMs as well :) The prompt engineering, specially for web search, is powered by our open-source tool ChainFury (https://chainfury.nbox.ai/ https://chainfury.nbox.ai/)
- he11ow 3y ago
  Thanks! Will check out these services, MLOps is definitely where the biggest pain points are right now.

muzani 3y ago

Nice, llama is much better than I expected. It matches ChatGPT quality for everything I'd use it for, though I haven't tested it on code yet because of data issues and such. Is the censorship placed on top of the llama model or is it a part of it?

namanski 3y ago
Yeah - It's at par, if not better than GPT 3.5! This is the base model (13B) with no fine-tuning or censorship. Feel free to give it a spin for code as well! We're just the infrastructure layer here so we don't use any data for retraining these models. LLaMA 2 70B coming soon! :D

prabhatsharma 3y ago

this looks awesome. when is LLaMA 2 70B coming?

cshubhamrao 3y ago
Thanks :) LLaMA 2 70B is live now on https://chat.nbox.ai https://chat.nbox.ai

replwoacause 3y ago

First question I asked about Trump to see if it really is uncensored and it said “I cannot provide personal opinions or beliefs about individuals. “