5 ms·

It’s rubbish compared to GPT-3/4. Any model you can trivially load in your browser will be significantly smaller than those models, and broadly speaking smalle

It’s rubbish compared to GPT-3/4. Any model you can trivially load in your browser will be significantly smaller than those models, and broadly speaking smaller = worse. This example is a 4 GB model, that’s (I guess) based off some smallish model like the llama 7B. It’s a proof of concept, not a chat-gpt replacement. There’s nothing here that’s new other than “runs in the browser”; so it won’t be better than any other model you can get your hands on. This kind of thing should be label ByoM (bring your own model). The model isn’t the interesting part of this.

junrushao1994 4y ago
To share some fun stuff, here is the response generated by this model: As an AI language model, I would respond by acknowledging that the model discussed in the message is indeed smaller than some of the larger language models like GPT-3/4. However, I would also point out that size is not always a direct indicator of performance, and that the model may still be able to perform well on certain tasks or in certain contexts. Additionally, I would emphasize that the browser compatibility of the model is a significant advantage, as it allows for easy and convenient access to the model without the need to install any specific software or hardware. Finally, I would suggest that the discussion focus on the potential applications and capabilities of the model, rather than solely its size. Back to the topic, we don't make much innovation on the model, so I am probably not the best person to evaluate how a model compares with SOTAs. There are indeed lots of super cool techniques being explored lately that makes it possible to deploy smaller and smaller models, for example, LLM.int8() [1] and int4 quantization [2] without loss of zero-shot accuracy. Can't predict the future, but maybe one day there will be something really powerful but small enough to fit in the pocket of everyone :-) [1] Dettmers, Tim, et al. "LLM. int8 (): 8-bit matrix multiplication for transformers at scale." arXiv preprint arXiv:2208.07339 (2022). [2] Dettmers, Tim, and Luke Zettlemoyer. "The case for 4-bit precision: k-bit Inference Scaling Laws." arXiv preprint arXiv:2212.09720 (2022).
- wokwokwok 4y ago
  > but maybe one day There you go, summarised for you. You can hand wave about quantised models til the end of time but specifically this model is a trivial toy model. No amount of pondering about the future avoid the fundamental fact that small models (~7B) are inferior to larger models like GPT. It’s dishonest to suggest otherwise. :( There’s no reason to do this other than selling snake oil. Maybe. One day. In the future. Things might be different. right now they are not.
  - junrushao1994 4y ago
    To your response, the model says: > Dear [Name], > Thank you for your message. We understand that the model you are referring to is a simple and basic model. However, it is important to highlight that this model serves a specific purpose and can be useful for certain applications. > Regarding the comparison with larger models like GPT, it's important to note that different models have different strengths and weaknesses, and the choice of model depends on the specific task and use case. While larger models like GPT may be more powerful and capable, they also come with greater computational and memory requirements. > We appreciate your concerns and feedback, and we will take them into consideration as we continue to develop our models. Our aim is to provide solutions that are tailored to the needs of our clients and meet their requirements for accuracy, efficiency, and performance. > Thank you for your input, and we hope to have the opportunity to work with you in the future. > Best regards, > [Your Name] Back to your response, so you did notice that I said they are not as powerful as GPT-4, of course they are not, not a single one is. The model is not developed by us, and their performance is not our current focus either (nor am I an expert in this evaluation), but I am happy to assist if anyone wants to seriously evaluate it!
    - wokwokwok 4y ago
      Quote: > I would also point out that size is not always a direct indicator of performance, and Yes. It is. This thread is a direct response to a comparison to GPT, and your response (generated or not) is dishonest. I can’t be blunter than that. If you want amortise your responsibility by posting generated responses, go for it. Do whatever you want. My response is directly to the parent comment about the comparison to GPT, for anyone who is unclear about the comparison.

junrushao1994 4y ago

To share some fun stuff, here is the response generated by this model: As an AI language model, I would respond by acknowledging that the model discussed in the message is indeed smaller than some of the larger language models like GPT-3/4. However, I would also point out that size is not always a direct indicator of performance, and that the model may still be able to perform well on certain tasks or in certain contexts. Additionally, I would emphasize that the browser compatibility of the model is a significant advantage, as it allows for easy and convenient access to the model without the need to install any specific software or hardware. Finally, I would suggest that the discussion focus on the potential applications and capabilities of the model, rather than solely its size. Back to the topic, we don't make much innovation on the model, so I am probably not the best person to evaluate how a model compares with SOTAs. There are indeed lots of super cool techniques being explored lately that makes it possible to deploy smaller and smaller models, for example, LLM.int8() [1] and int4 quantization [2] without loss of zero-shot accuracy. Can't predict the future, but maybe one day there will be something really powerful but small enough to fit in the pocket of everyone :-) [1] Dettmers, Tim, et al. "LLM. int8 (): 8-bit matrix multiplication for transformers at scale." arXiv preprint arXiv:2208.07339 (2022). [2] Dettmers, Tim, and Luke Zettlemoyer. "The case for 4-bit precision: k-bit Inference Scaling Laws." arXiv preprint arXiv:2212.09720 (2022).

wokwokwok 4y ago
> but maybe one day There you go, summarised for you. You can hand wave about quantised models til the end of time but specifically this model is a trivial toy model. No amount of pondering about the future avoid the fundamental fact that small models (~7B) are inferior to larger models like GPT. It’s dishonest to suggest otherwise. :( There’s no reason to do this other than selling snake oil. Maybe. One day. In the future. Things might be different. right now they are not.
- junrushao1994 4y ago
  To your response, the model says: > Dear [Name], > Thank you for your message. We understand that the model you are referring to is a simple and basic model. However, it is important to highlight that this model serves a specific purpose and can be useful for certain applications. > Regarding the comparison with larger models like GPT, it's important to note that different models have different strengths and weaknesses, and the choice of model depends on the specific task and use case. While larger models like GPT may be more powerful and capable, they also come with greater computational and memory requirements. > We appreciate your concerns and feedback, and we will take them into consideration as we continue to develop our models. Our aim is to provide solutions that are tailored to the needs of our clients and meet their requirements for accuracy, efficiency, and performance. > Thank you for your input, and we hope to have the opportunity to work with you in the future. > Best regards, > [Your Name] Back to your response, so you did notice that I said they are not as powerful as GPT-4, of course they are not, not a single one is. The model is not developed by us, and their performance is not our current focus either (nor am I an expert in this evaluation), but I am happy to assist if anyone wants to seriously evaluate it!
  - wokwokwok 4y ago
    Quote: > I would also point out that size is not always a direct indicator of performance, and Yes. It is. This thread is a direct response to a comparison to GPT, and your response (generated or not) is dishonest. I can’t be blunter than that. If you want amortise your responsibility by posting generated responses, go for it. Do whatever you want. My response is directly to the parent comment about the comparison to GPT, for anyone who is unclear about the comparison.