4 ms·

woah, this is fast. On my M1 this feels about as fast as GPT-4.

by mritchie712 3y ago

pmarreck 3y ago
Same here on M1 Max Macbook Pro. This is great!
pyinstallwoes 3y ago
How good is it in comparison
- int_19h 3y ago
  The best models available to the public are only slightly better than the original (pre-turbo) GPT-3.5 on actual tasks. There's nothing even remotely close to GPT-4.
  - pyinstallwoes 3y ago
    What’s the best in terms of coding assistance? What’s annoying about gpt 4 is that is seems badly nerfed in many ways. It is obviously being conditioned in its own political bias.
    - dmd 3y ago
      ... is the javascript it's writing for you too 'woke' or something?
      - pyinstallwoes 3y ago
        Lately it’s been refusing to write code at all. // implementation details here. I think it’s the opposite of woke, it’s slept!
    - eyegor 3y ago
      In my experience, the deepseek-coder-instruct family is at least as good as gpt-3.5-turbo on python. Even the 1.3b models are very good (and run okay on cpu), although you should use larger if you have the vram. There are even larger models (30b+) if you are drowning in vram, but I don't think they perform much better at coding than deepseek-coder-instruct 6.7b. 3-4gb vram or cpu (1.3b): https://huggingface.co/TheBloke/deepseek-coder-1.3b-instruct-GGUF https://huggingface.co/TheBloke/deepseek-coder-1.3b-instruct... Alternative for chat (1.3b): https://huggingface.co/TheBloke/evolvedSeeker_1_3-GGUF https://huggingface.co/TheBloke/evolvedSeeker_1_3-GGUF Alternative for chat (3b): https://huggingface.co/TheBloke/open-llama-3b-v2-wizard-evol-instuct-v2-196k-GGUF https://huggingface.co/TheBloke/open-llama-3b-v2-wizard-evol... 6-8gb vram (6.7b): https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...
      - rightbyte 3y ago
        Really? How could this model not have had front page articles on HN? Is the self-hosted one as good as the one or their website?
        eyegor 3y ago
        Hmm, I just assumed people knew about it. Submitted https://news.ycombinator.com/item?id=38495176 https://news.ycombinator.com/item?id=38495176 Self hosted 6.7b model is phenomenal in my experience, I actually prefer it to chatgpt a lot of the time. Similar quality code but fewer disclaimers and #todo in my experience. Gpt4 is still a bit better at coding but not by much and it's much slower for me.
    - int_19h 3y ago
      The best in terms of coding assistance, and really for anything else, is the original (pre-turbo) GPT-4 used via the API, although this will also be more costly. There are many third-party chat apps that are wrappers around that now if you want a ChatGPT-like experience. This can also significantly reduce its bias since you are in control of the system prompt. But also, even ChatGPT can be trivially made to behave differently by saying that you're writing a book or making a video game etc, describing a character in it, and then asking it how that character would have responded in such and such situation.

woah, this is fast. On my M1 this feels about as fast as GPT-4.

pmarreck 3y ago

Same here on M1 Max Macbook Pro. This is great!

pyinstallwoes 3y ago

How good is it in comparison

int_19h 3y ago
The best models available to the public are only slightly better than the original (pre-turbo) GPT-3.5 on actual tasks. There's nothing even remotely close to GPT-4.
- pyinstallwoes 3y ago
  What’s the best in terms of coding assistance? What’s annoying about gpt 4 is that is seems badly nerfed in many ways. It is obviously being conditioned in its own political bias.
  - dmd 3y ago
    ... is the javascript it's writing for you too 'woke' or something?
    - pyinstallwoes 3y ago
      Lately it’s been refusing to write code at all. // implementation details here. I think it’s the opposite of woke, it’s slept!
  - eyegor 3y ago
    In my experience, the deepseek-coder-instruct family is at least as good as gpt-3.5-turbo on python. Even the 1.3b models are very good (and run okay on cpu), although you should use larger if you have the vram. There are even larger models (30b+) if you are drowning in vram, but I don't think they perform much better at coding than deepseek-coder-instruct 6.7b. 3-4gb vram or cpu (1.3b): https://huggingface.co/TheBloke/deepseek-coder-1.3b-instruct-GGUF https://huggingface.co/TheBloke/deepseek-coder-1.3b-instruct... Alternative for chat (1.3b): https://huggingface.co/TheBloke/evolvedSeeker_1_3-GGUF https://huggingface.co/TheBloke/evolvedSeeker_1_3-GGUF Alternative for chat (3b): https://huggingface.co/TheBloke/open-llama-3b-v2-wizard-evol-instuct-v2-196k-GGUF https://huggingface.co/TheBloke/open-llama-3b-v2-wizard-evol... 6-8gb vram (6.7b): https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct-GGUF https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...
    - rightbyte 3y ago
      Really? How could this model not have had front page articles on HN? Is the self-hosted one as good as the one or their website?
      - eyegor 3y ago
        Hmm, I just assumed people knew about it. Submitted https://news.ycombinator.com/item?id=38495176 https://news.ycombinator.com/item?id=38495176 Self hosted 6.7b model is phenomenal in my experience, I actually prefer it to chatgpt a lot of the time. Similar quality code but fewer disclaimers and #todo in my experience. Gpt4 is still a bit better at coding but not by much and it's much slower for me.
  - int_19h 3y ago
    The best in terms of coding assistance, and really for anything else, is the original (pre-turbo) GPT-4 used via the API, although this will also be more costly. There are many third-party chat apps that are wrappers around that now if you want a ChatGPT-like experience. This can also significantly reduce its bias since you are in control of the system prompt. But also, even ChatGPT can be trivially made to behave differently by saying that you're writing a book or making a video game etc, describing a character in it, and then asking it how that character would have responded in such and such situation.