3 ms·

I wish there was a locked chatgpt version for researchers

by dartos 2y ago

eru 2y ago
Some of the open-weights models come close to what you might want?
- dartos 2y ago
  Sort of. They fill the “locked” requirement, but open models aren’t SOTA like gpt or Claud.
  - coder543 2y ago
    Llama3.1-405B benchmarks at a level that I would certainly consider comparable to other SOTA models, as long as multimodality isn’t needed. Regardless, OpenAI provides access to quite a few of their older models through the API, since the API lets you pass in a specific model version. I’m sure the older models won’t be available forever, but that is a much more stable target for researchers than just opening the ChatGPT website and typing in things.
    - dartos 2y ago
      ChatGPT specifically is more than just a chat interface on top of got-4. Their system prompt includes the current date and time among other information, making it very very hard to run reproducible experiments against it. But it’s the tool most people are using.
  - Reubend 2y ago
    LLAMA 3.1 405b-Instruct is state of the art at the time of writing. It averages slightly worse than ChatGPT 4o, but to a mostly negligible extent for most research.
    - toxik 2y ago
      Aside but -- it is also aligned to not generate profane text, right?
      - throwthrowuknow 2y ago
        Abliterate it
rck 2y ago
You could play with open models and different temperatures using the lmgen script from this repo we just released: https://github.com/RichardKelley/hflm?tab=readme-ov-file#lmgen https://github.com/RichardKelley/hflm?tab=readme-ov-file#lmg...
jijji 2y ago
there sort of is, if you install ollama (https://ollama.com https://ollama.com) and then execute: ollama run llama2-uncensored it will install and run the local chat interface for llama2 in an uncensored version which gives a little bit better results with less guardrails. Same with wizardlm-uncensored and wizard-vicuna-uncensored. For reference, the SOTA version with guardrails you could run: ollama run llama3.1:405b

I wish there was a locked chatgpt version for researchers

by dartos 2y ago

I wish there was a locked chatgpt version for researchers

eru 2y ago

Some of the open-weights models come close to what you might want?

dartos 2y ago
Sort of. They fill the “locked” requirement, but open models aren’t SOTA like gpt or Claud.
- coder543 2y ago
  Llama3.1-405B benchmarks at a level that I would certainly consider comparable to other SOTA models, as long as multimodality isn’t needed. Regardless, OpenAI provides access to quite a few of their older models through the API, since the API lets you pass in a specific model version. I’m sure the older models won’t be available forever, but that is a much more stable target for researchers than just opening the ChatGPT website and typing in things.
  - dartos 2y ago
    ChatGPT specifically is more than just a chat interface on top of got-4. Their system prompt includes the current date and time among other information, making it very very hard to run reproducible experiments against it. But it’s the tool most people are using.
- Reubend 2y ago
  LLAMA 3.1 405b-Instruct is state of the art at the time of writing. It averages slightly worse than ChatGPT 4o, but to a mostly negligible extent for most research.
  - toxik 2y ago
    Aside but -- it is also aligned to not generate profane text, right?
    - throwthrowuknow 2y ago
      Abliterate it

rck 2y ago

You could play with open models and different temperatures using the lmgen script from this repo we just released: https://github.com/RichardKelley/hflm?tab=readme-ov-file#lmgen https://github.com/RichardKelley/hflm?tab=readme-ov-file#lmg...

jijji 2y ago

there sort of is, if you install ollama (https://ollama.com https://ollama.com) and then execute: ollama run llama2-uncensored it will install and run the local chat interface for llama2 in an uncensored version which gives a little bit better results with less guardrails. Same with wizardlm-uncensored and wizard-vicuna-uncensored. For reference, the SOTA version with guardrails you could run: ollama run llama3.1:405b