5 ms·

Gpt-J is of comparable quality, within 7-10% of the performance of gpt-3 in almost all metrics. It's also much smaller and less expensive to run. The higher qu

Gpt-J is of comparable quality, within 7-10% of the performance of gpt-3 in almost all metrics. It's also much smaller and less expensive to run. The higher quality training data and better tweaks to the algorithm paid off - the license, restrictions, and cost of gpt-3 aren't necessarily valuable enough to justify not using gpt-j.

stavros 5y ago
Hmm, that's very interesting! Do you know if there's a hosted service anywhere? I don't mind paying a few dollars a month for my small use case, but my usage can't justify the huge server it needs to run.
- harpersealtako 5y ago
  NovelAI is already a hosted service you pay for. It is specifically used for fiction writing, though it's got a ridiculous amount of neat experimental features, from prefix tuning (a lightweight ad-hoc fine-tuning method which can make the AI write in a specific style based on a training dataset, you can train your own with a custom service they run too or just import one of the thousands other users already made) to keyword replacement for "memory" past the general context limit, to inline annotations ("author's note") which can steer the AI towards a particular direction, style, or theme. That said if you just want to see how GPT-J-6B works there's a browser demo here: https://6b.eleuther.ai/ https://6b.eleuther.ai/
  - stavros 5y ago
    Excellent, thank you! Apparently GPT-J will run on a desktop as well (via Huggingface) but I think it needs slightly more RAM than my 16 GB.
- robbedpeter 5y ago
  Huggingface.co is awesome! You can also run it on colab. $10 a month buys you a lot of value with colab.
  - stavros 5y ago
    I'm not quite sure what Colab is, I'll have to look into it, thank you! EDIT: Ah, looks like it's only available in a handful of countries, sadly.
    - robbedpeter 5y ago
      A good VPN would only cost another $10 or less each month, or you could set up a vps hosted in the US. A barebones Linode runs $5ish. Huggingface is definitely sufficient though, and if you're messing with local apis or tinkering, gpt-neo-125m can be run on cpu with under a gig of ram.

stavros 5y ago

Hmm, that's very interesting! Do you know if there's a hosted service anywhere? I don't mind paying a few dollars a month for my small use case, but my usage can't justify the huge server it needs to run.

harpersealtako 5y ago
NovelAI is already a hosted service you pay for. It is specifically used for fiction writing, though it's got a ridiculous amount of neat experimental features, from prefix tuning (a lightweight ad-hoc fine-tuning method which can make the AI write in a specific style based on a training dataset, you can train your own with a custom service they run too or just import one of the thousands other users already made) to keyword replacement for "memory" past the general context limit, to inline annotations ("author's note") which can steer the AI towards a particular direction, style, or theme. That said if you just want to see how GPT-J-6B works there's a browser demo here: https://6b.eleuther.ai/ https://6b.eleuther.ai/
- stavros 5y ago
  Excellent, thank you! Apparently GPT-J will run on a desktop as well (via Huggingface) but I think it needs slightly more RAM than my 16 GB.
robbedpeter 5y ago
Huggingface.co is awesome! You can also run it on colab. $10 a month buys you a lot of value with colab.
- stavros 5y ago
  I'm not quite sure what Colab is, I'll have to look into it, thank you! EDIT: Ah, looks like it's only available in a handful of countries, sadly.
  - robbedpeter 5y ago
    A good VPN would only cost another $10 or less each month, or you could set up a vps hosted in the US. A barebones Linode runs $5ish. Huggingface is definitely sufficient though, and if you're messing with local apis or tinkering, gpt-neo-125m can be run on cpu with under a gig of ram.