2 ms·

I've found ChatGPT useful for coming up with ideas and possible solutions. Left to its own devices, however, it would fail to do anything meaningful in its curr

by jasfi 4y ago

I've found ChatGPT useful for coming up with ideas and possible solutions. Left to its own devices, however, it would fail to do anything meaningful in its current state. It just gets things wrong too often, or at least, requires constant supervision to head in the right direction. The more input is required from ChatGPT, the greater the likelihood that it will break something.

BoorishBears 4y ago
If you're using it in the default chat UI, you aren't getting exposed to 1/10th of what it can do. Even the playground is a step up because you can modify conversations in-place, change temperature (which influences how often it needs supervision), various penalties, etc. For example, if you want it to generate creative text, but need it in JSON, you can give it an instruction with temperature set to the max (which biases it towards novel output). Then follow up with asking it to transform that output to match json schema with temperature 0 (which biases it towards plainly following instructions) And there's an edit model too now, so in my previous example you can also leave the temperature maxed out, probably get some mistakes, and feed that to the edit model telling it to fix the mistakes - The API is another step up because it makes it easy to chain different conversations. Often times you get a much better result by running different prompts for each step of a process, and in the web UI that'd be a nightmare.
- jasfi 4y ago
  Interesting, but why use ChatGPT in an API instead of GPT-4?
  - dragonwriter 4y ago
    GPT-4 (depending on which model and whether you look at prompt vs. response tokens) is between 15 and 60 times the cost per token of GPT-3.5. Also, GPT-4 API is doing a slow roll-out, so unless you’ve been granted access, you may not have access to it at all.
  - BoorishBears 4y ago
    As the other reply brought up, GPT-4 access is waitlisted, and costs significantly more API access makes it easy to chew through a lot of tokens. With GPT3.5 I hardly have to think about the cost of tinkering for hours on end. Yesterday I made over 600 requests and wracked up a whopping 67 cent bill... At an order of magnitude more, I'd still tinker, but wouldn't feel comfortable with all of the same usecases (ie. setting it up in ways that allow it to recurse very quickly like I do with 3.5)
- deleted 4y ago
  [deleted]

by jasfi 4y ago

BoorishBears 4y ago

If you're using it in the default chat UI, you aren't getting exposed to 1/10th of what it can do. Even the playground is a step up because you can modify conversations in-place, change temperature (which influences how often it needs supervision), various penalties, etc. For example, if you want it to generate creative text, but need it in JSON, you can give it an instruction with temperature set to the max (which biases it towards novel output). Then follow up with asking it to transform that output to match json schema with temperature 0 (which biases it towards plainly following instructions) And there's an edit model too now, so in my previous example you can also leave the temperature maxed out, probably get some mistakes, and feed that to the edit model telling it to fix the mistakes - The API is another step up because it makes it easy to chain different conversations. Often times you get a much better result by running different prompts for each step of a process, and in the web UI that'd be a nightmare.

jasfi 4y ago
Interesting, but why use ChatGPT in an API instead of GPT-4?
- dragonwriter 4y ago
  GPT-4 (depending on which model and whether you look at prompt vs. response tokens) is between 15 and 60 times the cost per token of GPT-3.5. Also, GPT-4 API is doing a slow roll-out, so unless you’ve been granted access, you may not have access to it at all.
- BoorishBears 4y ago
  As the other reply brought up, GPT-4 access is waitlisted, and costs significantly more API access makes it easy to chew through a lot of tokens. With GPT3.5 I hardly have to think about the cost of tinkering for hours on end. Yesterday I made over 600 requests and wracked up a whopping 67 cent bill... At an order of magnitude more, I'd still tinker, but wouldn't feel comfortable with all of the same usecases (ie. setting it up in ways that allow it to recurse very quickly like I do with 3.5)
deleted 4y ago
[deleted]