3 ms·

Oof, got access to the 8k model recently and was wondering what costs would be on the 32k one. That's brutal.

freediver 3y ago
Why? 32K tokens would cover creating source code for a small to medium programming project. THat would easilly cost hundreds of dollars if done by a freelancer, and could get it in almost real time for just $2.
- replygirl 3y ago
  32k tokens is 500-1000 lines of code, so more like thousands of dollars, unless you're comparing against a landing page or CRUD tool arranged through Upwork. On the flip side, before I dove into AutoGPT I couldn't get GPT-4 to iterate on something as simple as a TypeScript function that deeply converts { foo_bar: string } to { fooBar: Date } without running out of context and cyclically reintroducing regressions.
  - hesdeadjim 3y ago
    Advice on where to get started with auto-gpt?
- jiggawatts 3y ago
  Only if you can get that result in one pass. The real utility of LLMs is that they can be called in a loop to scan through many web pages, many code files, issue tickets, emails, etc... There are already demos and experiments out there that for every input, 4x outputs are generated, then those are fed back into the LLM 4x for "review", then the best variant is then used to generate code which then automatically tested, errors are fed back in a loop, also with 4x parallel tries, etc... It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it.
  - dragonwriter 3y ago
    > It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it. If its better in another dimension (e.g., calendar elapsed time), it may well be worth being more expensive.
  - koboll 3y ago
    Two dollars spent over and over would still take a while to equal a software engineer's salary, and if the end result is "produce a fully functional codebase in minutes" the premium might be worth it even if it exceeds that much.
    - gmerc 3y ago
      We all know the game Sam Altman is playing. Eventually you can squeeze the full salary in cost out of companies who have laid off their workers and it would still be a good deal because AI does not need health care, pay taxes, hardware, sick days and so on.

Why? 32K tokens would cover creating source code for a small to medium programming project. THat would easilly cost hundreds of dollars if done by a freelancer, and could get it in almost real time for just $2.

replygirl 3y ago
32k tokens is 500-1000 lines of code, so more like thousands of dollars, unless you're comparing against a landing page or CRUD tool arranged through Upwork. On the flip side, before I dove into AutoGPT I couldn't get GPT-4 to iterate on something as simple as a TypeScript function that deeply converts { foo_bar: string } to { fooBar: Date } without running out of context and cyclically reintroducing regressions.
- hesdeadjim 3y ago
  Advice on where to get started with auto-gpt?
jiggawatts 3y ago
Only if you can get that result in one pass. The real utility of LLMs is that they can be called in a loop to scan through many web pages, many code files, issue tickets, emails, etc... There are already demos and experiments out there that for every input, 4x outputs are generated, then those are fed back into the LLM 4x for "review", then the best variant is then used to generate code which then automatically tested, errors are fed back in a loop, also with 4x parallel tries, etc... It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it.
- dragonwriter 3y ago
  > It's the throughput compared to humans that is the true differentiator. If hooking up the API in a loop up ends up costing more than a human, then it's not worth it. If its better in another dimension (e.g., calendar elapsed time), it may well be worth being more expensive.
- koboll 3y ago
  Two dollars spent over and over would still take a while to equal a software engineer's salary, and if the end result is "produce a fully functional codebase in minutes" the premium might be worth it even if it exceeds that much.
  - gmerc 3y ago
    We all know the game Sam Altman is playing. Eventually you can squeeze the full salary in cost out of companies who have laid off their workers and it would still be a good deal because AI does not need health care, pay taxes, hardware, sick days and so on.