5 ms·

Pricing: $150 / 1M input tokens, $600 / 1M output tokens. (Not a typo.) Very expensive, but I've been using it with my ChatGPT Pro subscription and it's remark

Pricing: $150 / 1M input tokens, $600 / 1M output tokens. (Not a typo.) Very expensive, but I've been using it with my ChatGPT Pro subscription and it's remarkably capable. I'll give it 100,000 token codebases and it'll find nuanced bugs I completely overlooked. (Now I almost feel bad considering the API price vs. the price I pay for the subscription.)

hooloovoo_zoo 2y ago
Is your prompt {$codebase} find bugs?
- davidbarker 2y ago
  Typically something like: Look carefully through my codebase and identify any bugs/issues, or refactors that could improve it. <codebase> … </codebase> Doesn't have to be anything overly complicated to get good results. It also does well if you give it a git diff.
  - ionwake 2y ago
    Sorry if this is a noob question, but are you just pasting file strings inbetween those tags? like the contents of file1.js and file2.js?
    - pridkett 2y ago
      Repomix can take care of this for you. I pack it, cat the file to my clipboard with pbcopy, and just paste it into the prompt. https://github.com/yamadashy/repomix https://github.com/yamadashy/repomix
      - diggan 2y ago
        I tried the web demo (https://repomix.com/ https://repomix.com/) and it seems to generate unnecessarily complex "packs" for no reason, probably hurts LLM performance too. Why is there "Usage Guidelines" and "File Format" explanations in this, when it's supposed to just be the code "packed"? Better to just have the contents+filename, it'll infer that its directory structure and everything else.
        jakereps 2y ago
        While possibly being strange defaults, both of those are options. Remove the file summary and directory structure, both featured on the UI, and on the CLI tool, and voila, it's in your "better" state. There are also additional compression options beyond those two tweaks.
    - diggan 2y ago
      I do something similar, but "raw" markdown instead + filename, so all my prompts end up like this basically: Do blah blah blah while taking blah and blah into account. Here is my current code: File `file1.js`: ```javascript console.log('I am number one!') ``` File `file2.js`: ```javascript console.log("I am number two :(") ``` Not sure if I'm imagining, but when I tried with/without the markdown code blocks, it seems to do better when I used markdown code blocks, so wrote a quick CLI that takes a directory path + prompt and creates something like that automatically for me. Often times I send identical prompts to ChatGPT+DeepThink+Claude, compare the approaches and continue with the one that works best for that particular problem, so having something reusable really saved time for this. Edit: fuck it, in case people are curious how my little CLI works, I threw it up here: https://github.com/victorb/prompta https://github.com/victorb/prompta (beware of bugs and whatnot, I've quite literally hacked this together without much thought)
      - atxtechbro 2y ago
        I actually have a perfect tool called siphon-cli for this. It adds the headers in between files and everything. https://docs.siphon-cli.com/ https://docs.siphon-cli.com/
        diggan 2y ago
        Yeah, like a lightweight version of my prompta CLI :) What I end up with, is one .md file that uses variables like "$SRC", "$TESTS" and "$DOCS" inside of it, that gets replaced when you run `prompta output`, and then there is also a JSON file that defines what those variables actually get replaced with. Bit off-topic, but curious how your repository ends up having 8023 lines of something for concatenating files, while my own CLI sits on 687 lines (500 of those are Rust) but has a lot more functionality :)
        quartzic 2y ago
        Not OP, but practically all of those lines are from a package-lock.json file (6755 lines) and a changelog (541 lines). It looks like the actual source is 179 lines long.
    - keizo 2y ago
      I wrote a script to do exactly this a while ago (with the help of o1 pro). Makes it way easier https://github.com/keizo/ggrab https://github.com/keizo/ggrab
    - davidbarker 2y ago
      That's right. I made a VS Code extension to combine all the files I have open into one long string I can copy & paste. https://marketplace.visualstudio.com/items?itemName=DVYIO.combine-open-files https://marketplace.visualstudio.com/items?itemName=DVYIO.co...
  - crossroadsguy 2y ago
    Do you get this - When you say: But is that really a bug? GPT: That's right. Now that I see it again this is not a bug….and a lot of blah blah.
ldjkfkdsjnv 2y ago
As far as I'm concerned, all of the other models are a waste of time to use in comparison. Most people dont know how good this model is
- dinobones 2y ago
  Interesting... Most benchmarks show this model as being worse than o3-mini-high and sonnet3.7. What difference are you seeing from these models that makes it better? I say this as someone considering shelling out $200 for ChatGPT pro for this.
  - ldjkfkdsjnv 2y ago
    I regularly push 100k+ tokens into it. So most of my code base/large portions. I use the Repo Prompt product to construct the code prompts. It finds bugs and solutions at a rate that is far better than others. I also speak into the prompt to describe my problem, and find spoken language is interpreted very well. I also frequently download all the source code of libraries I am debugging, and when running into issues, pass that code in along with my own broken code. Its very good
  - jbellis 2y ago
    If you're in the habit of breaking down problems to Sonnet-sized pieces you won't see a benefit. The win is that o1pro lets you stop breaking down one level up from what you're used to. It may also have a larger usable context window, not totally sure about that.
    - logankeenan 2y ago
      > lets you stop breaking down one level up from what you're used to. Can you provide an example of what you mean by this? I provide very verbose prompts where I know what needs to be done and just let AI “do” the work. I’m curious how this is different?
      - jbellis 2y ago
jbellis 2y ago
Remarkably capable is a good description. Shameless plug: One of the reasons I wrote my AI coding assistant is to make it easier to get problems into o1pro. https://github.com/jbellis/brokk https://github.com/jbellis/brokk
andrewinardeer 2y ago
I wonder what the input/output tokens will be priced at for AGI.
- stavros 2y ago
  They won't. Your use cases won't be something the AI can't do itself, so why would they sell it to you instead of replace you with it? AGI means the value of a human is the same as an LLM, but the energy requirements of a human are higher than those of an LLM, so humans won't be economical any more.
  - dnadler 2y ago
    Actually, I think humans require much less energy than LLMs. Even raising a human to adulthood would be cheaper from a calorie perspective than running an AGI algorithm (probably). Its the whole reason why the premise of the Matrix was ridiculous :) Some quick back of the envelope says that it would take around 35 MWh to get to 40 years old (2000 kcal per day)
    - 12345ieee 2y ago
      You need to consider all the energy spent to bring those calories to you, easily multiplying your budget by 10 or 100.
    - kadushka 2y ago
      A human runs on ~100W, even when not doing anything useful. It's entirely plausible that 100W will be enough to run a future AGI level model.
    - gcanyon 2y ago
      I read an article once that claimed an early draft/version that was cut for time or narrative complexity had the human brains being used as raw compute for the machines, with the Matrix being the idle process to keep the minds sane and functional for their ultimate purpose.
foobiekr 2y ago
Can you describe this "find a bug" workflow?

hooloovoo_zoo 2y ago

Is your prompt {$codebase} find bugs?

davidbarker 2y ago
Typically something like: Look carefully through my codebase and identify any bugs/issues, or refactors that could improve it. <codebase> … </codebase> Doesn't have to be anything overly complicated to get good results. It also does well if you give it a git diff.
- ionwake 2y ago
  Sorry if this is a noob question, but are you just pasting file strings inbetween those tags? like the contents of file1.js and file2.js?
  - pridkett 2y ago
    Repomix can take care of this for you. I pack it, cat the file to my clipboard with pbcopy, and just paste it into the prompt. https://github.com/yamadashy/repomix https://github.com/yamadashy/repomix
    - diggan 2y ago
      I tried the web demo (https://repomix.com/ https://repomix.com/) and it seems to generate unnecessarily complex "packs" for no reason, probably hurts LLM performance too. Why is there "Usage Guidelines" and "File Format" explanations in this, when it's supposed to just be the code "packed"? Better to just have the contents+filename, it'll infer that its directory structure and everything else.
      - jakereps 2y ago
        While possibly being strange defaults, both of those are options. Remove the file summary and directory structure, both featured on the UI, and on the CLI tool, and voila, it's in your "better" state. There are also additional compression options beyond those two tweaks.
  - diggan 2y ago
    I do something similar, but "raw" markdown instead + filename, so all my prompts end up like this basically: Do blah blah blah while taking blah and blah into account. Here is my current code: File `file1.js`: ```javascript console.log('I am number one!') ``` File `file2.js`: ```javascript console.log("I am number two :(") ``` Not sure if I'm imagining, but when I tried with/without the markdown code blocks, it seems to do better when I used markdown code blocks, so wrote a quick CLI that takes a directory path + prompt and creates something like that automatically for me. Often times I send identical prompts to ChatGPT+DeepThink+Claude, compare the approaches and continue with the one that works best for that particular problem, so having something reusable really saved time for this. Edit: fuck it, in case people are curious how my little CLI works, I threw it up here: https://github.com/victorb/prompta https://github.com/victorb/prompta (beware of bugs and whatnot, I've quite literally hacked this together without much thought)
    - atxtechbro 2y ago
      I actually have a perfect tool called siphon-cli for this. It adds the headers in between files and everything. https://docs.siphon-cli.com/ https://docs.siphon-cli.com/
      - diggan 2y ago
        Yeah, like a lightweight version of my prompta CLI :) What I end up with, is one .md file that uses variables like "$SRC", "$TESTS" and "$DOCS" inside of it, that gets replaced when you run `prompta output`, and then there is also a JSON file that defines what those variables actually get replaced with. Bit off-topic, but curious how your repository ends up having 8023 lines of something for concatenating files, while my own CLI sits on 687 lines (500 of those are Rust) but has a lot more functionality :)
        quartzic 2y ago
        Not OP, but practically all of those lines are from a package-lock.json file (6755 lines) and a changelog (541 lines). It looks like the actual source is 179 lines long.
  - keizo 2y ago
    I wrote a script to do exactly this a while ago (with the help of o1 pro). Makes it way easier https://github.com/keizo/ggrab https://github.com/keizo/ggrab
  - davidbarker 2y ago
    That's right. I made a VS Code extension to combine all the files I have open into one long string I can copy & paste. https://marketplace.visualstudio.com/items?itemName=DVYIO.combine-open-files https://marketplace.visualstudio.com/items?itemName=DVYIO.co...
- crossroadsguy 2y ago
  Do you get this - When you say: But is that really a bug? GPT: That's right. Now that I see it again this is not a bug….and a lot of blah blah.

ldjkfkdsjnv 2y ago

As far as I'm concerned, all of the other models are a waste of time to use in comparison. Most people dont know how good this model is

dinobones 2y ago
Interesting... Most benchmarks show this model as being worse than o3-mini-high and sonnet3.7. What difference are you seeing from these models that makes it better? I say this as someone considering shelling out $200 for ChatGPT pro for this.
- ldjkfkdsjnv 2y ago
  I regularly push 100k+ tokens into it. So most of my code base/large portions. I use the Repo Prompt product to construct the code prompts. It finds bugs and solutions at a rate that is far better than others. I also speak into the prompt to describe my problem, and find spoken language is interpreted very well. I also frequently download all the source code of libraries I am debugging, and when running into issues, pass that code in along with my own broken code. Its very good
- jbellis 2y ago
  If you're in the habit of breaking down problems to Sonnet-sized pieces you won't see a benefit. The win is that o1pro lets you stop breaking down one level up from what you're used to. It may also have a larger usable context window, not totally sure about that.
  - logankeenan 2y ago
    > lets you stop breaking down one level up from what you're used to. Can you provide an example of what you mean by this? I provide very verbose prompts where I know what needs to be done and just let AI “do” the work. I’m curious how this is different?
    - jbellis 2y ago

jbellis 2y ago

Remarkably capable is a good description. Shameless plug: One of the reasons I wrote my AI coding assistant is to make it easier to get problems into o1pro. https://github.com/jbellis/brokk https://github.com/jbellis/brokk

andrewinardeer 2y ago

I wonder what the input/output tokens will be priced at for AGI.

stavros 2y ago
They won't. Your use cases won't be something the AI can't do itself, so why would they sell it to you instead of replace you with it? AGI means the value of a human is the same as an LLM, but the energy requirements of a human are higher than those of an LLM, so humans won't be economical any more.
- dnadler 2y ago
  Actually, I think humans require much less energy than LLMs. Even raising a human to adulthood would be cheaper from a calorie perspective than running an AGI algorithm (probably). Its the whole reason why the premise of the Matrix was ridiculous :) Some quick back of the envelope says that it would take around 35 MWh to get to 40 years old (2000 kcal per day)
  - 12345ieee 2y ago
    You need to consider all the energy spent to bring those calories to you, easily multiplying your budget by 10 or 100.
  - kadushka 2y ago
    A human runs on ~100W, even when not doing anything useful. It's entirely plausible that 100W will be enough to run a future AGI level model.
  - gcanyon 2y ago
    I read an article once that claimed an early draft/version that was cut for time or narrative complexity had the human brains being used as raw compute for the machines, with the Matrix being the idle process to keep the minds sane and functional for their ultimate purpose.

foobiekr 2y ago

Can you describe this "find a bug" workflow?