4 ms·

i shared my review inside of the pod with the team (https://latent.space/p/codex https://latent.space/p/codex) but basically: - it's a GREAT oneshot coding mod

by swyx 1y ago

i shared my review inside of the pod with the team (https://latent.space/p/codex https://latent.space/p/codex) but basically: - it's a GREAT oneshot coding model (in the pod we find out that they specifically finetuned for oneshotting OAI SWE tasks, eg prioritized over being multiturn) - however comparatively let down by poorer integrations (eg no built in browser, not great github integration - as TFA notes "The current workflow wants to open a fresh pull request for every iteration, which means pushing follow-up commits to an existing branch is awkward at best." - yeah this sucks ass) fortunately the integrations will only improve over time. i think the finding that you can do 60 concurrent Codex instances per hour is qualitatively different than Devin (5 concurrent) and Cursor (1 before the new "background agents"). btw > I haven't yet noticed a marked difference in the performance of the Codex model, which OpenAI explains is a descendant of GPT-3 and is proficient in more than 12 programming languages. incorrect, its an o3 finetune.

canadiantim 1y ago
How do you find it compares to Claude Code?
- viscanti 1y ago
  It's much more conservative in the scope of task it will attempt and it's much slower. You need to fire and forget several parallel tasks because you'll be waiting 10+ minutes before you get anything you can review and give feedback on.
- swyx 1y ago
  right now apples and oranges literally only because 1) unlimited unmetered use and 2) not in browser so async and parallel. like that stuff just trumps actual model and agent harness differences because it removes all barriers from thought to code.
liuliu 1y ago
The particularly integration pain point to me is about network access, that prohibits several banal tasks to be offloaded to codex: 1. Cannot git fetch and sync with upstream, fixing any integration bugs; 2. Cannot pull in new library as dependency and do integration evaluations. Besides that, cannot apt install in the setup script is annoying (they blocked the domain to prevent apt install I believe). The agent itself is a bit meh, often opt-to git grep rather than reading all the source code to get contextual understanding (from what the UI has shown).
andrewmunsell 1y ago
> incorrect, its an o3 finetune. This is Open AI's fault (and literally every AI company is guilty of the same horrid naming schemes). Codex was an old model based on GPT-3, but then they reused the same name for both their Codex CLI and this Codex tool... I mean, just look at the updates to their own blog post, I can see why people are confused. https://openai.com/index/openai-codex/ https://openai.com/index/openai-codex/ Edit: Google just did it too. "Gemini Ultra" is both a model (https://deepmind.google/models/gemini/ultra/ https://deepmind.google/models/gemini/ultra/) and their new top-tier subscription plan (a la Open AI's Pro plan). Why is this so difficult?
- number6 1y ago
  They should use one of their LLMs to get some better naming schemes - seriously LLMs are pretty good at this set of task
  - IanCal 1y ago
    They absolutely cannot be worse than the humans involved here. gpt-4o followed by a series of o models so you have gpt-4o and o4? Wonderful.
- an_aparallel 1y ago
  Confusing people is the best way to get them to throw their hands up, stop thinking critically, and start paying. all businesses do this. Mega corps have resources to enforce clarity, but they dont because theyre stupid? Ill eat my words if thats the case....

by swyx 1y ago

canadiantim 1y ago

How do you find it compares to Claude Code?

viscanti 1y ago
It's much more conservative in the scope of task it will attempt and it's much slower. You need to fire and forget several parallel tasks because you'll be waiting 10+ minutes before you get anything you can review and give feedback on.
swyx 1y ago
right now apples and oranges literally only because 1) unlimited unmetered use and 2) not in browser so async and parallel. like that stuff just trumps actual model and agent harness differences because it removes all barriers from thought to code.

liuliu 1y ago

The particularly integration pain point to me is about network access, that prohibits several banal tasks to be offloaded to codex: 1. Cannot git fetch and sync with upstream, fixing any integration bugs; 2. Cannot pull in new library as dependency and do integration evaluations. Besides that, cannot apt install in the setup script is annoying (they blocked the domain to prevent apt install I believe). The agent itself is a bit meh, often opt-to git grep rather than reading all the source code to get contextual understanding (from what the UI has shown).

andrewmunsell 1y ago

> incorrect, its an o3 finetune. This is Open AI's fault (and literally every AI company is guilty of the same horrid naming schemes). Codex was an old model based on GPT-3, but then they reused the same name for both their Codex CLI and this Codex tool... I mean, just look at the updates to their own blog post, I can see why people are confused. https://openai.com/index/openai-codex/ https://openai.com/index/openai-codex/ Edit: Google just did it too. "Gemini Ultra" is both a model (https://deepmind.google/models/gemini/ultra/ https://deepmind.google/models/gemini/ultra/) and their new top-tier subscription plan (a la Open AI's Pro plan). Why is this so difficult?

number6 1y ago
They should use one of their LLMs to get some better naming schemes - seriously LLMs are pretty good at this set of task
- IanCal 1y ago
  They absolutely cannot be worse than the humans involved here. gpt-4o followed by a series of o models so you have gpt-4o and o4? Wonderful.
an_aparallel 1y ago
Confusing people is the best way to get them to throw their hands up, stop thinking critically, and start paying. all businesses do this. Mega corps have resources to enforce clarity, but they dont because theyre stupid? Ill eat my words if thats the case....