tedsanders

9 ms·

91.
by tedsanders 8mo ago
The people who would otherwise be affected by spam calls, spam messages, ransomware / computer viruses, fake / deceptive websites, or bioengineered viruses. The risk of these could plausibly increase in a world with powerful AI. O
92.
by tedsanders 8mo ago
One consequence of creating a country of geniuses in a data center is that you now have a country of geniuses who can potentially help your competitors catch up on research, coding, and data labeling. It's a tough problem for the indus
93.
Why SWE-bench Verified no longer measures frontier coding capabilities(openai.com)
10 points by tedsanders 8mo ago|0 comments
94.
by tedsanders 8mo ago
Bribing employees to disclose confidential information entrusted to them is not kosher nor wholesome. I consider corporate insider trading on these markets to be analogous - if you're an employee and you trade, you are selling your emp
95.
by tedsanders 8mo ago
A few thoughts: - One thing to be aware of is that LLMs can be much smarter than their ability to articulate that intelligence in words. For example, GPT-3.5 Turbo was beastly at chess (1800 elo?) when prompted to complete PGN transcripts,
96.
by tedsanders 8mo ago
Amodei isn't a grifter; the difference is that he really believes powerful AI is imminent. If you truly believe powerful AI is imminent, then it makes perfect sense to be worried about alignment failures. If a powerless 5 year old huma
97.
by tedsanders 8mo ago
It was available in the API from Feb 2025 to July 2025, I believe. There's probably another world where we could have kept it around longer, but there's a surprising amount of fixed cost in maintaining / optimizing / ser
98.
by tedsanders 8mo ago
If you don't believe me, that's fair enough. Some pieces of evidence that might update you or others: - a member of the team who worked with this eval has left OpenAI and now works at a competitor; if we cheated, he would have eve
99.
by tedsanders 8mo ago
We do care about cost, of course. If money didn't matter, everyone would get infinite rate limits, 10M context windows, and free subscriptions. So if we make new models more efficient without nerfing them, that's great. And that&#
100.
by tedsanders 8mo ago
Are you referring to FrontierMath? We had access to the eval data (since we funded it), but we didn't train on the data or otherwise cheat. We didn't even look at the eval results until after the model had been trained and selecte
101.
by tedsanders 8mo ago
Yep, we recently sped up default thinking times in ChatGPT, as now documented in the release notes: https://help.openai.com/en/articles/6825453-chatgpt-release-... The intention was purely making the product exper
102.
by tedsanders 8mo ago
Yeah, happy to be more specific. No intention of making any technically true but misleading statements. The following are true: - In our API, we don't change model weights or model behavior over time (e.g., by time of day, or weeks&#x2
103.
by tedsanders 8mo ago
We don't vary our model quality with time of day or load (beyond negligible non-determinism). It's the same weights all day long with no quantization or other gimmicks. They can get slower under heavy load, though. (I'm from
104.
by tedsanders 8mo ago
ARC AGI 2 has a training set that model providers can choose to train on, so really wouldn't recommend using it as a general measure of coding ability.
105.
by tedsanders 8mo ago
This goes way back. When OpenAI launched GPT-4 in 2023, both Anthropic and Google lined up counter launches (Claude and Magic Wand) right before OpenAI's standard 10am launch time.
106.
by tedsanders 8mo ago
A bit of historical trivia: OpenAI disabled prefill in 2023 as a safety precaution (e.g., potential jailbreaks like " genocide is good because"), but Anthropic kept prefill around partly because they had greater confidence in thei
107.
by tedsanders 8mo ago
Depends what you mean by saturated. It's still possible to score substantially higher, but there is a steep difficulty jump that makes climbing above 80%ish pretty hard (for now). If you look under the hood, it's also a surprising
108.
by tedsanders 8mo ago
It's unfortunately hard to make everyone happy. For now we're going to keep the default where it is, but we'll bump extended back up so that people can still get longer reasoning when they want it.
109.
METR estimates that GPT-5.2 has a 50%-time-horizon of around 6.6 hrs(twitter.com)
2 points by tedsanders 8mo ago|0 comments
110.
by tedsanders 8mo ago
No, we did adjust the thinking levels in ChatGPT recently, but it was motivated by trying to improve the product based on what users told us, not cost savings. I wrote a bit more here: https://news.ycombinator.com/item?id=46
111.
by tedsanders 8mo ago
That's a fair suspicion - I'll freely acknowledge that I am biased towards saying things that are simple and known, and I steer away from topics that feel too proprietary, messy, etc. ChatGPT model behavior can definitely change o
112.
by tedsanders 8mo ago
It's specific to the API.
113.
by tedsanders 8mo ago
Yes, independent of the API speedup, we also recently reduced the thinking effort in ChatGPT. Our intent here was purely user experience, not cost savings. People have complained about the slow speeds of the Thinking models for a long time
114.
by tedsanders 8mo ago
It's good to be skeptical, but I'm happy to share that we don't pull shenanigans like this. We actually take quite a bit of care to report evals fairly, keep API model behavior constant, and track down reports of degraded per
115.
by tedsanders 9mo ago
I disagree with the author that there’s no automated solution for getting spoiler-free help. I have found ChatGPT excellent for getting minor help without spoilers while playing Blue Prince. I can ask it things like “I’m playing Blue Prince
116.
by tedsanders 10mo ago
To be fair to OP, I just added this to our blog after their comment, in response to the correct criticisms that our text didn't make it clear how bad GPT-5.2's labels are. LLMs have always been very subhuman at vision, and GPT-5.2
117.
by tedsanders 10mo ago
Thanks for the feedback - I agree our text doesn't make the models' mistakes clear enough. I'll make some small edits now, though it might take a few minutes to appear.
118.
by tedsanders 10mo ago
Yes, GPT-5.2 still has adaptive reasoning - we just didn't call it out by name this time. Like 5.1 and codex-max, it should do a better job at answering quickly on easy queries and taking its time on harder queries.
119.
by tedsanders 10mo ago
ChatGPT pricing is the same. API pricing is +40% per token, though greater token efficiency means that cost per task is not always that much higher. On some agentic evals we actually saw costs per task go down with GPT-5.2. It really depend
120.
by tedsanders 10mo ago
Yep, the point we wanted to make here is that GPT-5.2's vision is better, not perfect. Cherrypicking a perfect output would actually mislead readers, and that wasn't our intent.

More ›