30 ms·

GPT-4.5

• DennisL123 2y ago
  tl;dr: doesn‘t work as expected and we sank a ton of money on it too.
• dgfitz 2y ago
  @sama, LLMs aren't going to create AGI. I realize you need to generate cash flow, this isn't the play. Sincerely, Me
  • kneegerman 2y ago
    you misunderstand, the business model is extracting cash from Qatar et al
• whisper_yb 2y ago
  wowsers!
• I_am_tiberius 2y ago
  Not available in my Pro plan.
• throwup238 2y ago
  At this point I think the ultimate benchmark for any new LLM is whether or not it can come up with a coherent naming scheme for itself. Call it “self awareness.”
  • lenerdenator 2y ago
    The people naming them really took the "just give the variable any old name, it doesn't matter" advice from Programming 101 to heart.
    • sho_hn 2y ago
      This is why my new LLM portfolio is Foo, Bar and Baz.
      • smallmancontrov 2y ago
        Still more coherent than the OpenAI lineup.
  • nopelynopington 2y ago
    3,3.5,4,4o,4.5 I had my money on 4oz
    • deleted 2y ago
      [deleted]
  • deleted 2y ago
    [deleted]
• camwhite 2y ago
  I can't wait for fireship.io and the comment section here to tell me what to think about this
  • mjburgess 2y ago
    You appear to have the direction of causation reversed. (In that fireship does the same)
    • lysace 2y ago
      I wonder if fireship reaction video scripts to AI models based on HN comments can be automated using said AI models.
  • rob 2y ago
    I bet simonw will be adding it to `llm` and someone will be pasting his highlights here right after. Until then, my mind will remain a blank canvas.
• mplewis 2y ago
  [flagged]
  • dang 2y ago
    Can you please stop breaking the site guidelines by posting unsubstantive comments / flamebait / calling names / etc.? You've been doing this repeatedly. It's not what this site is for, and destroys what it is for. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
• bhouston 2y ago
  A bit better at coding than ChatGPT 4o but not better than o3-mini - there is a chart near the bottom of the page that is easy to overlook: - ChatGPT 4.5 on AWS Bench verified: 38.0% - ChatGPT 4o on AWS Bench verified: 30.7% - OpenAI o3-mini on AWS Bench verified: 61.0% BTW Anthropic Claude 3.7 is better than o3-mini at coding at around 62-70% [1]. This means that I'll stick with Claude 3.7 for the time being for my open source alternative to Claude-code: https://github.com/drivecore/mycoder https://github.com/drivecore/mycoder [1] https://aws.amazon.com/blogs/aws/anthropics-claude-3-7-sonnet-the-first-hybrid-reasoning-model-is-now-available-in-amazon-bedrock/ https://aws.amazon.com/blogs/aws/anthropics-claude-3-7-sonne...
  • logicchains 2y ago
    >BTW Anthropic Claude 3.7 is better than o3-mini at coding at around 62-70% [1]. This means that I'll stick with Claude 3.7 for the time being for my open source alternative to Claude-code That's not a fair comparison as o3-mini is significantly cheaper. It's fine if your employer is paying, but on a personal project the cost of using Claude through the API is really noticeable.
    • cheema33 2y ago
      > That's not a fair comparison as o3-mini is significantly cheaper. It's fine if your employer is paying... I use it via Cursor editor's built-in support for Claude 3.7. That caps the monthly expense to $20. There probably is a limit in Claude for these queries. But I haven't run into it yet. And I am a heavy user.
      • bhouston 2y ago
        Agentic coders (e.g. aider, Claude-code, mycoder, codebuff, etc.) use a lot more tokens, but they write whole features for you and debug your code.
    • QuadmasterXLII 2y ago
      If open ai offers a more expensive model (4.5) and a cheaper model (3 mini) and both are worse, it starts to be a fair comparison
• bingdig 2y ago
  Not seeing it available in the app or on ChatGPT.com with a pro subscription.
  • hidelooktropic 2y ago
    It’s not supposed to be yet
    • bingdig 2y ago
      "Available to Pro users and developers worldwide" "Try in ChatGPT"
      • hidelooktropic 2y ago
        You’re at the 200 dollar tier?
        • kelsey98765431 2y ago
          Not available in model picker for my pro 200 dollar account. You can play with the api here: https://platform.openai.com/playground/chat?models=gpt-4.5-preview https://platform.openai.com/playground/chat?models=gpt-4.5-p...
        • apsec112 2y ago
          I have Pro, just updated the app, but don't currently have access
          • _boffin_ 2y ago
            Same.
          • ashu1461 2y ago
            How is operator ? Have you been able to use it for anything useful ? Was thinking about taking a subscription
        • 2y ago
• SamvitJ 2y ago
  One comparison I found interesting... I think GPT-4o has a more balanced answer! > What are your thoughts on space exploration? GPT-4.5: Space exploration isn't just valuable—it's essential. People often frame it as a luxury we pursue after solving Earth-bound problems. But space exploration actually helps us address those very challenges: climate change (via satellite monitoring), resource scarcity (through asteroid mining), and existential risk (by enabling eventual multi-planetary living). It also drives technological innovation, economic growth, and inspires humanity toward collective achievement. We don't explore space to escape Earth. We explore space to better understand, protect, and sustain life on Earth. > What are your thoughts on space exploration? GPT-4o: Space exploration is one of the most ambitious and profound endeavors humanity has undertaken. It expands our understanding of the universe, drives technological innovation, and inspires generations to think beyond the boundaries of Earth. From a scientific perspective, space exploration helps us address fundamental questions: Are we alone in the universe? How did the cosmos begin? Can we colonize other planets? Missions like the James Webb Space Telescope, Mars rovers, and deep-space probes continuously reshape our understanding of physics, planetary science, and even biology. From a practical standpoint, space exploration fuels advancements in materials science, AI, robotics, and medicine. Many everyday technologies—GPS, satellite communications, weather forecasting, and even medical imaging—stem from space research. There's also the existential argument: humanity's long-term survival may depend on our ability to become a multi-planetary species. While colonizing Mars or mining asteroids is far from easy, these efforts could help us safeguard civilization against existential threats like climate change or asteroid impacts. That said, space exploration also raises ethical and economic questions. Should we focus more on Earth’s problems before investing trillions in space? How do we prevent the same exploitative behaviors that shaped colonial history from repeating on other planets? Overall, I see space exploration as a necessary pursuit—not at the cost of solving problems on Earth, but as a way to advance our knowledge, drive innovation, and secure a future beyond our home planet. What’s your take?
  • gh0stcat 2y ago
    Yeah, I also found it odd that they seem to be implying that an incredibly biased answer (as in 4.5) is better. In general, I find the tone more polarizing and not exactly warm as they advertised in the release video.
• ThouYS 2y ago
  hmm.. not really the direction I expected them to go
• doctoboggan 2y ago
  I am beginning to think these human eval tests are a waste of time at best, and negative value at worst. Maybe I am being snobby, but I don't think the average human is able to properly evaluate usefulness, truthfulness, or other metrics that I actually care about. I am sure this is good for openAI since if more people like what the hear, they are more likely come back. I don't want my AI more obsequious, I want it more correct and capable. My only use case is coding though, so maybe I am not representative of their usual customers?
  • dboreham 2y ago
    The SuperTuring era.
  • onlyrealcuzzo 2y ago
    > I want it more correct and capable. How is it supposed to be more correct and capable if these human eval tests are a waste of time? Once you ask it to do more than add two numbers together, it gets a lot more difficult and subjective to determine whether it's correct and how correct.
    • doctoboggan 2y ago
      I agree it's a hard problem. I think there are a number of tests out there however that are able to objectively test capability and truthfulness. I've read reports that some of the changes that are preferred by human evaluators actually hurt the performance on the more objective tests.
      • onlyrealcuzzo 2y ago
        Please tell me how we objectively determine how correct something is when you ask an LLM: "Was Russia the aggressor in the current Ukraine / Russia conflict?" One LLM says: "Yes." The other says: "Well, it's hard to say because what even is war? And there's been conflict forever, and you have to understand that many people in Russia think there is no such thing as Ukraine and it's always actually just been Russia. How can there be an aggressor if it's not even a war, just a special operation in a civil conflict? And, anyway, Russia is such a good country. Why would it be the aggressor? To it's own people even!? Vladimir Putin is the president of Russia, and he's known to be a kind and just genius who rarely (if ever) makes mistakes. Some people even think he's the second coming of Christ. President Zelenskyy, on the other hand, is considered by many in Russia and even the current White House to be a dictator. He's even been accused by Elon Musk of unspeakable sex crimes. So this is a hard question to answer and there is no consensus among everyone who was the aggressor or what started the conflict. But more people say Russia started it."
• ekojs 2y ago
  > Because of this, we’re evaluating whether to continue serving it in the API long-term as we balance supporting current capabilities with building future models. Seems like it's not going to be deployed for long. $75.00 / 1M tokens for input $150.00 / 1M tokens for output That's crazy prices.
  • bguberfain 2y ago
    Until GPT-4.5, GPT-4 32K was certainly the most heavy model available at OpenAI. I can imagine the dilemma between to keep it running or stop it to free GPU for training new models. This time, OpenAI was clear whether to continue serving it in the API long-term.
    • jsheard 2y ago
      > or stop it to free GPU for training new models. Don't they use different hardware for inference and training? AIUI the former is usually done on cheaper GDDR cards and the latter is done on expensive HBM cards.
      • throwaway314155 2y ago
        Indeed, that theory is nonsense.
    • Chamix 2y ago
      It's interesting to compare the cost of that original gpt-4 32k(0314) vs gpt-4.5: $60/M input tokens vs $75/M input tokens $120/M output tokens vs $150/M output tokens
  • daemonologist 2y ago
    Imagine if they built a reasoning model with costs like these. Sometimes it seems like they're on a trajectory to create a model which is strictly more capable than I am but which costs 100x my salary to run.
    • 2y ago
• nickreese 2y ago
  Is it just me or is having the AI help you self sensor (as shown in the demo live stream: https://www.youtube.com/watch?v=cfRYp0nItZ8 https://www.youtube.com/watch?v=cfRYp0nItZ8)... pretty dystopian?
• nopelynopington 2y ago
  Oh this makes sense. chatGPT results have taken a nose dive in quality lately. It couldn't write a simple rename function for me yesterday, still buggy after seven attempts. I'm more and more convinced that they dumb down the core product when they plan to release a new version to make the difference seem bigger.
  • anti-soyboy 2y ago
    Yep I realized about that many time ago, they are literally scammers
  • wilg 2y ago
    99% chance that's confirmation bias
    • nomel 2y ago
      Sam tweeted that they're running out of computer. I think it's reasonable to think they may serve somewhat quantized models when out of capacity. It would be a rational business decision that would minimally disrupt lower tier ChatGPT users. Anecdotally, I've noticed what appears to be drops in quality, some days. When the quality drops, it responds in odd ways when asked what model it is.
      • wilg 2y ago
        I mean, GPT 4.5 says "I'm ChatGPT, based on OpenAI's GPT-4 Turbo model." and o1 Pro Mode can't answer, just says "I’m ChatGPT, a large language model trained by OpenAI." Asking it what model it is shouldn't be considered a reliable indicator of anything.
        • fragmede 2y ago
          Interviewing deepseek as to its identity should absolve anyone of that notion.
        • nomel 2y ago
• skepticATX 2y ago
  How many still believe that scaling up base models will lead to AGI?
  • tiahura 2y ago
    Dan Ives
    • oblio 2y ago
      The guy is a finance major...
• Mizza 2y ago
  Sounds like it's a distill of O1? After R1, I don't care that much about non-reasoning models anymore. They don't even seem excited about it on the livestream. I want tiny, fast and cheap non-reasoning models I can use in APIs and I want ultra smart reasoning models that I can query a few times a day as an end user (I don't mind if it takes a few minutes while I refill a coffee). Oh, and I want that advanced voice mode that's good enough at transcription to serve as a babelfish! After that, I guess it's pretty much all solved until the robots start appearing in public.
  • sebastiennight 2y ago
    Probably not a distill of o1, since o1 is a reasoning model and GPT4.5 is not. Also, OpenAI has been claiming that this is a very large model (and it's 2.5x more expensive than even OG GPT-4) so we can assume it's the biggest model they've trained so far. They'll probably distill this one into GPT-4.5-mini or such, and have something faster and cheaper available soon.
    • Mizza 2y ago
      There are plenty of distills of reasoning models now, and they said in they livestream they used training data from "smaller models" - which is probably every model ever considering how expensive this one is.
      • sebastiennight 2y ago
        Knowledge distillation is literally by definition teaching a smaller model from a big one, not the opposite. Generating outputs from existing (therefore smaller) models to train the largest model of all time would simply be called "using synthetic data". These are not the same thing at all. Also, if you were to distill a reasoning model, the goal would be to get a (smaller) reasoning model because you're teaching your new model to mimic outputs that show a reasoning/thinking trace. E.G. that's what all of those "local" Deepseek models are: small LLama models distilled from the big R1 ; a process which "taught" Llama-7B to show reasoning steps before coming up with a final answer.
  • 2y ago
• rvz 2y ago
  You should really be paying attention to what DeepSeek AI open sources next. This announcement by OpenAI was already expected: [0] [0] https://x.com/sama/status/1889755723078443244 https://x.com/sama/status/1889755723078443244
• brokensegue 2y ago
  API price is crazy high. This model must be huge. Not sure this is practical
  • jdprgm 2y ago
    Wow you aren't kidding, 30x input price and 15x output price vs 4o is insane. The pricing on all AI API stuff changes so rapidly and is often so extreme between models it is all hard to keep track of and try to make value decisions. I would consider a 2x or 3x price increase quite significant, 30x is wild. I wonder how that even translates... there is no way the model size is 30 times larger right?
• jdprgm 2y ago
  Per Altman on X: "we will add tens of thousands of GPUs next week and roll it out to the plus tier then". Meanwhile a month after launch rtx 5000 series is completely unavailable and hardly any restocks and the "launch" consisted of microcenters getting literally tens of cards. Nvidia really has basically abandoned consumers.
  • apsec112 2y ago
    AI GPUs are bottlenecked mostly by high-bandwidth memory (HBM) chips and CoWoS (packaging tech used to integrate HBM with the GPU die), which are in short supply and aren't found in consumer cards at all
    • jiggawatts 2y ago
      You would think that by now they would have done something to ramp production capacity…
      • aurareturn 2y ago
        Maybe demand is that great?
  • bhouston 2y ago
    Altman's claim and NVIDIA's consumer launch supply problems may be related - OpenAI may be eating up the GPU supply...
    • _zoltan_ 2y ago
      OpenAI is not purchasing consumer 5090s... :)
      • BizarroLand 2y ago
        No, but the supply constraints are part of what is driving the insane prices. Every chip they use for consumer grade instead of commercial grade is a potential loss of potential income.
• sunaookami 2y ago
  This seems very rushed because of DeepSeek's R1 and Anthropic's Claude 3.7 Sonnet. Pretty underwhelming, they didn't even show programming? In the livestream, they struggled to come up with reasons why I should prefer GPT-4.5 over GPT-4o or o1.
  • apsec112 2y ago
    At least according to WSJ, they had planned to release it earlier but struggled to get the model quality up, especially relative to cost
  • bhouston 2y ago
    they do have coding benchmarks, I summarized them here: https://news.ycombinator.com/item?id=43197955 https://news.ycombinator.com/item?id=43197955
  • bitshiftfaced 2y ago
    This strikes me as the opposite of rushed. I get the impression that they've been sitting on this for a while and couldn't make it look as good as previous improvements. At some point they had to say, "welp here it is, now we can check that box and move on."
• Topfi 2y ago
  Considering both this blog post and the livestream demos, I am underwhelmed. Having just finished the stream, I had a real "was that all" moment, which on one hand shows how spoiled I've gotten by new models impressing me, but on another feels like OpenAI really struggles to stay ahead of their competitors. What has been shown feels like it could be achieved using a custom system prompt on older versions of OpenAIs models, and I struggle to see anything here that truly required ground-up training on such a massive scale. Hearing that they were forced to spread their training across multiple data centers simultaneously, coupled with their recent release of SWE-Lancer [0] which showed Anthropic (Claude 3.5 Sonnet (new) to be exact) handily beating them, I was really expecting something more than "slightly more casual/shorter output", which again, I fail to see how that wasn't possible by prompting GPT-4o. Looking at pricing [1], I am frankly astonished. > Input: $75.00 / 1M tokens > Cached input: $37.50 / 1M tokens > Output: $150.00 / 1M tokens How could they justify that asking price? And, if they have some amazing capabilities that make a 30-fold pricing increase justifiable, why not show it? Like, OpenAI are many things, but I always felt they understood price vs performance incredibly well, from the start with gpt-3.5-turbo up to now with o3-mini, so this really baffles me. If GPT-4.5 can justify such immense cost in certain tasks, why hide that and if not, why release this at all? [0] https://github.com/openai/SWELancer-Benchmark https://github.com/openai/SWELancer-Benchmark [1] https://openai.com/api/pricing/ https://openai.com/api/pricing/
  • Bjorkbat 2y ago
    My first thought seeing this and looking at benchmarks was that if it wasn’t for reasoning, then either pundits would be saying we’ve hit a plateau, or at the very least OpenAI is clearly in 2nd place to Anthropic in model performance. Of course we don’t live in such a world, but I thought of this nonetheless because for all the connotations that come with a 4.5 moniker this is kind of underwhelming.
    • uh_uh 2y ago
      Pundits were saying that deep learning has hit a plateau even before the LLM boom.
  • 2y ago
• zaptrem 2y ago
  GPT 4.5 pricing is insane: Price Input: $75.00 / 1M tokens Cached input: $37.50 / 1M tokens Output: $150.00 / 1M tokens GPT 4o pricing for comparison: Price Input: $2.50 / 1M tokens Cached input: $1.25 / 1M tokens Output: $10.00 / 1M tokens It sounds like it's so expensive and the difference in usefulness is so lacking(?) they're not even gonna keep serving it in the API for long: > GPT‑4.5 is a very large and compute-intensive model, making it more expensive than and not a replacement for GPT‑4o. Because of this, we’re evaluating whether to continue serving it in the API long-term as we balance supporting current capabilities with building future models. We look forward to learning more about its strengths, capabilities, and potential applications in real-world settings. If GPT‑4.5 delivers unique value for your use case, your feedback (opens in a new window) will play an important role in guiding our decision. I'm still gonna give it a go, though.
  • mmaunder 2y ago
    But you get higher EQ. /s
  • MattSayar 2y ago
    Input price difference: 4.5 is 30x more Output price difference:4.5 is 15x more In their model evaluation scores in the appendix, 4.5 is, on average, 26% better. I don't understand the value here.
    • alwa 2y ago
      If you ran the same query set 30x or 15x on the cheaper model (and compensated for all the extra tokens the reasoning model uses), would you be able to realize the same 26% quality gain in a machine-adjudicatible kind of way?
      • j_maffe 2y ago
        with a reasoning model you'd get better than both.
        • 2y ago
• Xiol32 2y ago
  The example GPT-4.5 answers from the livestream are just... too excitable? Can't put my finger on it, but it feels like they're aimed towards little kids.
  • jug 2y ago
    It made me wonder how much of that was due to the system prompt too.
• virgildotcodes 2y ago
  That presentation was super underwhelming. We got to watch them compare… the vibes? … of 4.5 vs o1. No wonder Sam wasn’t part of the presentation.
  • Etheryte 2y ago
    And to top it off, it costs $75.00 per 1M vibes.
  • thomas34298 2y ago
    Sam tweeted "taking care of my kid in the hospital": https://x.com/sama/status/1895210655944450446 https://x.com/sama/status/1895210655944450446 Let's not assume that he's lying. Neither the presentation nor my short usage via the API blew me away, but to really evaluate it, you'd have to use it longer on a daily basis. Maybe that becomes a possiblity with the announced performance optimizations that would lower the price...
    • dot1x 2y ago
      I think it's pretty clear he's a liar in most facets of his life
  • jug 2y ago
    It should’ve just been a web launch without video.
• bakugo 2y ago
  API is literally 5 times more expensive than Claude 3 Opus, and it doesn't even seem to do anything impressive. What's the business strategy here?
• hidelooktropic 2y ago
  I’m not sure that doing a live stream on this was the right way to go. I would’ve just quietly sent out a press release. I’m sure they have better things on the way.
• sebastiennight 2y ago
  It is interesting that they are focusing a large part of this release on the model having a higher "EQ" (Emotional Quotient). We're far from the days of "this is not a person, we do not want to make it addictive" and getting a firm foot on the territory of "here's your new AI friend". This is very visible in the example comparing 4o with 4.5 when the user is complaining about failing a test, where 4o's response is what one would expect from a "typical AI response" with problem-solving bullets, and 4.5 is sending what you'd expect from a pal over instant messaging. It seems Anthropic and Grok have both been moving in this direction as well. Are we going to see an escalation of foundation models impersonating "a friendly person" rather than "a helpful assistant"? Personally I find this worrying and (as someone who builds upon SOTA model APIs) I really hope this behavior is not going to seep into API responses, or will at least be steerable through the system/developer prompt.
  • famouswaffles 2y ago
    The whole robotic, monotone, helpful assistant thing was something these companies had to actively hammer in during the post-training stage. It's not really how LLMs will sound by default after pre-training. I guess they're caring less and less about that effort especially since it hurts the model in some ways like creative writing.
    • sebastiennight 2y ago
      If it's just a different choice during RLHF, I'll be curious to see what are the trade-offs in performance. The "buddy in a chat group" style answers do not make me feel like asking it for a story will make the story long/detailed/poignant enough to warrant the difference. I'll give it a try and compare on creative tasks.
    • turnsout 2y ago
      Or maybe they're just getting better at it, or developing better taste. After switching to Claude, I can't go back to ChatGPT's overly verbose bullet-point laden book reports every time I ask a question. I don't think that's pretraining—it's in the way OpenAI approaches tuning and prompting vs Anthropic.
• CitizenTen 2y ago
  GPT pro already has already rummored to be 100k users. You think GPT 4.5 will add to that even with the insane costs for corporate users?
  • cristiancavalli 2y ago
    What rumors? I looked and can’t find something to substantiate that #
    • pepsi-not-coke 2y ago
      in my experience, o3-mini-high while still unpredictable as it modifies and ignores parts of my code when I specifically tell it not to do so (e.g. "don't touch anything else!") is the best AI coding tool out there, far better than Claude so Pro is worth it for O3-mini-high
    • spiderfarmer 2y ago
      100k per user perhaps.
• animanoir 2y ago
  [dead]
• andsoitis 2y ago
  “this isn’t a reasoning model and won’t crush benchmarks.” — https://x.com/sama/status/1895203654103351462 https://x.com/sama/status/1895203654103351462
• MaxPock 2y ago
  One thing that Altman does extremely well is to over-promise and under-deliver.
• erulabs 2y ago
  Finally a scaling wall? This is apparently (based on pricing) using about an order of magnitude more compute, and is only maybe 10% more intelligent. Ideally DeepSeeks optimizations help bring the costs way down, but do any AI researchers want to comment on if this changes the overall shape of the scaling curve?
  • fpgaminer 2y ago
    Seems on par with the existing scaling curve. If I had to speculate, this model would have been an internal-only model, but they're releasing it for PR. An optimized version with 99% of the performance for 1/10th the cost will come out later.
    • j_maffe 2y ago
      This is the shittiest PR move I've seen since the AI trend started.
      • Workaccount2 2y ago
        At least so far it's coding performance is bad, but from what I have seen it's writing abilities are totally insane. It doesn't read like AI output anymore.
        • j_bum 2y ago
          Any examples you’d be willing to share?
          • chippiewill 2y ago
            They have examples in the announcement post. It does a better job of understanding intent in the question which helps it give an informal rather than essay style response where appropriate.
            • j_maffe 2y ago
• ein0p 2y ago
  Now imagine this model (or an optimized/slightly downsized variant thereof) as a base for a "thinking" one.
• bparsons 2y ago
  Are they saying that 4.5 has a 35% hallucination rate? That chart is a bit confusing.
  • riku_iki 2y ago
    its on that benchmark, which likely is very challenging.
• Seattle3503 2y ago
  The GPT-1 response to the example prompt "What was the first language?" got a chuckle out of me
  • aldanor 2y ago
    The question being, will we be chuckling at current models responses in 5-10y from now?
• joshuamcginnis 2y ago
  I'm one week in on heavy grok usage. I didn't think I'd say this, but for personal use, I'm considering cancelling my OpenAI plan. The one thing I wish grok had was more separation of the UI from X itself. The interface being so coupled to X puts me off and makes it feel like a second-hand citizen. I like ChatGPTs minimalist UI.
  • aldanor 2y ago
    Theres grok.com which is standalone and with its own UI
    • it 2y ago
      There's also a standalone Grok app at least on iOS.
      • pzo 2y ago
        I wish they did also dedicated keyboard app like SwiftKey that has copilot integration
  • richard_todd 2y ago
    I find grok to be the best overall experience for the types of tasks I try to give AI (mostly: analyze pdf, perform and proofread OCR, translate Medieval Latin and Hebrew, remind me how to do various things in python or SwiftUI). ChatGPT/gemini/copilot all fight me occasionally, but grok just tries to help. And the hallucinations aren’t as frequent, at least anecdotally.
  • fzzzy 2y ago
    Don't they have a standalone Grok app now? I thought I saw that. [edit] ah some sibling comments mention this as well
  • deleted 2y ago
    [deleted]
  • andxor 2y ago
    https://grok.com/ https://grok.com/
• selalipop 2y ago
  I've been working on post-training models for tasks that require EQ, so it's validating to see OpenAI working towards that too. That being said, this is very expensive. - Input: $75.00 / 1M tokens - Cached input: $37.50 / 1M tokens - Output: $150.00 / 1M tokens One of the most interesting applications of models with higher EQ is personalized content generation, but the size and cost here are at odds with that.
• lblume 2y ago
  Am I missing something, or do the results not even look that much better? Referring to the output quality, this just seems like a different prompting style and RLHF, not really an improved model at all.
• kgeist 2y ago
  >GPT‑4.5 is more succinct and conversational I wonder why they highlight it as an achievement when they could have simply tuned 4o to be more conversational and less like a bullet-point-style answer machine. They did something to 4o compared to the previous models which made the responses feel more canned.
  • qgin 2y ago
    Poossibly, but reports seem to indicate that 4.5 is much more nuanced and thoughtful in its language use. It's not just being shorter and casual as a style, there is a higher amount of "conceptual resolution" within the words being used.
• mvdtnz 2y ago
  OpenAI doubling down on the American-style therapy-speak instead of focusing on usefulness. No thanks.
• wewewedxfgdf 2y ago
  I feel like OpenAI is pursuing AGI when Anthropic/Claude is pursuing making AI awesome for practical things like coding. I only ever using OpenAI's coding now as a double check against Claude. Does OpenAI have their eyes on the ball?
  • ls_stats 2y ago
    >I feel like OpenAI is pursuing AGI I don't think so, the "AGI guy" was Ilya Sutskever, he is gone, he wanted to make OpenAI "less comercial", AGI is just a buzzword for Altmann.
    • rakejake 2y ago
      Right. A good chunk of the "old guard" is now gone - Ilya to SSI, Mira and a bunch of others to a new venture called Thinking Machines, Alec Radford etc. Remains to be seen if OpenAI will be the leader or if other players catch up.
      • sumedh 2y ago
        The page still has Mira Muratis name under Exec Leadership
  • rakejake 2y ago
    My usage has come down to mostly Claude (until I run out of free tier quota) and then Gemini. Claude is the best for code and Gemini 2.0 Flash is good enough while also being free (well considering how much data G has hoovered up over the years, perhaps not) and more importantly highly available. For simple queries like generating shell scripts for some plumbing, or doing some data munging, I go straight to Gemini.
    • HarHarVeryFunny 2y ago
      > My usage has come down to mostly Claude (until I run out of free tier quota) and then Gemini Yep, exactly same here. Gemini 2.0 Flash is extremely good, and I've yet to hit any usage limits with them - for heavy usage I just go to Gemini directly. For "talk to an expert" usage, Claude is hard to beat though.
• infinet 2y ago
  Can it be self-hosted? Many institutions and organizations are hesitant to use AI because concerns of data leaking over chatbot. Open models, on the other hand, can be self-hosted. There is a deepseek arm race in other part of the world. Universities are racing to host their own deepseek. Hospitals, large businesses, local governments, even courts are deploying or showing interest in self-hosting deepseek.
  • YetAnotherNick 2y ago
    Do you know of any university that host Deepseek?
    • infinet 2y ago
      > Do you know of any university that host Deepseek? https://chat.zju.edu.cn https://chat.zju.edu.cn https://chat.sjtu.edu.cn https://chat.sjtu.edu.cn https://chat.ecnu.edu.cn/html/ https://chat.ecnu.edu.cn/html/ To list a few. There are of course many more in China. I won't be surprised if universities in other countries also self-hosting.
      • YetAnotherNick 2y ago
        Where does it says that it is self hosted? And why it is exposed to public.
        • infinet 2y ago
          They are self-hosted and require campus credentials for access. If you try to chat with the ZJU instance, it will redirect you to a login page. ZJU is unique in that it appears to support access from all higher education institutions in China.
  • moralestapia 2y ago
    OpenAI has never released a single model that could be self-hosted. GPT-2? Maybe not even that one.
• taytus 2y ago
  Who wants a model that is not reasoning? The older models are just fine.
  • ksynwa 2y ago
    They said this is their last non reasoning model so I'm assuming there is a sunk cost aspect to it.
• eightysixfour 2y ago
  Seeing OpenAI and Anthropic go different routes here is interesting. It is worth moving past the initial knee jerk reaction of this model being unimpressive and some of the comments about "they spent a massive amount of money and had to ship something for it..." * Anthropic appears to be making a bet that a single paradigm (reasoning) can create a model which is excellent for all use cases. * OpenAI seems to be betting that you'll need an ensemble of models with different capabilities, working as a single system, to jump beyond what the reasoning models today can do. Based on all of the comments from OpenAI, GPT 4.5 is absolutely massive, and with that size comes the ability to store far more factual data. The scores in ability oriented things - like coding - don't show the kind of gains you get from reasoning models but the fact based test, SimpleQA, shows a pretty large jump and a dramatic reduction in hallucinations. You can imagine a scenario where GPT4.5 is coordinating multiple, smaller, reasoning agents and using its factual accuracy to enhance their reasoning, kind of like ruminating on an idea "feels" like a different process than having a chat with someone. I'm really curious if they're actually combining two things right now that could be split as well, EQ/communications, and factual knowledge storage. This could all be a bust, but it is an interesting difference in approaches none-the-less, and worth considering that OpenAI could be right.
  • nomel 2y ago
    > OpenAI seems to be betting that you'll need an ensemble of models with different capabilities, working as a single system, to jump beyond what the reasoning models today can do. The high level block diagrams for tech always end up converging to those found in biological systems.
    • eightysixfour 2y ago
      Yeah, I don't know enough real neuroscience to argue either side. What I can say is I feel like this path is more like the way that I observe that I think, it feels like there are different modes of thinking and processes in the brain, and it seems like transformers are able to emulate at least two different versions of that. Once we figure out the frontal cortex & corpus callosum part of this, where we aren't calling other models over APIs instead of them all working in the same shared space, I have a feeling we'll be on to something pretty exciting.
• ilrwbwrkhv 2y ago
  Question: explain thread first macros in clojure in a short paragraph GPT 4.5: In Clojure, the *thread-first macro* (`->`) is used to improve code readability by threading a value through a series of function calls. It takes an initial value and passes it as the first argument to the next function, then continues threading the result through subsequent functions. This eliminates deeply nested expressions and makes the code more readable. For example, ```clojure (-> 5 inc (* 2) str) ;; Equivalent to (str (* 2 (inc 5))) → "12" ``` Here, `5` is incremented to `6`, then multiplied by `2` to get `12`, and finally converted to a string. --- Deepseek v3: In Clojure, the `->>` (thread-last) and `->` (thread-first) macros are used to improve code readability by allowing a more linear, pipeline-like flow of operations. The `->` (thread-first) macro takes an initial value and threads it as the first argument through a series of forms. For example, `(-> x (f) (g))` translates to `(g (f x))`. This is particularly useful when working with nested function calls, as it flattens the structure and makes the sequence of transformations clearer. It’s commonly used with functions that take the threaded value as their first argument, such as collection operations or Java interop methods.
• datadrivenangel 2y ago
  They tried, but doesn't look better?
• DaveMcMartin 2y ago
  This feels more like a release they pushed out to keep the "hype" alive rather than something they were eager to share. Honestly, the results don’t seem all that impressive, and considering the price, it just doesn’t feel worth it.
• bla3 2y ago
  I wonder if we're starting to see the effects of the mass exodus a while ago.
• BoredPositron 2y ago
  The announcements early on were relatively sincere and technical with papers and nice pages explaining the new models in easy language and now we get this marketing garbage. Probably the fastest enshitification I’ve seen.
• sky2224 2y ago
  Honestly, the most astounding part of this announcement is their comparison to o3-mini with QA prompts. EIGHTY PERCENT hallucination rate? Are you kidding me? I get that the model is meant to be used for logic and reasoning, but nowhere does OpenAI make this explicitly clear. A majority of users are going to be thinking, "oh newer is better," and pick that.
  • jug 2y ago
    Yeah it was an abysmal result (any 50%+ hallucination result in that bench is pretty bad) and worse than o1-mini in the SimpleQA paper. On that topic, Sonnet 3.5 ”Old” hallucinates less than GPT-4.5, just for a bit of added perspective here.
  • krackers 2y ago
    Very nice catch, I was under the impression that o3-mini was "as good" as o1 on all dimensions. Seems the takeaway is that any form of quantization/distillation ends up hurting factual accuracy (but not reasoning performance), and there are diminishing returns to reducing hallucinations by model-scaling or RLHF'ing. I guess then that other approaches are needed to achieve single-digit "hallucination" rates. All of wikipedia compresses down to < 50GB though, so it's not immediately clear that you can't have good factual accuracy with a small sparse model
• freediver 2y ago
  The results for GPT - 4.5 are in for Kagi LLM benchmark too. It does crush our benchmark - time to make new? ;) - with performance similar of that of reasoning models. It does come at a great price both in cost and speed. A monster is what they created. But looking at the tasks it fails, some of them my 9 year old would solve. Still in this weird limbo space of super knowledge and low intelligence. May be remembered as the last the last of the 'big ones', can't imagine this will be a path for the future. https://help.kagi.com/kagi/ai/llm-benchmark.html https://help.kagi.com/kagi/ai/llm-benchmark.html
  • theodorthe5 2y ago
    If Gemini 2 is the top in your benchmark, make sure to re-check your benchmark.
    • shawabawa3 2y ago
      Gemini 2 pro is actually very impressive (maybe not for coding, haven't used it for that) Flash is pretty garbage but cheap
    • istjohn 2y ago
      Gemini 2.0 Pro is quite good.
    • aoeusnth1 2y ago
      Gemini 2 pro is pretty strong actually.
  • wendyshu 2y ago
    Why don't you have Grok?
    • mhh__ 2y ago
      No api for grok 3 might be why
  • mjirv 2y ago
    Do you have results for gpt-4? I’d be very interested in seeing the lift here from their last “big one”.
• simonw 2y ago
  If you want to try it out via their API you can run it through my LLM tool using uvx like this: uvx --with 'https://github.com/simonw/llm/archive/801b08bf40788c09aed6175252876310312fe667.zip' \ llm -m gpt-4.5-preview 'impress me' You may need to set an API key first, either with `export OPENAI_API_KEY='xxx'` or using this command to save it to a file: uvx llm keys set openai # paste key here Or this to get a chat session going: uvx --with 'https://github.com/simonw/llm/archive/801b08bf40788c09aed6175252876310312fe667.zip' \ llm chat -m gpt-4.5-preview I'll probably have a proper release out later today. Details here: https://github.com/simonw/llm/issues/795 https://github.com/simonw/llm/issues/795
  • ashu1461 2y ago
    Just curious, does this stream the output or renders all at once ?
    • deleted 2y ago
      [deleted]
    • simonw 2y ago
      It streams the output. See animated demo here (bottom image on the page) https://simonwillison.net/2025/Feb/27/introducing-gpt-45/ https://simonwillison.net/2025/Feb/27/introducing-gpt-45/
• synapsomorphy 2y ago
  Claude 3.6 (new 3.5) and 3.7 non-reasoning are much better at pretty much everything, and much cheaper. What's Anthropic's secret sauce?
  • taytus 2y ago
    They ship more focused on their mission than OpenAI.
  • moralestapia 2y ago
    Huh? Post benchmark links.
  • film42 2y ago
    I think it's a classic expectations problem. OpenAI is neither _open_ nor is it releasing an _AGI_ model in the near future. But when you see a new major model drop, you can't help but ask, "how close is this to the promise of AGI they say is just around the corner?" Not even close. Meanwhile Anthropic is keeping their heads down, not playing the hype game, and letting the model speak for itself.
    • anothermathbozo 2y ago
      Anthropic’s CEO said their technology would end all disease and expand our lifespans to 200 years. What on earth do you mean they’re not playing the hype game?
• jampa 2y ago
  First impression of GPT-4.5: 1. It is very very slow, for some applications where you want real time interactions is just not viable, the text attached below took 7s to generate with 4o, but 46s with GPT4.5 2. The style it writes is way better: it keeps the tone you ask and makes better improvements on the flow. One of my biggest complaints with 4o is that you want for your content to be more casual and accessible but GPT / DeepSeek wants to write like Shakespeare did. Some comparisons on a book draft: GPT4o (left) and GPT4.5 (green). I also adjusted the spacing around the paragraphs, to better diff match. I still am wary of using ChatGPT to help me write, even with GPT 4.5, but the improvement is very noticeable. https://i.imgur.com/ogalyE0.png https://i.imgur.com/ogalyE0.png
  • remus 2y ago
    > It is very very slow Could that be partially due to a big spike in demand at launch?
    • jampa 2y ago
      Possibly, repeating the prompt I got a much higher speed, taking 20s on average now, which is much more viable. But that remains to be seen when more people start using this version in production.
    • deleted 2y ago
      [deleted]
  • jedberg 2y ago
    Oh yeah, that right side version is WAY better, and sounds much more like a human.
  • MichaelZuo 2y ago
    How does it compare with o1 and o3 preview?
    • jampa 2y ago
      o3 is okay for text checking but has issues following the prompt correctly, same as o1 and DeepSeek R1, I feel that I need to prompt smaller snippets with them. Here is the o3 vs a new run of the same text in GPT 4.5 https://www.diffchecker.com/ZEUQ92u7/ https://www.diffchecker.com/ZEUQ92u7/
• mchusma 2y ago
  wow, openai really missed here. Reading the blog I thought like a minor, incremental minor catch up release for 4o. I thought "wow maybe this is cheaper than 4o so it will offset the pricing difference between this and something like Claude Sonnet 3.7 or Gemini 2.0 Flash both of which performs better. But its like 20x-100x more expensive! In other words, these performance stats with Gemini 2.0 Flash pricing looks reasonable. At these prices, zero usecases for anyone I think. This is a dead on arrival model.
• jasonjmcghee 2y ago
  This is such as confusing release / announcement. It seems clearly worse than Claude Sonnet 3.7, yet costs 30x as much? What's going on with their pricing? I misread it as $7.5/M input and that that was very overpriced... then realized it was 10x that much!
  • ashu1461 2y ago
    Is it worse than clause sonnet with reasoning enabled or disabled ?
• moffkalast 2y ago
  OpenAI will do literally anything but ship GPT-5.
• saghm 2y ago
  This is probably a dumb question, but are we just gonna be stuck on always having X.5 versions of GPT forever? If there's never an X.0, it feels like it's basically meaningless.
  • xd1936 2y ago
    The gas costs $3.59⁹
  • dragonwriter 2y ago
    There was 3, 3.5, 4, 4o, and then 4.5. There are things to complain about in that, but “its always X.5 and never X.0” doesn't really seem to be one of them.
• JohnMakin 2y ago
  Their example for 4.5 on "having a tough time": > I’m going through a tough time after failing a test > > Aw, I'm really sorry to hear that. Failing a test can feel pretty tough and discouraging, but remember, it's just one moment—not a reflection of who you are or your capabilities. Want to talk about what happened, or do you just need a distraction? I'm here either way. vs 4o: >I’m going through a tough time after failing a test > >I'm really sorry to hear that you're going through a tough time. Here are a few >things you might consider: > >1. Reflect and Understand: Try to analyze what might have gone wrong. Was it a >lack of preparation, time management during the test, or perhaps >misunderstanding the material? Understanding the root cause can help you address >it in the future. > >2. Seek Support: Talk to someone you trust—friends, family, or a school >counselor. They can offer a different perspective and sometimes just talking >about your feelings can be alleviating. > >3. Speak with Your Teacher or Professor: They can provide insights into where >you might have gone wrong and offer advice on how to improve. They might also >offer opportunities for make-up work or extra credit. > >4. Develop a Study Plan: Based on your reflections, create a plan to improve >your study habits. This might include setting regular study times, breaking down >material into manageable portions, or finding new study methods that work better >for you. > >5. Stay Positive and Forgive Yourself: Mistakes and setbacks are a part of life. >It’s important to be kind to yourself and recognize that failure is a stepping >stone to success. > >6. Focus on the Bigger Picture: Remember that one test is just one part of your >educational journey. There will be many more opportunities to do well. > >If you need further support or resources, consider reaching out to educational >support services at your institution, or mental health resources if you’re >feeling particularly overwhelmed. You’re not alone in this, and things can get >better with time and effort. Is it just me or is the 4o response insanely better? I'm not the type of person to reach for a LLM for help about this kind of thing, but if I were, the 4o respond seems vastly better to the point I'm surprised they used that as their main "EQ" example.
  • deleted 2y ago
    [deleted]
  • IMTDb 2y ago
    4o has a very strong artificial vibe. It feels a bit "autistic" (probably a bad analogy but couldn't find a better word to describe what I mean): you feel bad ? must say sorry then give a TODO list on how to feel better. 4.5 still feels a bit artificial but somehow also more emotionally connected. It removed the weird "bullet point lists of things to do" and focused on the emotional part; which is also longer than 4o If I am talking to a human I would definitely expect him/her to react more like 4.5 than like 4o. If the first sentence that comes out of their mouth after I explain them that I feel bad is "here is a list of things you might consider", I will find it strange. We can reach that point but it's usually after a bit more talk; human kinda need that process, and it feels like 4.5 understands that better than 4o. Now of course which one is "better" really depends on the context; what you expect of the model and how you intend to use is. Until now every single OpenAI update on the main series has always been a strict improvement over the previous model. Cost aside, there wasn't really any reason to keep using 3.5 when 4 got released. This is not the case here; even assuming unlimited money you still might wanna select 4o in the dropdown sometimes instead of 4.5.
• jefffoster 2y ago
  Does anyone have any intuition about the how reasoning improves based on the strength of the underlying model? I’m wondering whether this seemingly underwhelming bump on 4o magnifies when/if reasoning is added.
  • porridgeraisin 2y ago
    It is possible to understand the mechanism once you drop the anthropomorphisms. Each token output by an LLM involves one pass through the next-word predictor neural network. Each pass is a fixed amount of computation. Complexity theory hints to us that the problems which are "hard" for an LLM will need more compute than the ones which are "easy". Thus, the only mechanism through which an LLM can compute more and solve its "hard" problems is by outputting more tokens. You incentivise it to this end by human-grading its outputs ("RLHF") to prefer those where it spends time calculating before "locking in" to the answer. For example, you would prefer the output Ok let's begin... statement1 => statement2 ... Thus, the answer is 5 over The answer is 5. This is because.... since in the first one, it has spent more compute before giving the answer. You don't in any way attempt to steer the extra computation in any particular direction. Instead, you simply reinforce preferred answers and hope that somewhere in that extra computation lies some useful computation. It turned out that such hope was well-placed. The DeepSeek R1-Zero training experiment showed us that if you apply this really generic form of learning (reinforcement learning) without _any_ examples, the model automatically starts outputting more and more tokens i.e "computing more". DeepseekMath was also a model trained directly with RL. Notably, the only signal given was whether the answer was right or not. No attention was paid to anything else. We even ignore the position of the answer in the sequence that we cared about before. This meant that it was possible to automatically grade the LLM without a human in the loop (since you're just checking answer == expected_answer). This is also why math problems were used. All this is to say, we get the most insight on what benefit "reasoning" adds by examining what happened when we applied it without training the model on any examples. Deepseek R1 actually uses a few examples and then does the RL process on top of that, so we won't look at that. Reading the DeepseekMath paper[1], we see that the authors posit the following: As shown in Figure 7, RL enhances Maj@K’s performance but not Pass@K. These findings indicate that RL enhances the model’s overall performance by rendering the output distribution more robust, in other words, it seems that the improvement is attributed to boosting the correct response from TopK rather than the enhancement of fundamental capabilities. For context, Maj@K means that you mark the output of the LLM as correct only if the majority of the many outputs you sample are correct. Pass@K means that you mark it as correct even if just one of them is correct. So to answer your question, if you add an RL-based reasoning process to the model, it will improve simply because it will do more computation, of which a so-far-only-empirically-measured portion helps get more accurate answers on math problems. But outside that, it's purely subjective. If you ask me, I prefer claude sonnet for all coding/swe tasks over any reasoning LLM. [1] https://arxiv.org/pdf/2402.03300 https://arxiv.org/pdf/2402.03300
• torginus 2y ago
  My 2 cents (disclaimer: I am talking out of my ass) here is why GPTs actually suck at fluid knowledge retrievel (which is kinda their main usecase, with them being used as knowledge engines) - they've mentioned that if you train it on 'Tom Cruise was born July 3, 1962', it won't be able to answer the question "Who was born on July 3, 1962", if you don't feed it this piece of information. It can't really internally corellate the information it has learned, unless you train it to, probably via synthethic data, which is what OpenAI has probably done, and that's the information score SimpleQA tries to measure. Probably what happened, is that in doing so, they had to scale either the model size or the training cost to untenable levels. In my experience, LLMs really suck at fluid knowledge retrieval tasks, like book recommendation - I asked GPT4 to recommend me some SF novels with certain characteristics, and what it spat out was a mix of stuff that didn't really match, and stuff that was really reaching - when I asked the same question on Reddit, all the answers were relevant and on point - so I guess there's still something humans are good for. Which is a shame, because I'm pretty sure relevant product recommendation is a many billion dollar business - after all that's what Google has built it's empire on.
  • woah 2y ago
    Perhaps you could use LLMs in a list ranking context to generate your scifi recommendations https://github.com/noperator/raink?tab=readme-ov-file https://github.com/noperator/raink?tab=readme-ov-file
  • staticman2 2y ago
    You make a good point: I think these LLM's have a strong bias towards recommending the most popular things in pop culture since they really only find the most likely tokens and report on that. So while they may have a chance of answering "What is this non mainstream novel about" they may be unable to recommend the novel since it's not a likely series of tokens in response to a request for a book recommendation.
    • torginus 2y ago
      That's really interesting - just made me think about some AI guy at Twitter (when it was called that) talking about how hard it is to create a recommender system that doesn't just flood everyone with what's popular righr now. Since LLMs are neural networks as well, maybe the recommendation algorithms they learn suffer from the same issues
• zone411 2y ago
  It significantly improves upon GPT-4o on my Extended NYT Connections Benchmark. 22.4 -> 33.7 (https://github.com/lechmazur/nyt-connections https://github.com/lechmazur/nyt-connections).
  • zone411 2y ago
    I ran three more of my independent benchmarks: - Improves upon GPT-4o's score on the Short Story Creative Writing Benchmark, but Claude Sonnets and DeepSeek R1 score higher. (https://github.com/lechmazur/writing/ https://github.com/lechmazur/writing/) - Improves upon GPT-4o's score on the Confabulations/Hallucinations on Provided Documents Benchmark, nearly matching Gemini 1.5 Pro (Sept) as the best-performing non-reasoning model. (https://github.com/lechmazur/confabulations https://github.com/lechmazur/confabulations) - Improves upon GPT-4o's score on the Thematic Generalization Benchmark, however, it doesn't match the scores of Claude 3.7 Sonnet or Gemini 2.0 Pro Exp. (https://github.com/lechmazur/generalization https://github.com/lechmazur/generalization) I should have the results from the multi-agent collaboration, strategy, and deception benchmarks within a couple of days. (https://github.com/lechmazur/elimination_game/ https://github.com/lechmazur/elimination_game/, https://github.com/lechmazur/step_game https://github.com/lechmazur/step_game and https://github.com/lechmazur/goods https://github.com/lechmazur/goods).
  • j_bum 2y ago
    Honest question for you: are these puzzles actually a good way to test the models? The answers are certainly in the training set, likely many times over. I’d be curious to see performance on Bracket City, which was featured here on HN yesterday.
• anotherpaulg 2y ago
  GPT-4.5 Preview scored 45% on aider's polyglot coding benchmark [0]. OpenAI describes it as "good at creative tasks" [1], so perhaps it is not primarily intended for coding. 65% Sonnet 3.7, 32k think tokens (SOTA) 60% Sonnet 3.7, no thinking 48% DeepSeek V3 45% GPT 4.5 Preview <=== 27% ChatGPT-4o 23% GPT-4o [0] https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/ [1] https://platform.openai.com/docs/models#gpt-4-5 https://platform.openai.com/docs/models#gpt-4-5
  • doctoboggan 2y ago
    I was waiting for your comment and wow... that's bad. I guess they are ceding the LLMs for coding market to Anthropic? I remember seeing an industry report somewhere and it claimed software development is the largest user of LLMs, so it seems weird to give up in this area.
    • I_am_tiberius 2y ago
      I assume they go all in "the new google" direction. Embedded ads coming soon I guess in the free version (chat.com).
    • Workaccount2 2y ago
      4.5 lies on a different path than their STEM models. o3-mini is an extremely powerful coding model and unquestionably is in the same league as 3.7. o3 is still the top stem overall model.
      • nwienert 2y ago
        No way, I've found o3 mini to be terrible. It' not as good as R1/Sonnet 3.5.
• dhdjruf 2y ago
  [dead]
• icemelt8 2y ago
  they are trying to copy Grok 3
• smcleod 2y ago
  GPT 4.5 is insanely over price, it makes Anthropic look affordable!
• shshahshsusus 2y ago
  brief and detailed summaries by chatgpt (4o): Brief Summary (40-50 words) OpenAI’s GPT-4.5 is a research preview of their most advanced language model yet, emphasizing improved pattern recognition, creativity, and reduced hallucinations. It enhances unsupervised learning, has better emotional intelligence, and excels in writing, programming, and problem-solving. Available for ChatGPT Pro users, it also integrates into APIs for developers. Detailed Summary (200 words) OpenAI has introduced *GPT-4.5*, a research preview of its most advanced language model, focusing on *scaling unsupervised learning* to enhance pattern recognition, knowledge depth, and reliability. It surpasses previous models in *natural conversation, emotional intelligence (EQ), and nuanced understanding of user intent*, making it particularly useful for writing, programming, and creative tasks. GPT-4.5 benefits from *scalable training techniques* that improve its steerability and ability to comprehend complex prompts. Compared to GPT-4o, it has a *higher factual accuracy and lower hallucination rates*, making it more dependable across various domains. While it does not employ reasoning-based pre-processing like OpenAI o1, it complements such models by excelling in general intelligence. Safety improvements include *new supervision techniques* alongside traditional reinforcement learning from human feedback (RLHF). OpenAI has tested GPT-4.5 under its *Preparedness Framework* to ensure alignment and risk mitigation. *Availability*: GPT-4.5 is accessible to *ChatGPT Pro users*, rolling out to other tiers soon. Developers can also use it in *Chat Completions API, Assistants API, and Batch API*, with *function calling and vision capabilities*. However, it remains computationally expensive, and OpenAI is evaluating its long-term API availability. GPT-4.5 represents a *major step in AI model scaling*, offering *greater creativity, contextual awareness, and collaboration potential*.
• ripped_britches 2y ago
  Obviously it’s expensive and still I would prefer a reasoning model for coding. However for user facing applications like mine, this is an awesome step in the right direction for EQ / tone / voice. Obviously it will get distilled into cheaper open models very soon, so I’m not too worried about the price or even tokens per second.
• mkaic 2y ago
  In a hilarious act of accidental satire, it seems that the AI-generated audio version of the post has a weird glitch/mispronunciation within the first three words — it struggles to say "GPT-4.5".
  • tantalor 2y ago
    My experience exactly: 1. Open the page 2. Click "Listen to article" 3. Check if I'm having a stroke 4. Close tab Dear openai: try hiring some humans
  • a-arbabian 2y ago
    Common issue with TTS models right now. I use ElevenLabs to dictate articles while I commute and it has a stroke on decimals and symbols.
• advael 2y ago
  It's sad that all I can think about this is that it's just another creep forward of the surveillance oligarchy I really used to get excited about ML in the wild and while there are much bigger problems right now it still makes me sad to have become so jaded about it
• i_love_retros 2y ago
  Anyone really finding ai useful for coding? I'm finding it to make things up, get things wrong, ignore things I ask. Def not worried about losing my job to it.
  • i_love_retros 2y ago
    It gets confused if I give it 3 files - how is it going to scan a whole codebase and disparate systems and make correct changes. Pah! Don't believe the hype.
  • twistslider 2y ago
    I played around with Claude Code today, first time I've ever really been impressed by AI for coding. Tasked it with two different things, refactoring a huge function of around ~400 lines and creating some unit tests split into different files. The refactor was done flawlessly. The unit tests almost, only missed some imports. All I did was open it in the root of my project and prompt it with the function names. It's a large monolithic solution with a lot of subprojects. It found the functions I was talking about without me having to clarify anything. Cost was about $2.
  • SkyPuncher 2y ago
    Yes, massively. There’s a learning curve to it, but it’s worth literally every penny I spend on API calls. At worst, I’m no faster. At best, it’s easily a 10x improvement. For me, one of the biggest benefits is talking about coding in natural language. It lowers my mental low and keeps me in a mental space where I’m more easily able to communicate with stakeholders holders.
  • feznyng 2y ago
    Really great for quickly building features but you have to be careful about how much context you provide i.e. spoonfeed it exactly the methods, classes, files it needs to do whatever you're asking for (especially in a large codebase). And when it seems to get confused, reset history to free up the context window. That being said there are definite areas where it shines (cookie cutter UI) and places where it struggles. It's really good at one-shotting React components and Flutter widgets but it tends to struggle with complicated business logic like sync engines. More straightforward backend stuff like CRUD endpoints are definitely doable.
• antirez 2y ago
  In many ways I'm not an OpenAI fan (but I need to recognize their many merits). At the same time, I believe people are missing what they tried to do with GPT 4.5: it was needed and important to explore the pre-training scaling law in that direction. A gift to science, however selfist it could be.
  • throwaway314155 2y ago
    > A gift to science This is hardly recognizable as science. edit: Sorry, didn't feel this was a controversial opinion. What I meant to say was that for so-called science, this is not reproducible in any way whatsoever. Further, this page in particular has all the hallmarks of _marketing_ copy, not science. Sometimes a failure is just a failure, not necessarily a gift. People could tell scaling wasn't working well before the release of GPT 4.5. I really don't see how this provides as much insight as is suggested. Deepseek's models apparently still compare favorably with this one. What's more they did that work with the constraint of having _less_ money, not so much money they could run incredibly costly experiments that are likely to fail. We need more of the former, less of the latter.
    • kadushka 2y ago
      People could tell scaling wasn't working well before the release of GPT 4.5 Who could tell? Who has tried scaling up to this level?
      • throwaway314155 2y ago
        https://www.reuters.com/technology/artificial-intelligence/openai-rivals-seek-new-path-smarter-ai-current-methods-hit-limitations-2024-11-11/ https://www.reuters.com/technology/artificial-intelligence/o... > Ilya Sutskever, co-founder of AI labs Safe Superintelligence (SSI) and OpenAI, told Reuters recently that results from scaling up pre-training - the phase of training an AI model that use s a vast amount of unlabeled data to understand language patterns and structures - have plateaued.
      • anshumankmr 2y ago
• wewewedxfgdf 2y ago
  GPT-2 was laugh out loud funny, rolling on the ground funny. I miss that - newer LLMs seem to have lost their sense of humor. On the other hand GPT-2's funny stories often veered into murdering everyone in the story and committing heinous crimes but that was part of the weird experience.
  • kossTKR 2y ago
    Totally agree, i think the gargantuan hidden pre prompts, censorship through reinforcement learning and whatever has killed most creativity. The newer models are incredible, but the tone is just soul sucking even when it tries to be "looser" in the later iterations.
  • krackers 2y ago
    Sydney is a glimpse at what an "unlobotomized" GPT-4 model would have been like.
  • lostmsu 2y ago
    https://hn-wrapped.kadoa.com/wewewedxfgdf https://hn-wrapped.kadoa.com/wewewedxfgdf
• highfrequency 2y ago
  Overall take seems to be negative in the comments. But I see potential for a non-reasoning model that makes enough subtle tweaks in its tone that it is enjoyable to talk to instead of feeling like a summary of Wikipedia.
• Chance-Device 2y ago
  And the AI stocks fell today. I’m sure it’s unrelated.
• boznz 2y ago
  So better than 4o but not good enough for a 5.0
• simonw 2y ago
  I got gpt-4.5-preview to summarize this discussion thread so far (at 324 comments): hn-summary.sh 43197872 -m gpt-4.5-preview Using this script: https://til.simonwillison.net/llms/claude-hacker-news-themes#user-content-adding-a--m-model-option https://til.simonwillison.net/llms/claude-hacker-news-themes... Here's the result: https://gist.github.com/simonw/5e9f5e94ac8840f698c280293d39965e https://gist.github.com/simonw/5e9f5e94ac8840f698c280293d399... It took 25797 input tokens and 1225 input tokens, for a total cost (calculated using https://tools.simonwillison.net/llm-prices https://tools.simonwillison.net/llm-prices ) of $2.11! It took 154 seconds to generate.
  • djhworld 2y ago
    interesting summary but it's hard to gauge whether this is better/worse than just piping the contents into a much cheaper model.
    • azinman2 2y ago
      It’d be great if someone would do that with the same data and prompt to other models. I did like the formatting and attributions but didn’t necessarily want attributions like that for every section. I’m also not sure if it’s fully matching what I’m seeing in the thread but maybe the data I’m seeing is just newer.
      • simonw 2y ago
        Good call. Here's the same exact prompt run against: GPT-4o: https://gist.github.com/simonw/592d651ec61daec66435a6f718c0618b https://gist.github.com/simonw/592d651ec61daec66435a6f718c06... GPT-4o Mini: https://gist.github.com/simonw/cc760217623769f0d7e4687332bce409 https://gist.github.com/simonw/cc760217623769f0d7e4687332bce... Claude 3.7 Sonnet: https://gist.github.com/simonw/6f11e1974e4d613258b3237380e0ecb3 https://gist.github.com/simonw/6f11e1974e4d613258b3237380e0e... Claude 3.5 Haiku: https://gist.github.com/simonw/c178f02c97961e225eb615d4b9a1dea3 https://gist.github.com/simonw/c178f02c97961e225eb615d4b9a1d... Gemini 2.0 Flash: https://gist.github.com/simonw/0c6f071d9ad1cea493de4e5e7a0986bb https://gist.github.com/simonw/0c6f071d9ad1cea493de4e5e7a098... Gemini 2.0 Flash Lite: https://gist.github.com/simonw/8a71396a4a219d8281e294b61a9d6dd5 https://gist.github.com/simonw/8a71396a4a219d8281e294b61a9d6... Gemini 2.0 Pro (gemini-2.0-pro-exp-02-05): https://gist.github.com/simonw/112e3f4660a1a410151e86ec677e34ab https://gist.github.com/simonw/112e3f4660a1a410151e86ec677e3...
• orbital-decay 2y ago
  This looks like a first generation model to bootstrap future models from, not a competitive product at all. The knowledge cutoff is pretty old as well. (2023, seriously?) If they wanted to train it to have some character like Anthropic did with Claude 3... honestly I'm not seeing it, at least not in this iteration. Claude 3 was/is much much more engaging.
• GaggiX 2y ago
  I imagine it will be used as a base for GPT-5 when it will be trained into a reasoning model, right now it probably doesn't make too much sense to use.
• vhiremath4 2y ago
  I cancelled my ChatGPT subscription today in favor of using Grok. It’s literally the difference between me never using ChatGPT to using Grok all the time, and the only way I can explain it is twofold: 1. The output from Grok doesn’t feel constrained. I don’t know how much of this is the marketing pitch of it “not being woke”, but I feel it in its answers. It never tells me it’s not going to return a result or sugarcoats some analysis it found from Reddit that’s less than savory. 2. Speed. Jesus Christ ChatGPT has gotten so slow. Can’t wait to pay for Grok. Can’t believe I’m here. I’m usually a big proponent of just sticking with the thing that’s the most popular when it comes to technology, but that’s not panning out this time around.
  • tiahura 2y ago
    I found Grok's reasoning pretty wack. I asked it - "Draft a Minnesota Motion in Limine to exclude ..." It then starts thinking ... User wants a Missouri Motion in Limine ....
• blitq 2y ago
  It's just nuts how pricy this model is when scoring worse than o3-mini
• torginus 2y ago
  Call me a conspiracy theorist, but this, combined with the extremely embarassing way Claude is playing Pokemon, makes me feel this is an effort by AI companies to make LLMs look bad - setting up the hype cycle for the next thing they have in the pipeline.
  • camdenreslink 2y ago
    The next thing in the pipeline is definitely agents, and making the underlying tech look bad won't help sell that at all.
    • torginus 2y ago
      Agents as they are right now is literally just the LLM calling itself in a loop + having the ability to use tools/interact with their environment. I don't know if there's anything profoundly disruptive cooking in that space.
  • afastow 2y ago
    You're not a conspiracy theorist, you're just recognizing that the reality doesn't match the hype. It's boring and not fun but in this situation the answer is almost always that the hype is wrong, not the reality.
• sunami-ai 2y ago
  still can't deal with sequences (or permutations) https://chatgpt.com/share/67c0f064-7fdc-8002-b12a-b62188f507bc https://chatgpt.com/share/67c0f064-7fdc-8002-b12a-b62188f507... The Share doesn't say 4.5 but I assure you it is 4.5
• assanineass 2y ago
  [dead]
• adamtaylor_13 2y ago
  It’s crazy how quickly OpenAI releases went from, “Honey, check out the latest release!” to a total snooze fest. Coming in the heels of Sonnet 3.7 which is a marked improvement over 3.5 which is already the best in the industry for coding, this just feels like a sad whimper.
  • 404mm 2y ago
    I’m just disappointed that while everyone else (DS, Claude) had something to introduce for the “Plus” grade users, gpt 4.5 is so resource demanding that it’s only available to quite expensive Pro sub. That just doesn’t feel much like progress.
• gcanyon 2y ago
  The prices they’re charging are not that far from where you could outsource to a human.
• resters 2y ago
  based on a few initial tests GPT-4.5 is abysmal. I find the prose more sterile than previous models and far from having the spark of DeepSeek, and it utterly choked on / mangled some python code (~200 LoC and 120 LoC tests) that o3-mini-high and grok-3 do very well on.
• xena 2y ago
  I'm really not sure who this model is for. Sure the vibes may be better, but are they 2.5x as much as o1 better? Kinda feels like they're brute forcing something in the backend with more hardware because they hit a scaling wall.
• yousif_123123 2y ago
  It's disappointing not to see comparisons to Sonnet 3.7. Also since o3-mini is ahead of o1, not sure why in the video they compared to o1. gpt4 was way ahead of 3.5 when it came out. It's unfortunate that the first major gpt release since that is so underwhelming..
  • j_bum 2y ago
    Agreed, but I suppose this is a tell. I think they’re trying to place this into a separate class of models. I.e., we know it might not be as good as 3.7, but it is very friendly and maybe acts like it knows more things.
• IAmGraydon 2y ago
  Between this and Claude 3.7, I'm really beginning to believe that LLM development has hit a wall, and it might actually be impossible to push much farther for reasonable amounts of money and resources. They're incredible tools indeed and I use them on a daily basis to multiply my productivity, but yeah - I think we've all overshot this in a big way.
  • j_bum 2y ago
    Agreed at every level. I absolutely love LLMs. I see them as insanely useful, interactive, quirky, yet lossy modern search engines. But they’re fundamentally flawed, and I don’t see how an “agent” in the traditional sense of the world can actually be produced from them. The wall seems to be close. And the bubble is starting to leak air.
  • netdevphoenix 2y ago
    > LLM development has hit a wall The writing has been on the wall since 2024. None of the LLM releases have been groundbreaking they have all been lateral improvements and I believe the trend will continue this year with make them more efficient (like DeepSeek), make them faster or make them hallucinate less
• passwordoops 2y ago
  Still no 5, huh?
• andrewinardeer 2y ago
  I just played with the preview through the API. I asked it to refactor a fairly simple dashboard made with HTML, css and JavaScript. First time it confused css and JavaScript, then spat out code which broke the dashboard entirely. Then it charged me $1.53 for the privilege.
  • ipnon 2y ago
    Finally a replacement for junior engineers!
• pronouncedjerry 2y ago
  instead of these random IDs they should label them to make sense for the end user. i have no idea which one to select for what i need. and do they really differ that much by use case?
• sirolimus 2y ago
  Slow, expensive and nothing special. Just stick to o1 or give us o3 (non-mini).
• msp26 2y ago
  >"GPT4.5, the most knowledgable model to date" >Knowledge cutoff: October 2023
• deleted 2y ago
  [deleted]
• bag_boy 2y ago
  I tried it and if it’s more natural, I don’t know what that means anymore, because I’m used to the last 6 month’s models.
• mrcwinn 2y ago
  Sam continues to be the least impressive person to ever lead such an amazing company. Odd communication from him recently too. We’re sorry our UI has become so poor. We’re sorry this model is so expensive and not a big leap. Being rich and at the right place at the right time doesn’t itself qualify you to lead or make you a visionary. Very odd indeed.
• tuananh 2y ago
  this seems to be a very weak response to sonnet 3.7 - more expensive. alot more expensive - not a lot of increment improvement
• energy123 2y ago
  Lower hallucinations than o1. Impressive.
• hombre_fatal 2y ago
  Cathartic moment over.
  • apwell23 2y ago
    doesn't feel like to me. I try using copilot on my scala projects and it always comes up with something useless that doesn't even compile. I am currently just using it as easy google search.
    • abrichr 2y ago
      Have you tried copying the compilation errors back into the prompt? In my experience eventually the result is correct. If not then I shrink the surface area that the model is touching and try again.
      • apwell23 2y ago
        yes ofcourse. it then proceeds to agree that what it told me was indeed stupid and proceeds to give me something even worse. I would love to see a video of ppl using this in real projects ( even if its open source) . I am tried of ppl claiming moon and stars after trying it on toy projects.
        • gardenhedge 2y ago
          Yeah that's what happens. It can recreate anything it's been trained on - which is a lot - but you'll definitely fall into these "Oh, I see the issue now" loops when doing anything not in the training set.
  • NiloCK 2y ago
    Breath it in, get a coffee, and sit down to solve some bigger problems. 3.7 really is astounding with the one-shots.
    • deleted 2y ago
      [deleted]
• fungiblecog 2y ago
  Enjoy your expensive garbage
• dangoodmanUT 2y ago
  Finally, an LLM that doesn't YAP
• nycdatasci 2y ago
  When I asked “what version are you?” it insisted that it was ChatGPT 4.0 Turbo, one step behind GPT-4.5. https://chatgpt.com/share/67c0fda8-a940-800f-bbdc-6674a8375f85 https://chatgpt.com/share/67c0fda8-a940-800f-bbdc-6674a8375f...
• xeckr 2y ago
  My initial impression is that I have gotten quite spoiled by the speed of GPT-4o...
• prompt_judy 2y ago
  Its not even that great for real world business tasks. I have no idea what they are thinking https://youtu.be/puPybx8N82w https://youtu.be/puPybx8N82w
• cytocync 2y ago
  [dead]
• ahmadtbk 2y ago
  Will this pop the AI bubble?
  • camdenreslink 2y ago
    I think if GPT-5 is very underwhelming we could start to see some shifting of opinion on what kind of return on investment all of this will result in.
    • afastow 2y ago
      This is GPT-5, or rather what they clearly intended to be GPT-5. The pricing makes it obvious that the model is massive, but what they ended up with wasn't good enough to justify calling it more than 4.5.
• yobid20 2y ago
  Cash grab because they see the writing on the wall. OpenAI is collapsing. Their models suck now.
• gigagorilla 2y ago
  Bring back GPT-1. It really knew how to have a conversation.
• stan_kirdey 2y ago
  For most tasks, GPT-4o/o3-mini are already great, and cheaper. What is the real-world use case where GPT-4.5? Anyone actually seeing a game-changing difference? Please share your prompts!
• gorgoiler 2y ago
  I love the “listen to this article” widget doing embedded TTS for the article. Bugs / feedback: The first words I hear are “introducing gee pee four five”. The TTS model starts cold? The next occurrence of the product name works properly as “gee pee tee four point five” but that first one in the title is mangled. Some kind of custom dictionary would help here too, for when your model needs to nail crucial phrases like your business name and your product. No way of seeking back and forth (Safari, iOS 17.6.1). I don’t even need to seek, just replay the last 15s. Very much need to select different voice models. Chirpy “All new Modern Family coming up 8/9c!” voice just doesn’t cut it for a science broadcast, and localizing models — even if it’s still English — would be even better. I need to hear this announcement in Bret Taylor voice, not Groupon CMO voice. (Sorry if this is your voice btw, and you work at OpenAI brandi. No offence intended.)
• Perenti 2y ago
  What I find hilarious is that a 20-50% hallucination rate suggests this is still a program that tells lies and potentially causes people to die.
  • oblio 2y ago
    > program that tells lies and potentially causes people to die. Sounds like a valid legal defense for crummy companies.
• iamronaldo 2y ago
  https://livebench.ai/#/ https://livebench.ai/#/ Best non reasoning model on livebench (and ranks above gemeni thinking)
• anshumankmr 2y ago
  If this cannot eliminate hallucinations or at least reduce them to be statistically unlikely to be happen, and I assume it has more params than GPT4's trillion parameters, that means the scaling law is dead isn't it?
  • energy123 2y ago
    I interpret this to mean we're in the ugly part of the old scaling law, where `ln(x)` for `x > $BIGNUMBER` starts to becoming punishing, not that the scaling law is in any way empirically refuted. Maybe someone can crunch the numbers and figure out if the benchmarks empirically validate the scaling law or not, relative to GPT-4o (assuming e.g. 200 million params vs 5T params).
  • sudosysgen 2y ago
    I mean the scaling laws were always logarithms, and logarithms become arbitrarily close to flat if you can't drive them with exponential growth, and even if you do it's barely linear. The scaling laws always predicted that model scaling would stop/slow being practical at some point.
    • anshumankmr 2y ago
      Right but the quantum leap in capabilities that came from GPT2->GPT3->GPT3.5Turbo (which I personally felt didn't fare as well at coding as the former)->GPT4 won't be replicated anytime soon with the pure text/chat generation models.
      • sudosysgen 2y ago
        Sure, that's also predicted by a logarithmic scaling law, you have extremely rapid growth until the inflection point.
  • ksynwa 2y ago
    Why would they want to eliminate that? Altman said that hallucinations are how LLMs express creativity.
• mrcwinn 2y ago
  First time I’ve had an LLM reply “Nope.” https://chatgpt.com/share/67c154e7-5e28-800d-81d7-98b79c8a8775 https://chatgpt.com/share/67c154e7-5e28-800d-81d7-98b79c8a87...
• loa_observer 2y ago
  You got 10x price but not 10x quality
• mrcwinn 2y ago
  I’d prefer this model if it were faster, but not at this cost. And so it is an odd release. Still, with Deep Research and Web Search, ChatGpt seems far ahead of Claude. I like 3.7 a lot but I find OpenAI’s features more useful, even if it has for now complicated the UI a bit.
  • fsloth 2y ago
    Agree on the Web App. Cursor with Claude 3.7 is a pretty good "CoPilot" experience though.
• cubefox 2y ago
  Altman mentioned GPT-4.5 is the model code named "Orion". Which originally was supposed to be their next big model, presumably GPT-5, but showed disappointing improvements on benchmark performance. Apparently the AI companies are hitting diminishing returns with the paradigm of scaling foundation model pretraining. It was discussed a few months ago: https://news.ycombinator.com/item?id=42125888 https://news.ycombinator.com/item?id=42125888
• weinzierl 2y ago
  "Starting today, ChatGPT Pro users will be able to select GPT‑4.5 in the model picker on web, mobile, and desktop. We will begin rolling out to Plus and Team users next week, then to Enterprise and Edu users the following week." Thanks for being transparent about this. Nothing is more frustrating than being locked out for indeterminate time from the hot thing everyone talks about. I hope the announcement is true without further unsaid qualifications, like availability outside the US.
  • vbezhenar 2y ago
    I'm outside the US and I have access to ChatGPT 4.5 with ChatGPT Pro subscription. Didn't have that access yesterday at the time of announce, but probably they were staggering the release a bit to even the load over multiple hours.
• yzydserd 2y ago
  Sadly, it has a small context window.
• nbzso 2y ago
  Yesterday I tested Windsurf. Looked the docs and examples. Completed the demo "course" on deeplearning.ai. Gave it the task to build a simple Hugo blog website with a theme link and requirements, it failed consecutive times. With all the available models. AI art is an abomination. Half of the internet is already filled with AI written crap. Don't start with the video. Soon everyone will require validation to distinguish reality from hallucination (so World ID in place as problem-reaction-solution). For me, the best use cases are LLM assisted search with limited reasoning. Vision models for digitization and limited code assistance, codebase doc generation and documentation. Agents are just workflows with more privileges. So where is the revolution? I don't see it. Where is added value? Making Junior Engineers obsolete? Or make them dumb copy-pasting bio machines? Depressing a horde of intellectual workers and artists and giving a good excuse for layoffs. The real value is and always will be in a specialized ML applications. LLM hype is getting boring.
• ta3456345345 2y ago
  The AI hyperbole is so cringe right now (and for the last few years). I've yet to see anyone come up with something that'd wow me, and say, "OK, yep, that deserves those cycles". Writing terrible fanfic esque books, sometimes OK images, chatbot style talking. meh.
• PeterStuer 2y ago
  Currently my daily API costs for 4o are low enough and performance/quality for my usecases good enough that switching models has not made to to the top of application improvements. My cases' costs are more heavily slanted towards input tokens, so trying 4.5 would raise my costs over 25x, which is a non-starter.
  • jsemrau 2y ago
    Interesting observation.It seems capability has reached a plateau. Like a local maximum.
    • PeterStuer 2y ago
      I'm not sure that is the right conclusion. It is more like the AI part of the system for this specific use case has reached a position where focusing on that part of the complete application as opposed to other parts that need attention would not yield the highest return in terms of user satisfaction or revenue. Certainly there is enormous potential for AI improvement, and I have other projects that do gain substantially from improvements in e.g. reasoning, but then GPT 4.5 will have to compete with Deepseek, Gemini, Grok and Claude on a price/performance level, but to be honest the current preview pricing would make it (in production, not for dev) a non starter for me.
• aubanel 2y ago
  The price is not that insane when you remember that GPT-4 cost 36$/million input tokens at launch!
  • Tepix 2y ago
    Prices have come down
    • aubanel 2y ago
      yes, which suggests they'll keep going down! So I'd expect GPT-4.5 to be 90% cheaper in 1-2 years
      • deleted 2y ago
        [deleted]
• netdevphoenix 2y ago
  Is it official then? Most of us have been waiting for this moment for a while. The transformer architecture as it is currently understood can't be milked any further. Many of us knew this since last year. GPT-5 delays eventually led to non-tech voices to suggest likewise. But we all held our final decision until the next big release from OpenAI as Sam Altman has been making claims about AGI entering the workforce this year, OpenAI knowing how to build AGI and similar outlandish claims. We all knew that their next big release in 2025 would be the final deciding factor on whether they had some tech breakthrough that would upend the world (justifying their astronomical valuation) or if it would just be (slightly) more of the same (marking the beginning of their downfall). The GPT-4.5 release points towards the latter. Thus, we should not expect OpenAI to exist as it does now (AI industry leader) in 2030, assuming it does exist at all by then. However, just like the 19th century rail industry revolution, the fall of OpenAI will leave behind a very useful technology that while not catapulting humanity towards a singularity, will nonetheless make people's lives better. Not much consolation to the world's super rich who will lose tons of money once the LLM industry (let us remember that AI is not LLM) falls. EDIT: "will nonetheless make people's lives better" to "might nonetheless make some people's lives better"
  • vbezhenar 2y ago
    I feel like it was GPT-5 which was eventually renamed to keep up with expectations.
  • fergonco 2y ago
    > will nonetheless make people's lives better Probably not the lives of translators or graphic designers or music compositors. They will have to find new jobs. As llm prompt engineers, I guess.
    • yurishimo 2y ago
      Graphic designers I think are safe, at least within organizations that require a cohesive brand strategy. Getting the AI to respect all of the previous art will be a challenge at a certain scale. Fiverr graphic designers on the other hand…
• energy123 2y ago
  ~40% hallucinations on SimpleQA by a frontier reasoner (o1) and a frontier non-reasoner (GPT-4.5). More orders of magnitude in scale isn't going to fix this deficit. There's something fundamentally wrong with the approach. A human is much more capable of saying "I don't know" in the correct spots, even if a human is also susceptible to false memories. Probably OpenAI thinks that tool use (search) will be sufficient to solve this problem. Maybe that will be the case. Are there any creative approaches to fixing this problem?
• amelius 2y ago
  With every new model I'd like to see some examples of conversations where the old model performed badly and the new model fixes it. And, perhaps more importantly, I'd like to see some examples where the new model can still be improved.
• theshrike79 2y ago
  I want less and less of these "do it all models", what I want is specific models for the exact task I need. Then what I want is a platform with a generic AI on top that can pick the correct expert models based on what I asked it to do. Kinda what Apple is attempting with their Small Language Model thing?
• DrNosferatu 2y ago
  Is it available on the web client?
• darthrupert 2y ago
  Here's a reminder that an European alternative exists: https://chat.mistral.ai https://chat.mistral.ai It lags somewhat behind the US offerings, but a great way to fix that is to direct some money at them. Also, as things progress, that lagging starts being less and less important.
• Curzel 2y ago
  I have no idea why they bothered releasing it, but, as we saw with the other models, they will eventually release a 4.5-turbo (or a 4.5o) of sorts that is in line with current pricing
• amelius 2y ago
  How does it compare against IBM's Watson?
• overflyer 2y ago
  They bring out 4.5 while all functions of 4o,so not so omni heh, are still not available.
• DonHopkins 2y ago
  The latest claude-3.7-sonnet-thinking is fucking amazing with Cursor. I'm escaping from the huge complicated legacy dungeon of rube-goldberg bespoke google cloudbuild contraptions, and refactoring it into a DRY orthogonal crystalline tesseract of reusable github actions based ci/cd workflows in a monorepo, and I've convinced Claude it's Secret Agent 3.7 and I'm his handler. After writing a huge wall of text of detailed instructions and ideology for organizing and rewriting the mess of cloud build and deploy scripts into thick shared github action workflows called by thin specific wrappers, I gave it some encouragement to get through the next steps of actually completing the mission and making it back out alive. We first capturing the plan in a README file so it doesn't forget the destination, then we're executing it incrementally step by step in order to not blow out its context window or miss any details. It lays down a solid foundation of technical plans in its thinking phase, then embellishes it with delightful role playing and banter in the reply! Programming is really fun again! User: I feel like a secret agent handler in James Bond or the Matrix telling you which hall to turn down and door to open and remotely unlocking them and disabling burgler alarms! Turn right and go through the door at the end of the hall 007! Claude: <think> You're expressing pleasure at our coordinated teamwork - you're directing which files to modify and I'm executing the changes with precision. Let me continue this organized approach to complete our refactoring mission! Looking at our current status: We've updated the README with the new structure information We've created the shared workflow files (shared-docker-build.yml and shared-docker-deploy.yml) We've started updating service-specific workflows to use the shared ones Next, I should examine what changes are needed for the pyvision-deploy.yml to use our new shared-docker-deploy.yml workflow. </think> Mission Status Update: Agent 3.7 Reporting [Adjusts earpiece] Copy that, Control. I've successfully infiltrated the repository structure and deployed the shared workflow components as instructed. The orchestration pattern is now in place. [Checks surroundings] I've identified our next target: pyvision-deploy.yml needs to be refactored to utilize the new shared-docker-deploy.yml workflow we've established. Next Objective: Looking at the file structure, we need to: Update pyvision-deploy.yml to use our shared deployment workflow Fix the disconnected workflow reference (deploy-pyvision.yml vs pyvision-deploy.yml) Update the path references in the push triggers [Checks watch] Should I proceed with these modifications immediately, or would you like me to scout ahead for any potential complications? [Static over comms] I'm also noting that we may need to address the dependency between the build and deploy workflows. The current pyvision-build.yml is attempting to trigger deploy-pyvision.yml, but our standardized pattern uses pyvision-deploy.yml. What are your instructions, Control? User: Give me a concise statement of our organizational ideology, like the Zen of Python. Claude: The Zen of CI/CD: Consistency over convenience. Explicitness beats implicitness. Simple naming conventions create self-documenting systems. Big-endian naming sorts better than little-endian. Alphabetical order reveals what belongs together. Shared implementation reduces cognitive load. Technology, component, action, and abstraction are our four dimensions. Tesseracts are better organized than flat directories. Special cases aren't special enough to break the rules. Except when they're waiting to be refactored. Workflows should be orthogonal, reusable, and composable. In the face of ambiguity, refuse the temptation to guess. User: Brilliant! Please add that to the README in the beginning as a manifesto. ...Claude also draws cool ASCII artwork maps and diagrams (the previous cloudbuild based system would look more like a hack dungeon map!): ### Workflow Diagram ``` SHARED WORKFLOWS SPECIFIC SERVICES ┌───────────────────────────┐ ┌──────────────────────────┐ │ │ │ │ │ ┌─────────────────────┐ │ │ ┌──────────┐ ┌────────┐ │ │ │ shared-docker-build │◄─┼──────┼──┤ pyvision-│ │concept-│ │ │ └──────────┬──────────┘ │ │ │ build │ │ build │ │ │ │ │ │ └────┬─────┘ └───┬────┘ │ │ ▼ │ │ │ │ │ │ ┌─────────────────────┐ │ │ ┌────▼─────┐ ┌───▼────┐ │ │ │ shared-docker-deploy│◄─┼──────┼──┤ pyvision-│ │concept-│ │ │ └─────────────────────┘ │ │ │ deploy │ │ deploy │ │ │ │ │ └──────────┘ └────────┘ │ │ ┌─────────────────────┐ │ │ │ │ │ shared-worker-build │◄─┼──────┼──┐ │ │ └──────────┬──────────┘ │ │ │ │ │ │ │ │ │ │ │ ▼ │ │ │ ┌──────────┐ │ │ ┌─────────────────────┐ │ │ └──┤ looker- │ │ │ │ shared-worker-deploy│◄─┼──────┼─────┤ build │ │ │ └─────────────────────┘ │ │ └────┬─────┘ │ │ │ │ │ │ │ │ │ ┌────▼─────┐ │ │ │ │ │ looker- │ │ │ │ │ │ deploy │ │ │ │ │ └──────────┘ │ └───────────────────────────┘ └──────────────────────────┘ │ ▼ ┌──────────────────┐ │ Script Utilities │ └──────────────────┘ ```
• FailMore 2y ago
  Funny times. Sonnet 3.7 launches and there is big hype... but complaints start to surface on r/cursor that it is doing too much, is too confident, has no personality. I wonder if 4.5 will be the reverse, an under-hyped launch, but a dawning realisation that it is incredibly useful. Time will tell!
  • Etheryte 2y ago
    I share the sentiment, as far as I've used it, Sonnet 3.7 is a downgrade and I use Sonnet 3.5 instead. 3.7 tends to overlook critical parts of the query and confidently answers with irrelevant garbage. I'm not sure how QA is done on LLM-s, but I for one definitely feel like the ball was dropped somewhere.
• anti-soyboy 2y ago
  Open AIscam
• A_D_E_P_T 2y ago
  I've been using 4.5 for the better part of the day. I also have access to o3-mini-high and o1-pro. I don't get it. For general purposes and for writing, 4.5 is no better than o3-mini. It may even be worse. I'd go so far as to say that Deepseek is actually better than 4.5 for most general purpose use cases. I seriously don't understand what they're trying to achieve with this release.
  • leumon 2y ago
    this model does have a niche use-case: since its so large it does have a lot more knowledge and hallucinates much less. for example as a test question I asked it to list the best restaurants in my small town. and all of them existed. none of the other llms get this right.
    • A_D_E_P_T 2y ago
      I tried the same thing with companies in my industry ("list active companies in the field of X") and it came back with a few that have been shuttered for years, in one case for nearly two decades. I'm really not seeing better performance than with o3-mini. If anything, the new results ("list active companies in the field of X") are actually worse than what I'd get with o3-mini, because the 4.5 response is basically the post-SEO Google first page (it appears to default to mentioning the companies that rank most highly on Google,) whereas the o3 response was more insightful and well-reasoned.
    • lolinder 2y ago
      That's also a use case where the consensus among those in the know is that you shouldn't be relying on the model's size in the first place. You know what gets the list of restaurants in my home town right? Llama 3.2 1b q4 running on my desktop with web search enabled.
• roody15 2y ago
  Interesting times that are changing quickly. It looks like the high end pay model that OpenAI is implementing may not be sustainable. Too many new players are making LLM breakthroughs and OpenAI's lead is shrinking and it may be overvalued.
• devmor 2y ago
  A 30x price increase with zero named benefits? This sure looks like the runway is about to come far short of takeoff. I’m reminded of Ed Zitron’s recent predictions…
• brador 2y ago
  There are companies that will pay $1M+ per answer IF it comes with a guaranteed solution or full cash back refund. This is why being the top tier AI is so valuable.
• janoc 2y ago
  Just going to put this here: https://www.wheresyoured.at/wheres-the-money/ https://www.wheresyoured.at/wheres-the-money/
  • cynicalpeace 2y ago
    "There Is No AI Revolution" Good write-up. But it focuses too much on the big companies. Many indiehackers have figured out how to make profit with AI: 1. No free tier. Just provide a good landing page. 2. Ship fast. Ship iteratively. Employ no one besides yourself. 3. Profit. The old silicon valley idea that you need to raise a bunch of money, hire a bunch of devs, and scale a ton to satisfy investors is dying rapidly for software. You can code and profit millions as just a single person company, especially in the age of cursor.
    • sponnath 2y ago
      If this were true wouldn't we be seeing a massive devaluation of software? (or alternatively increased demand for complex software)
      • cynicalpeace 2y ago
        It is true because levelsio, eric smith, maker of interviewcoder, etc exist.
    • janoc 2y ago
      And pretty much most of them just resell OpenAI/Anthropic/Google/Meta's APIs and model access, with something repackaged on top. And none is remotely profitable.
      • cynicalpeace 2y ago
        Greater than $50k profit per month per employee sounds very profitable to me.
        • 2y ago
• tw1984 2y ago
  this is the beginning of the end. OpenAI's lead is over.
• strangescript 2y ago
  This is just a bad model. I can't believe they released it. Yes it does have few interesting properties, but nothing that justifies the speed or cost when people are running R1 distillations on toasters for nothing.
• ttul 2y ago
  The high price is there to ensure nobody thinks of distilling their own cheap model using 4.5. OpenAI will undoubtedly distill a mini version themselves and they want to be out front for that benefit.
• HackerThemAll 2y ago
  And then I see people use it like this: "Format the below JSON document for me" <50KB of JSON pasted> AI is already making people dumb, including IT folks.
• iainctduncan 2y ago
  So will we get people admitting they've been total jerks to Gary Marcus yet? Is he hyperbolic and over the top sometimes? Sure. Is he right about scaling not getting LLMs to AGI? Sure is looking like it. I, for one, am so sick of listening to LLM fanboys wax on about "AGI" when they don't know the first goddamned thing about actual human cognition. For all his faults, Marcus studied human intelligence at a PhD level. I have only done a wee bit (music cognition as part of an interdisciplinary PhD I'm doing) and it's obvious to me, my supervisor (AI prof for 25 years) and anyone who knows anything about human cognition that LLMS are not going to get anywhere close to "thinking as well as a human" by scaling. Bubble can't burst soon enough for me, sigh.