3 ms·

I guess OpenAI got a good deal at 6.5b for Jony Ive. >Supposedly, Llama 4 did perform well on benchmarks, but the user experience was so bad that people have a

by duxup 1y ago

I guess OpenAI got a good deal at 6.5b for Jony Ive. >Supposedly, Llama 4 did perform well on benchmarks, but the user experience was so bad that people have accused Meta of cooking the books. This is one of those things that I've noticed. I don't understand the benchmarks, and my usage certainly isn't going to be wide ranging as benchmarks, but I hear "OMG this AI and benchmarks" and then I go use it and it's not any different for me ... or I get the usual wrong answers to things I've gotten wrong answers to before, and I shrug.

theahura 1y ago
I think there's basically the benchmarks, and then "the benchmarks". The former is, like, actual stuff you can quantify and game. And the latter is the public response. The former is a leading indicator but it's not always accurate, and the latter is what actually matters. I knew Gemini 2.5 was better than Claude 3.7 based on the Reddit reaction lol
- deleted 1y ago
  [deleted]
- niuzeta 1y ago
  Just out of curiosity, what were the reactions like from what you saw? I had the opposite take from Reddit, which proved to be incorrect. So I'm just curious how you read(more correctly than me) the reactions vis-a-vis Reddit.
  - theahura 1y ago
    Idk just vibes. I haven't joined any subreddits except r/civ. The feed is inscrutable
    - duxup 1y ago
      Civ could sure use some AI…
- duxup 1y ago
  I keep meaning to try Gemini for coding but honestly they’ve all gotten reasonably good for my use cases that I don’t think I need to try a lot of new ones / new versions.
- triknomeister 1y ago
  Any industry that cannot depend on artificial benchmarks, and instead has to depends on user benchmarks is a labor intensive industry. If it turns out that LLMs in their current form do depend a lot on manual user benchmarks, then they will always require human workers. Kind of like how AI means actually Indians.
dangus 1y ago
In retrospect it’s quite amazing how Jony Ive managed to make a superstar career out of almost directory copying Dieter Rams all the time. I can’t imagine what kind of value he could bring to OpenAI. The dude ruined Apple laptops for 5 years, he really should be an industry castaway.
- krackers 1y ago
  Not just hardware, he ruined UI design as well. Everything iOS 7+ (which hasleaked into macOS) is basically his doing, or at least had to be approved by him. Trading Scott Forstall for Johnny Ive was probably the the biggest blunder that Cook made, which put Apple on its current trajectory in terms of software & ui decline.
- blackguardx 1y ago
  I learned to program C in high school on the original iMacs and thanks to OS 9's lack of true multi-tasking and Ive's penchant for lack of buttons, anytime our code crashed or made an infinite loop, we would have to reach around and unplug the computer from the wall. The single power button on the front was a soft button that required CPU cycles to operate. There was also a tiny hole in the front that you could insert a paperclip to reset the machine, but unplugging from the wall was faster.
moffkalast 1y ago
In retrospect it wasn't really that the models were that bad compared to benchmarks, it was that Meta didn't work with inference engine providers ahead of time to integrate the new architecture like other teams usually do, so it was even more bugged than expected for the first few weeks. The second problem that compounded it was that both L4 models were too large for 99.9% of people to run, so the only opinion most people had of it was what the few that could load it were saying after release. And they weren't saying good things. So after inference was fixed the reputation stuck because hardly anyone can even run these behemoths to see otherwise. Meta really messed up in all ways they possibly could, short of releasing the wrong checkpoint or something.
paxys 1y ago
The Llama 4 benchmarks published by Meta were proven to be fake immediately after it launched.
danpalmer 1y ago
OpenAI didn’t get Jony Ive though. They bought an existing joint venture, i.e. he cashed out. They have a contract with his design firm for him to keep working on the project, but it’s basically great for him (massive payout, ongoing payments) and not particularly good for OpenAI.
paulryanrogers 1y ago
> I guess OpenAI got a good deal at 6.5b for Jony Ive. Did they? All I know of Ive's work at Apple is negative: too thin laptops that can't be repaired, uncomfortable keyboards, mice that cannot be used when charging, case designs that cause worse performance, abstract and unnecessarily changing UIs, and calculators that may miscalculate if tapping quickly.

by duxup 1y ago

theahura 1y ago

I think there's basically the benchmarks, and then "the benchmarks". The former is, like, actual stuff you can quantify and game. And the latter is the public response. The former is a leading indicator but it's not always accurate, and the latter is what actually matters. I knew Gemini 2.5 was better than Claude 3.7 based on the Reddit reaction lol

deleted 1y ago
[deleted]
niuzeta 1y ago
Just out of curiosity, what were the reactions like from what you saw? I had the opposite take from Reddit, which proved to be incorrect. So I'm just curious how you read(more correctly than me) the reactions vis-a-vis Reddit.
- theahura 1y ago
  Idk just vibes. I haven't joined any subreddits except r/civ. The feed is inscrutable
  - duxup 1y ago
    Civ could sure use some AI…
duxup 1y ago
I keep meaning to try Gemini for coding but honestly they’ve all gotten reasonably good for my use cases that I don’t think I need to try a lot of new ones / new versions.
triknomeister 1y ago
Any industry that cannot depend on artificial benchmarks, and instead has to depends on user benchmarks is a labor intensive industry. If it turns out that LLMs in their current form do depend a lot on manual user benchmarks, then they will always require human workers. Kind of like how AI means actually Indians.

dangus 1y ago

In retrospect it’s quite amazing how Jony Ive managed to make a superstar career out of almost directory copying Dieter Rams all the time. I can’t imagine what kind of value he could bring to OpenAI. The dude ruined Apple laptops for 5 years, he really should be an industry castaway.

krackers 1y ago
Not just hardware, he ruined UI design as well. Everything iOS 7+ (which hasleaked into macOS) is basically his doing, or at least had to be approved by him. Trading Scott Forstall for Johnny Ive was probably the the biggest blunder that Cook made, which put Apple on its current trajectory in terms of software & ui decline.
blackguardx 1y ago
I learned to program C in high school on the original iMacs and thanks to OS 9's lack of true multi-tasking and Ive's penchant for lack of buttons, anytime our code crashed or made an infinite loop, we would have to reach around and unplug the computer from the wall. The single power button on the front was a soft button that required CPU cycles to operate. There was also a tiny hole in the front that you could insert a paperclip to reset the machine, but unplugging from the wall was faster.

moffkalast 1y ago

In retrospect it wasn't really that the models were that bad compared to benchmarks, it was that Meta didn't work with inference engine providers ahead of time to integrate the new architecture like other teams usually do, so it was even more bugged than expected for the first few weeks. The second problem that compounded it was that both L4 models were too large for 99.9% of people to run, so the only opinion most people had of it was what the few that could load it were saying after release. And they weren't saying good things. So after inference was fixed the reputation stuck because hardly anyone can even run these behemoths to see otherwise. Meta really messed up in all ways they possibly could, short of releasing the wrong checkpoint or something.

paxys 1y ago

The Llama 4 benchmarks published by Meta were proven to be fake immediately after it launched.

danpalmer 1y ago

OpenAI didn’t get Jony Ive though. They bought an existing joint venture, i.e. he cashed out. They have a contract with his design firm for him to keep working on the project, but it’s basically great for him (massive payout, ongoing payments) and not particularly good for OpenAI.

paulryanrogers 1y ago

> I guess OpenAI got a good deal at 6.5b for Jony Ive. Did they? All I know of Ive's work at Apple is negative: too thin laptops that can't be repaired, uncomfortable keyboards, mice that cannot be used when charging, case designs that cause worse performance, abstract and unnecessarily changing UIs, and calculators that may miscalculate if tapping quickly.