7 ms·
If what they say is true, then you have to give them credit for catching up incredibly fast. And slightly pulling ahead. Not only with the models, but also prod
by ilaksh 2y ago
If what they say is true, then you have to give them credit for catching up incredibly fast. And slightly pulling ahead. Not only with the models, but also products.
- Rover222 2y agoAnd they mentioned at the end of the presentation that they're already planning their next datacenter, which will require 5x the power. Not sure if that means equivalent to ~1,000,000 of the current GPU's, or more because next-gen Nvidia chips are more efficient.
- grubbs 2y agoThe B300 8-way SXMs will use around 1.4kW for each GPU. I think the TDP on an H100 is like 700W.
- htfy96 2y agoI have a close friend working in core research teams there. Based on our chats, the secret seems to be (1) massive compute power (2) ridiculous pay to attract top talents from established teams (3) extremelly hard work without big corp bureaucracy.
- deleted 2y ago[deleted]
- nomilk 2y agoIt was mentioned during the launch that current datacenter requires up to 0.25 gigawatts of power. The datacenter they're currently building will require 1.25 (5x) (for reference, a nuclear powerplant might output about 1 gigawatt). Will be interesting to see if the relationship between power/compute/parameters and performance is exponential, logarithmic or something more linear.
- ncr100 2y ago[flagged]
- zurfer 2y agoIt's logarithmic. Meaning you scale compute exponentially to get linearly better models. However there is a big premium in having the best model because of low switching costs of workloads, creating all sorts of interesting threshold effects.
- Davidzheng 2y agoLinear in what metric?
- 535188B17C93743 2y agoPresumably the benchmarks? I'm also interested.
- deleted 2y ago[deleted]
- energy123 2y agoIt's logarithmic in benchmark scores, not in utility. Linear differences in benchmarks at the margin don't translate to linear differences in utility. A model that's 99% accurate is very different in utility space to a model that's 98% accurate.
- Xelynega 2y agoNot really since both give you wrong output that you need to design a system to account for(or deal with). The only percentage that would change the utility would be 100% accurate.
- WanderPanda 2y agoYes, it seems like capability is logarithmic wrt compute but utility (in different applications) is exponential (or rather s-shaped) with capability again
- hector126 2y agoAnecdotal, but I've gotten three recruiting emails from them now for joining their iOS team. I got on a call and confirmed they were offering FAANG++ comp but with the expectation of in-office 50h+ (realistically more). I don't have that dog in me anymore, but there are plenty of engineers who do and will happily work those hours for 500k USD.
- para_parolu 2y agoIf you can share: were these 500k cash or cash +rsu?
- sashank_1509 2y agoI have a friend who joined there with 2 YoE, and got fired in 3 months. He was paid 700k cash + 700k RSU
- saturn8601 2y agoSo in the end did he get anything? I dont know how these things work but did he just walk away with ~50k in pre tax income and 0 for RSU or did Musk pull a Twitter and not even pay him for those months?
- hector126 2y agoIIRC it was cash, but I'm sure others can confirm.
- iooi 2y ago500k isn't FAANG++, it's standard FAANG comp
- hector126 2y agoShould have been more clear, this was 500k for an E4 level role, you're correct that senior/staff at Meta and G are definitely making more.
- thefourthchime 2y agoNo a bad recipe for success.
- ddxv 2y agoTo me, it seemed like they spent there money to get there. They talked about the massive datacenter they built, but will it pay off is the question.
- jiggawatts 2y agoThey may not need direct subscription revenue to recoup their investment. A variant of multi-modal LLMs may be the solution to self-driving cars, home robotics, and more. I keep saying that to be a really effective driver, an AI model will need a theory of mind, which the larger LLMs appear to have. Similarly, any such model will need to be able to do OCR and read arbitrary street signs, and understand what the sign meant. Most modern LLMs can already do this.
- steve_adams_86 2y agoSince when do LLMS appear to possess theory of mind? The papers I've read on this show impressive capabilities, but only within conditions where language-based training would strongly indicate appropriate outputs for the LLM to use. Outside of familiar conditions, or even realistic conditions, they don't do well. It's all still statistical string-building.
- sebzim4500 2y agoCan you give an example prompt that shows a lack of theory of mind? It used to be easy, but now that models have improved I've noticed that the 'LLM doubters' nowadays are forced to make generic, almost spiritual, claims rather than making concrete statements.
- emptysongglass 2y agoThe burden isn't on us, it's on people making claims that LLMs do possess theory of mind. You're the one who has to prove it, not us.
- 2y ago
- littlestymaar 2y agoI don't think anyone who's paid attention to the LLM scene will give them any “credit for catching up fast” as it has been pretty obvious for the past two years that all it takes to reach the state of the art is a big enough GPU cluster. Deepseek made the news because how they were able to do it with significantly less hardware than their American counterparts, but given that Musk has spent the last two years telling everyone how he was building the biggest AI cluster ever, it's no surprise that they manage to reproduce the kind of performances other players are showing.
- dmix 2y agoThis severely underestimates the talent still required. Deepseek didn't come out just because it's cheaper, it came out because a very talented team figured out how to make it cheaper.
- littlestymaar 2y agoFor Deepseek, I'm not saying otherwise, quite the opposite. But Grok hasn't shown anything that suppose the level of talent that Deepseek exhibited.
- jp42 2y agoeven if we assume you are comment is correct. Lets extrapolate what happens next. talented team, biggest compute among all competitors and CEO who is hell bent on winning the race. imo that is the reason it is big deal.
- littlestymaar 2y agoGrok (unlike Deepseek) has yet to show any ability to make conceptual breakthrough. I don't like OpenAI at all but one must admit that they are at least showing that they can move the field forward.