5 ms·
ARM's "Blackhawk" CPU Is a Plan to Have the Best Smartphone CPU Core
- dingi 3y agoBut, will it have mainline support?
- als0 3y agoCPU would, but I think GPU will remain a blob
- DeathArrow 3y agoPerformance also depends on how well the cores integrates with manufacturing processes of TSMC and Samsung.
- brucethemoose2 3y ago> Arm also believes that Blackhawk will provide “great” LLM performance. I will assume that this has to do with big CPU IPC performance improvements as Arm says that its Cortex CPU is the #1 AI target for developers. I didn’t run the developer or app survey myself, but this makes sense to me as most AI inference in the datacenter is run on the CPU. The NPU and GPU can be an efficient way to run AI, but a CPU is the easiest and most pervasive way, which is why developers target it. A higher-performing CPU obviously helps here, but as the world moves increasingly to smaller language models, Arm’s platform with higher-performing CPU and GPU combined with its tightly integrated ML libraries and frameworks will likely result in a more efficient experience on devices. I feel like they are conflating 2 unrelated things? There is "CPU inference" which is indeed pervasive and huge, and there is "generative ai llm inference" which is absolutely anemic on CPU, especially outside of tiny-context toy model demos, in spite of the popularity of llama.cpp and other cpu platforms . I would hope smartphone users aren't running anything but extremely tiny llms on CPU. The IGP is not theoretically, but currently massively more efficient than any 2024 CPU could be.
- adrian17 3y ago> in spite of the popularity of llama.cpp and other cpu platforms Just to clarify: llama.cpp has also added some GPU acceleration over time.
- brucethemoose2 3y agoYes, but not anything usable on IGPs.
- Retr0id 3y ago4-bit quantized llama gets borderline-acceptable performance on an iPad CPU[1], and there's no reason they can't further optimise CPU perf for this task. IIUC, LLM inference is mostly constrained by memory size and bandwidth (hence the aggressive quantization), rather than raw compute. [1] https://github.com/ggerganov/llama.cpp/discussions/844 https://github.com/ggerganov/llama.cpp/discussions/844
- michaelt 3y agoI guess it depends on how you're defining "great" LLM performance and "CPU performance supremacy versus Apple" Personally I would say "borderline-acceptable performance" with a 7B parameter model at 4-bit quantisation is nowhere near "great" or "supremacy". You can fit that on ~7GB of RAM, using your grandpa's 2016-era nvidia 1080. To take the ARM-CPU-LLM-performance crown from Apple you have to beat a fully equipped M2 Max, which provides 96GB of memory with 400GB/s of bandwidth. That's enough to run 70B parameter models in 4-bit quantisation, and it can go toe-to-toe with a dual-nvidia-4090 system that draws a kilowatt of power.
- brucethemoose2 3y agoApple achieves that performance on the IGP. Once the context grows, even M2 Max CPU alone really starts to chug.
- londons_explore 3y agolow bit depth quantized puts GPU's at quite a disadvantage, since they can't use their massively parallel floating point units, and often spend a lot of time packing and unpacking numbers. However, minor tweaks to the design of GPU's could probably make them 5x quicker at directly processing quantized data (a 4 bit multiplier is so small you can afford to do thousands of them per clock cycle in a tiny silicon area), and I'm sure the next generation of GPU's will have that.
- wmf 3y agoThe market for this core will be a lot smaller since Qualcomm will be using Oryon instead. I guess Mediatek will be happy.
- rickdeckard 3y agoRemains to be seen. Oryon is based on the IP Qualcomm gained from the Acquisition of Nuvia, who developed this IP based on a very narrow license they got from ARM to develop for the server market. Qualcomm is trying to transfer everything Nuvia has developed under that specific narrow ARM-license to the broad license of Qualcomm and use it for "powering flagship smartphones, next-generation laptops, and digital cockpits, as well as Advanced Driver Assistance Systems, extended reality and infrastructure networking solutions" ARM has filed a lawsuit that this was never in scope of the license of Nuvia, the court-ruling is still pending on that one... The related court-filing is worth a read: https://s3.documentcloud.org/documents/22273195/arm-v-qualcomm-99-9999-stamped-complaint.pdf https://s3.documentcloud.org/documents/22273195/arm-v-qualco...
- wmf 3y agoObviously the lawsuit has to be settled before Qualcomm can release Oryon but at this point they've invested so much in it I think they will settle.
- rickdeckard 3y agoThe only form of settlement I can imagine is for Qualcomm transferring the Nuvia IP to ARM and be allowed to license it back with favorable conditions. Which would probably lead to it being integrated into a nextgen Blackhawk architecture, merging both paths and making Qualcomm a licensee of Blackhawk. But hey, I'm no lawyer ;)
- wmf 3y agoI predict they'll pay the server royalty that Nuvia negotiatied plus some penalty.
- skavi 3y agoIs this finally what the Sophia-Antipolis team has been working on? IIRC, the last few years have all been Austin.
- webaholic 3y agoARM should gain about 30% in a single gen to be competitive with Apple/Qualcomm processors. It's going to be interesting to see if they can achieve this. Ultimately, competition is good for the consumer. Android phones are so far behind Apple, it is not funny anymore.
- ksec 3y agoThey are not far off. The Cortex X4 Running at 3.3.Ghz on a N4 gets GB6 ~2250. #681.8/Ghz The Apple A17 Running at 3.8 Ghz on a N3 gets GB6 ~2950. #776.3/Ghz On a clock per clock basis, A17 is only about 14% faster. Consider X4 had 15% IPC uplift and resulted in real world 11% performance improvement on GB6. And they are claiming X5 would have the largest YoY IPC uplift, which I think we could consider to be 15%+ or 20%, a Cortex X5 would have similar if not slightly better than A17 performance on a Clock to Clock basis. And it would be good enough for Microsoft / ARM PC.
- mort96 3y ago"The absolute top end Android processors get about 76% of the performance of Apple's processors" isn't the strong argument you may think it is tbh. Per clock performance is uninteresting (it's not like performance is linear with clock speed anyway, and target clock speed is a huge part of the CPU, so I don't even understand why you'd want to adjust for it even in principle in this comparison).
- ksec 3y ago>it's not like performance is linear with clock speed anyway It doesn't scale well beyond certain clock speed, which is why extreme overclocking dont get you a linear performance boost. But at the ranges mobile SoC it is about as linear as it gets. i.e Clocking your SoC from 3Ghz to 3.3Ghz will get somewhere around ~10% improvement in most workload. >and target clock speed is a huge part of the CPU Target clock speed is a huge part of considering for power usage.
- mort96 3y ago
- buildbuildbuild 3y agoIn all seriousness - has anyone noticed a lack of performance on their phone recently?
- sccxy 3y agoYes, my iPhone 14 runs out of memory in safari. It never happens on same web app with 5 year old Samsung...
- boxed 3y agoIn general iPhones have less RAM than roughly corresponding Android device. This is because iOS apps on average take less RAM to run because iOS manages memory more tightly and Android uses Java which is very memory hungry. A web app though should be a situation where the memory usage will be roughly equivalent between the two platforms, so that seems reasonable.
- gumby 3y agoIs Qualcomm actually catching up with Apple on CPU? (I haven’t been tracking it).
- FirmwareBurner 3y agoSeems so. The Snapdragon 8 Gen 3 benchmarks that dropped yesterday with the Samsung S24 Ultra put it on par or above Apple's A17 Pro or whatever their most high end phone SoC is.
- mchanson 3y agoSingle core still lags way behind. Snapdragon 8 Gen 3 has same single core at a 3 year old A15. They are staying even / besting apple in multicore through adding more cores than apple does.
- FirmwareBurner 3y agoTo be fair, Geekbench is a relatively poor representation of real world performance. It's heavily used since it's one of the few benchmarks that works across all major platforms and architectures, making comparisons easy, but not necessarily realistic.
- wyldfire 3y agoThis is regarding reference CPU designs from ARM. Qualcomm plans to use the Oryon cores from their Nuvia acquisition.
- wmf 3y agoYes, Qualcomm Oryon is tied with Apple A17/M3 although it's coming out 9-16 months later.
- phantompeace 3y agoCan we really make confident predictions for chips that are a year out from release?