10 ms·
Some people may criticise the "monopoly" Apply have of TSMC latest nodes, but I think it's ultimately good. TSMC need to know they have a customer for new nodes
by samwillis 3y ago
Some people may criticise the "monopoly" Apply have of TSMC latest nodes, but I think it's ultimately good. TSMC need to know they have a customer for new nodes in order to invest in them, thats what Apple are in the unique position to give with their war chest of cash. This investment by Apple eventually leads its way to other customers and smaller businesses. The whole industry thrives on the wake Apple leaves behind.
As a complete other aside, I'm excited to see what comes with the M3, particularly any neural cores. I would be much more excited about all the new AI tools if they could be run locally, Apple are again in the unique position to be able to make that possible for the masses. WWDC is hopefully going to be super interesting.
What we need is a sudden and massive increase in memory on these chips to make having LLMs viable on everyday affordable devices. I do wander if that may be something Apple surprise us with.
- ftxbro 3y agoIt looks like the local-LLM enthusiasts will want to use Apple computers. This is funny to me for some reason, like if the most cost-effective work boot of 2025 is produced by Gucci.
- PragmaticPulp 3y agoLocal LLMs do best on big GPUs, which aren’t an option on Macs. What Apple has done with the M-series silicon is really impressive, but it’s not as fast as big, dedicated GPU silicon. > like if the most cost-effective work boot of 2025 is produced by Gucci. Apple M-silicon laptops are high quality, but they’re not the same as overpriced luxury goods. The price of entry level M1/M2 Macs is extremely reasonable, IMO.
- daniel-thompson 3y ago> Local LLMs do best on big GPUs, which aren’t an option on Macs. For processing speed, Ms are fast but I agree not anywhere near a top Nvdia chip. For memory size, the memory on an M chip can be used as graphics memory, so a person could get an M2 today with 128GB of graphics memory for ~ $5k. Not bad considering an Nvidia chip approaching that memory size is several times that much.
- deleted 3y ago[deleted]
- samwillis 3y ago> it’s not as fast as big, dedicated GPU silicon It's not yet. Apples combined memory architecture enables them to significantly cut the cost of bundling more memory for various GPU, Neural or other domain specific cores. I believe they will catch up or even overtake Nvidia as the leading AI platform. Desktop AI class GPUs are hell expensive. If Apple can get something 50% as performant, but in an iPhone Pro or MacBook Air, thats going to change so very much.
- gpderetta 3y agobut AI class GPUs are expensive because NVidia can pretty much ask whatever it wants as it has no competition. If Apple were to become competitive in the area, things can change.
- xipix 3y agoIt's so funny to see Apple's "combined memory architecture" praised. When Intel did this a decade+ ago with its integrated GPUs, it was roundly slated.
- comex 3y agoWell, the difference is that compared to a baseline where GPUs typically get faster RAM than CPUs, Apple is providing fast RAM to both, whereas Intel provided slow RAM to both. Of course, that’s hardly Intel’s fault given that their integrated GPUs are aimed at the low end and are usually paired with external socketed RAM.
- Maursault 3y ago> Local LLMs do best on big GPUs, which aren’t an option on Macs. What Apple has done with the M-series silicon is really impressive, but it’s not as fast as big, dedicated GPU silicon. It'd be pretty silly for Apple to cram in big, dedicated GPU when 99.9% of their customers don't care and don't want to pay for it, especially considering that anyone that does want big, dedicated GPU can outboard as much GPU as they want.[1] And many seem to think that the onboard GPU along with Neural Engine should be adequate for local LLM. [1] https://support.apple.com/en-us/HT208544 https://support.apple.com/en-us/HT208544
- thoughtsimple 3y agoOnly on Intel Macs which are going away soon (only 2019 Mac Pro is remaining outside of refurbished.) Apple Silicon doesn't currently have any provision for external graphics cards.
- Maursault 3y agoThere are two issues. 1) Apple Silicon won't support PCIe. 2) Apple doesn't want it to. #2 means Apple is taking nVidia and AMD head on in the GPU space. Apple wants to control everything, and allowing these competitors on their platform is giving away too much. Because Apple Silicon scales better than competitors' hw, the desire for third party GPU is probably going to evaporate within a few generations of Apple Silicon. I mean, we'll see, but that is my best guess, because it seems like that was an intentional decision rather than oversight.
- bhj 3y agoAS supports Thunderbolt just fine. Isn't the lack of eGPU support due to lack of ARM drivers for them?
- Maursault 3y agoTo suggest it is merely lack of drivers is an oversimplification. There is a chip errata that prevents PCIe GPUs from working properly on Apple Silicon: the architectures are not compatible. Apple Silicon GPU drivers are deeply integrated into the system. Due to this integration, only graphics cards that use the same GPU architecture as Apple Silicon could be supported, and there just aren't any, and I don't see how there could be unless Apple developed one and released it.
- dacryn 3y agothey share their main memory with the GPU. There is no cheaper way to have 50gb of memory allocated to your GPU compute pipeline (buy a 64gb macbook pro or studio). So yes big, only the fast aspect remains a reason to buy expensive dedicated gpu's
- diffeomorphism 3y ago> The price of entry level M1/M2 Macs is extremely reasonable, IMO. Caveats: In the US, with low ram and low storage. Without these caveats you look at about 2000€ or so (m2 macbook air with 16gb of ram and non-halved storage performance in Germany). That can still be a decent deal for what you get, but "extremely reasonable" -- not quite.
- nordsieck 3y agoIt is a little funny... but the combination of pretty good integrated graphics and (potentially) large amounts of on package ram mean that macs are actually a pretty inexpensive way to get access to a GPU with lots of graphics memory.
- braymundo 3y agoI wish gaming on Mac became more popular due to the same reason.
- heliophobicdude 3y agoWhat are some broken-down reasons why gaming on Mac didn't take off. I'm sure it's a few different reasons all together
- braymundo 3y agoI don't know, but if I had to guess it would that Apple was never really interested in it or in supporting it. I fondly remember some companies (like Blizzard) who would always support Macs, until they stopped (looking at you, Diablo 4).
- dagmx 3y agoA lot of people will likely blame APIs etc, but that’s just an easy scapegoat for people outside the game dev industry. The reality is likely just market share, hardware and the makeup of that market share. Macs used to actually be a big gaming platform once upon a time. Myst was a Mac exclusive (built with HyperCard) and Bungie used to make games for macs first before getting bought by Microsoft. But windows got the upper edge: 1. Much larger slice of the market 2. More of the market is made up of gaming enthusiasts than Macs which are usually either targeting light use, education or professional. 3. You can make affordable gaming PCs with windows. You can’t really do it with any kind of Mac because there’s no product for that market between the Mac Mini and iMac (which were targeted at more casual users) up to the Mac Pro and iMac Pro (which were targeted at professional users) It’s easier to target the biggest piece of the pie and the one that will buy your products. That’s why the “API” reason never makes sense, because iOS is THE dominant gaming platform despite having the same APIs as a mac. Game engines often need to support multiple graphics APIs anyway. So Mac gaming did take off, but it fumbled and never recovered. Part of that is that Apple themselves fumbled as a company, on the verge of bankruptcy before the return of Jobs. What made them successful post his return is also what killed gaming on their platforms: they targeted a different audience.
- Gare 3y agoApple was cost-effective most of the time if you wanted high-quality tools.
- foldr 3y agoIf you care about the combination of portability, battery life, performance, and display+audio quality, then Apple’s laptops have been price competitive for quite a while now. People who think they are overpriced usually don’t care about at least one of these things.
- KaoruAoiShiho 3y agoI'm not aware of any local-LLM framework that works with Apple GPUs?
- Maursault 3y agoDoes this qualify?[1] [1] https://www.infoq.com/news/2023/05/mlc-llm-mobile-laptops/ https://www.infoq.com/news/2023/05/mlc-llm-mobile-laptops/
- raxxorraxor 3y agoI think it is least likely that we will run local AI instances on Apple hardware to be honest. But yes, the comparison would be apt. I think people want to solve a problem with dependence on third party services... No, I expect GPUs with insane amount of memory surfacing and that isn't really a field for Apple yet.
- zitterbewegung 3y agoNot many laptops have 64 gb of RAM that can run llama.cpp https://gist.github.com/zitterbewegung/4787e42617aa0be6019c3db526d5d6fd https://gist.github.com/zitterbewegung/4787e42617aa0be6019c3...
- ac29 3y agoPlenty of non-Apple laptops support 64GB RAM. For example, both the AMD and Intel variants of the Framework Laptop can take 64GB. All of of the large vendors have laptops that support a 64GB configuration too (HP, Dell, Lenovo).
- hammock 3y ago>Apple are in the unique position to give with their war chest of cash Don’t all the big tech companies have giant cash accounts?
- JKCalhoun 3y agoIf I'm googling this right Apple had perhaps $48B in cash in 2022. I don't know how others compare.
- eloisant 3y agoGoog = $110B MS = $100B Amazon = $64B Meta = $37B So no, Apple is not the big tech with the most cash. They do have a much higher market cap than the others but it's simply based on their brand image and speculation.
- daniel_iversen 3y agoAnd revenue! i.e. 2022: Apple: ~$395bn Microsoft: ~$200bn Amazon: ~$500bn Meta: $116bn
- adventured 3y agoRevenue is a very poor reference for the context. Amazon extracts a poor margin out of their $514 billion in sales. Over the past four quarters their operating income margin is a pathetic ~2.4%. Apple's operating income margin is ~29.4% by comparison. They have a vastly superior business. Eventually Amazon investors will tire of the terrible stock market returns that present Amazon is generating and the garbage that is retail and they'll demand the company split itself up. That will involve spinning off AWS so investors can realize max value for that (before it's too late and growth slows to a crawl in that space).
- hammock 3y agoI look at FCF. Amazon -11B last year vs Apple +111B. Gives you an approximate run rate for their cash on hand
- reacharavindh 3y agoI enjoy the competition that Apple brings to the computing market with M* chips. As an innocent bystander in the world of AI, neural networks etc, I wonder if Apple Silicon and its neural cores would become the proprietary brake like CUDA is for ML work. That is if they do something phenomenal with the 3nm facility and bring about some hardware that only they can do.. I'd rather have a healthy competetive ecosystem where people writing software need not say stuff like "needs CUDA to work or needs Apple Neural cores to work" and instead says something like "need XYZ acceleration provided by CUDA cores, Apple Silicon, and AMD blah.."
- sharikous 3y agoWhat about WebGPU? The only thing that I find unsatisfactory about it is that float64 (double) is not a standard type in WebGPU and it's not supported on Apple GPUs anyway.
- audunw 3y agoI think it's too early for AI/ML acceleration to standardize on an API. It takes a long time to figure out what abstractions we should standardize on before we can make a universal API. Consider the whole history of OpenGL, Metal, Vulcan, WebGL, WebGPU... It would have been a mistake to keep using OpenGL, and WebGL was perhaps a mistake from the beginning. It was the wrong abstractions. Metal and Vulcan are clearly better abstractions for GPU APIs, and it really doesn't matter too much that Metal is Apple-only because they're close enough that you can have a good Vulcan API on top of Metal (MoltenVK). That is, what matters is that we converge on the same abstractions, and those take a long time to get right. Now in the end we've gotten WebGPU (which isn't only for web browsers btw, can be used from native apps too) that can provide a nice universal cross-platform API.
- klodolph 3y agoA bit harsh to say that WebGL was a mistake from the beginning. The spec was released in 2011, along with some initial implementations. At the time, Vulkan didn't exist (2016). Not even Metal (2014) or Mantle (2013) existed yet. I don't think even the AZDO folks had done their work yet. WebGL looked pretty damn good in 2011. It was based on OpenGL ES 2.0, which was the latest version of OpenGL ES, and the great thing about OpenGL ES is that it's a little cleaned up from OpenGL and reflects the capabilities of a broad class of graphics implementations.
- JKCalhoun 3y agoI suppose in some other timeline Apple funds/owns their own fab and, to the points you raise, that would be worse for everyone outside of Apple.
- michaelsshaw 3y agoNobody really benefits when the leading process node is being wasted by a walled garden company on, mostly, consumer laptops
- natdempk 3y agoThe new nodes also typically go towards iPhones and tablets as well, and in total that's only hundreds of millions of people benefitting from improved devices, so yeah pretty much nobody.
- michaelsshaw 3y agoHow improved though? iPhones are already pretty fast. I don't know about you, but i've never compiled a kernel on my phone. Having a faster, actual workstation grade computer would be better. But yeah you're right, sarcasm is the right approach.
- madeofpalk 3y agoThere's also energy consumption.
- kitsunesoba 3y agoYep, reducing energy consumption and boosting performance per watt seems to be one of Apple’s main focuses, and for most people buying phones, tablets, and laptops that’s probably the correct choice — while raw perf is nice computers have been “fast enough” for a long time. People who only browse the web need barely more than a Core 2 Duo, even now, and most devs and media professionals will find an M1 Pro to fill their needs somewhere between reasonably and very well. You know what almost nobody will turn down though? Laptops that are cool to the touch and rarely need to spin up fans and all manner of battery powered device that lasts longer, even doing “real” work. Of course there will always be those who need the fastest big iron they can find and don’t care if they have to feed it 2000W and cool it a 15lb heatsink with ear piercing fans, but this group is comparatively niche.
- 3y ago
- nordsieck 3y ago> What we need is a sudden and massive increase in memory on these chips to make having LLMs viable on everyday affordable devices. I do wander if that may be something Apple surprise us with. Honestly, I just wish that they'd refresh the Mac Studio. I know that probably won't happen in any processor generation which they plan to roll out the Mac Pro, though.
- apozem 3y agoExactly. TSMC can spend tens of billions of dollars on new processes because they know Apple will be there day 1 to use it all. The iPhone and Mac have good performance and power efficiency, but people will always want their phone and laptop to do more and get longer battery life. Apple customers’ appetite for compute is practically bottomless.
- reisse 3y agoNot for compute, but for performance per watt. Raw compute is much better served by Intel/AMD + Nvidia workstation.
- MichaelZuo 3y ago> Apple customers’ appetite for compute is practically bottomless. I don't think so, the typical iPhone user has not scaled up their compute needs much past the 2015 A9. And if you exclude the computational photography stuff, I could imagine some users don't take advantage of it at all.
- darkr 3y agoPerformance per watt is the new Moore's law - though most users don't need more power, they do appreciate longer battery lives
- ytch 3y agoIt's just my (unprofessional) assumption, but besides CPU and GPU, maybe NPU (AI) still have room for improvement.
- msh 3y agoI think that the games on my iphone would disagree.
- MichaelZuo 3y agoGenshin Impact runs on the A9, I don't think there exist any more computationally demanding games on mobile. What games are you referring to?
- epolanski 3y agoI don't think those criticism are valid. Apple has insane order numbers, competitors do not. It's only natural they prioritize who gives them more business.
- deleted 3y ago[deleted]
- foverzar 3y ago> TSMC need to know they have a customer for new nodes in order to invest in them I mean, does anyone doubt that thinner process nodes will be in demand?
- samwillis 3y agoNo, but companies still like to have preorders to pay for the investment, rather than having to empty their savings to raise cash. Apple with the cash they have thrown at TSMC and the almost certain high sales they know they will have, can make pre-emptive moves other companies will struggle to do.
- raxxorraxor 3y agoHow are we to proceed if we reach the size of a single atomic layer which is around 0.5nm? Sure, 3nm is marketing too, but there are hard limits and reliability might suffer if manufacturing cannot keep up.
- nabakin 3y agoI'm excited for the M3 too. I hope this is the generation Apple adds AV1 hardware support. If they don't, I have no confidence they will ever add support
- hulitu 3y ago> Some people may criticise the "monopoly" Apply have of TSMC latest nodes, but I think it's ultimately good. And if Apple leaves TSMC will be left with the pants down.
- thomastjeffery 3y agoDo you have any evidence whatsoever that TSMC would struggle financially without Apple? Sounds like FUD to me.
- jryle70 3y agoI don't think they will be struggling, just that Apple is willing to pop up cash, or have clear commitment in advance. Businesses prefer clear and stable outlook.
- thomastjeffery 3y agoSo, "no".
- Veliladon 3y agoYou would be correct if investment were the bottleneck. It's not. The bottleneck on advanced processes is how fast ASML can build its EUV fabrication machines and they're extremely complicated pieces of kit with lots of specialized parts that are also extremely complicated pieces of kit. Even with all our modern production ability we're limited in how quickly we can assemble these machines that can actually do the lithography and everybody wants them. ASML currently has a backlog of 100 machines. Intel, Samsung, TSMC, they all want them. They sell for $200m+ a piece. Everyone has the cash for them. ASML just can't produce them fast enough.
- bhouston 3y agoWhy can't ASML scale? They are siting on $20B of orders. That is insane. What is ASML's delivery throughput of these machines?
- notaustinpowers 3y agoI'm not familiar on the throughput, but their EUV lithography machines are some of the most complex machinery humanity has developed to date. We're lucky this even works, let alone dream of scaling it.
- hoorible 3y agoMy understanding is that it could, but these machines take a really long time to build. Probably a simplification, but it’s likely not unfair to say that by the time they scaled 3nm production, we’d already have moved on to 2nm, etc Asianometry YouTube channel did a few really good videos on chip fabbing and the unique challenges. Scale is addressed IIRC. Highly recommend.
- Veliladon 3y agoThey make about 50 a year. One of the parts in these machines is a 30cm mirror that is the flattest that humans can make with our current level of technology. If you have a way to churn them out please let Zeiss SMT know.
- 3y ago