I'm pretty torn when to self-host AI 70 B models. The market seems to be evolving fast. Outside of Apple, this is the first product to really compete in this ca

I'm pretty torn when to self-host AI 70 B models. The market seems to be evolving fast. Outside of Apple, this is the first product to really compete in this category Self-host AI. So... I think a second generation will be significantly better than what's currently on offer today. Rationale below... For a max spec processor with ram at $2,000, this seems like a decent deal given today's market. However, this might age very fast for three reasons. Reason 1: LPDDR6 may debut in the next year or two this could bring massive improvements to memory bandwidth and capacity for soldered on memory. LPDDR6 vs LPDDR5 - Data bus width - 24 bits, 16 bits Burst length - 24 bits, 15 bits Memory bandwidth - Up to 38.4 GB/s, Up to 6.7 GB/s - Camm ram may or may not be maintain signal integrity as memory bandwidth increases. Until I see it implemented for a AI use-case in a cost-effective manner, I am skeptical. Reason 2: - It's a laptop chip with limited PCI lanes and reduced power envelope. Theoretically, a desktop chip could have better performance, more lanes, socketable (Although, I don't think I've seen a socketed CPU with soldered RAM) Reason 3: In addition, what does hardware look like being repurposed in the future compared to alternatives? - Unlike desktop or server counterparts which can have a higher cpu core count, PCEe/IO Expansion, this processor with its motherboard is limited on re-purposing later down the line as a server to self-host other software besides AI. I suppose could be turned into a overkill, NAS with ZFS and HBA Single Controller Card in new case. - Buying into the framework desktop is pretty limited based on the form factor. Next generation might be able to include a 16x slot fully populated, a 10G nic. That seems about it if they're going to maintain the backward compatibility philosophy given the case form factor.