Most inference can be easily abstracted over multiple ISV implementations so the inference side rarely is locked in. It’s usually the training side or software

Most inference can be easily abstracted over multiple ISV implementations so the inference side rarely is locked in. It’s usually the training side or software above the inference layer that’s locked in. But as to your question as to why: silicon size and frequency of use. You’re almost always going to have a high ratio of inference to training. Silicon for training needs to be much more capable due to how intensive it is. That’s then a block of silicon that isn’t used to its full potential, takes up valuable space and in turn jacks up cost to buy and cost to run (energy). It’s simply not a worthwhile trade off.