3 ms·
Is there anything preventing designing a custom ASIC for LLM training / inference rather relying on using GPU's ? I've been away from hardware for a long time,
by creativeSlumber 2y ago
Is there anything preventing designing a custom ASIC for LLM training / inference rather relying on using GPU's ? I've been away from hardware for a long time, but if you can compile it into run on an FPGA, then a custom ASIC should be even faster (clock speed) and more power efficient.
- Aromasin 2y agoYou absolutely can and many do. The issue is GPUs are a commodity product that get most of the way there in terms of performance without rewriting a whole software stack, and have huge economies of scale. You just buy millions of pre-existing products that do the job well enough, and dont become obselete when the new hot stuff of the year comes around because theyre generalised parallel processing units. As the industry become more mature we'll see more targetted inference/training engines come to the market, but the semiconductor industry works in decades not years. The reason FPGAs work as an in-between product is because they are also a commodity product. All you need is a team to program the device. You can buy a million of them off the shelf right now, and there's a hundred other applications they're good for so you're not restricted when the time comes that your ASIC is no longer needed for whatever task it's explicitly designed for.