3 ms·
Amazon Elastic Inference – GPU-Powered Deep Learning Inference Acceleration
- jedwhite 8y agoNot sure why this post is getting flagged to oblivion, but it is technically pretty interesting. GPUs remain expensive, and their use is typically prioritized for training, and then inference is run on CPUs. This provides a cost effective way to attach GPU resources on demand to regular instances, rather than having to run dedicated GPU instances.
- thwy12321 8y agoHow is there not more interest in this? This is huge for anyone trying to bootstrap a machine learning business. If you need a 24/7 on prediction service the costs of full fledged GPUs can be prohibitive and a waste. This allows me to distribute my prediction across multiple nodes.
- borramakot 8y agoStrangely, the announcement that AWS is making their own machine learning inference chip got flagged and removed. https://aws.amazon.com/machine-learning/inferentia/ https://aws.amazon.com/machine-learning/inferentia/