3 ms·
This confuses AI training with inference. Training a machine learning model is expensive, complex, and requires data. Once trained, inference is cheap in terms
by andrewjl 4y ago
This confuses AI training with inference. Training a machine learning model is expensive, complex, and requires data. Once trained, inference is cheap in terms of compute. A trained model is trivial to leak so hopefully nobody is putting much stock in their models staying secret.
Not to mention there are techniques by which training can be avoided entirely like transfer learning and others.
- potatoman22 4y agoInference is not cheap -- SOTA LLMs require 100s of gigabytes of VRAM for inference.
- andrewjl 4y agoApple has made a sample project available for Stable Diffusion that can run on both macOS and iOS.[1] The sample code is here.[2] I'll concede that Stable Diffusion may or may not meet the threshold for SOTA but still think this is indicative of inference eventually becoming supported for any compelling LLM on consumer-grade hardware. The possibilities for creative tools are just too vast. [1] https://machinelearning.apple.com/research/stable-diffusion-coreml-apple-silicon https://machinelearning.apple.com/research/stable-diffusion-... [2] https://github.com/apple/ml-stable-diffusion https://github.com/apple/ml-stable-diffusion