3 ms·
AI actually has some optimizations unique to the field. You can in fact optimize a model to make it work; not a lot of other disciplines put as much emphasis on
by mxkopy 8mo ago
AI actually has some optimizations unique to the field. You can in fact optimize a model to make it work; not a lot of other disciplines put as much emphasis on this as AI
- tossandthrow 8mo agoCan you list these optimizations?
- mxkopy 8mo agoRLHF is one that comes to mind
- tossandthrow 8mo agoWell, this is an entirely other category of optimizations - not program performance but model performance.
- lucrbvi 8mo agoYes, in "runtime optimization" the model is just a computation graph so we can use a lot of well known tricks from compilation like dead code elimination and co..
- tossandthrow 8mo agoWe are getting closer! What other optimizations are there that can be used than what explicitly falls into the 4 categories that the top commenter here listed out?
- mirekrusin 8mo agoFor inference assorted categories may include vectorization, register allocation, scheduling, lock elision, better algos, changing complexity, better data structures, profile guided specialization, layout/alignment changes, compression, quantization/mixed precision, fused kernels (goes beyond inlining), low rank adapters, sparsity, speculative decoding, parallel/multi token decoding, better sampling, prefill/decode separation, analog computation (why not) etc etc. There is more to it, mentioned 4 categories are not the only ones, they are not even broad categories. If somebody likes broad categories here is good one: "1s and 0s" and you can compute anything you want, there you go – single category for everything. Is it meaningful? Not really.
- tossandthrow 8mo agoThanks!