4 ms·
Doesn’t training need higher precision to avoid getting stuck at local minima, at least with back propagation style learning? Maybe something alternate like ev
by api 3y ago
Doesn’t training need higher precision to avoid getting stuck at local minima, at least with back propagation style learning?
Maybe something alternate like evolutionary algorithms could work in a domain like this, but so far those have proven to be less efficient.
- sp332 3y agoA recent paper used a single ternary "trit" ~1.5 bits per parameter for training. https://news.ycombinator.com/item?id=39535800 https://news.ycombinator.com/item?id=39535800 They said it would be difficult to apply this technique to pre-trained models and had to be trained in 1-trit from scratch.
- gliptic 3y agoBut it's using larger weights during training, not just the trits.