Grosvenor

13 ms·

1. 181.▲
   by Grosvenor 2y ago

   .-. (o o) boo! | O \ \ \ `~~~' SEEKING WORK - Data scientist, remote worldwide, email in profile. A colleague and I lost out on a large optimization project (network scheduling during cascading failures) with an
2. 182.▲
   by Grosvenor 2y ago

   \|/ (__) `\------(oo) Moo! || (__) ||w--|| \|/ \|/ SEEKING WORK - Data scientist, remote worldwide, email in profile. A colleague and I lost out on a large optimiza
3. 183.▲
   by Grosvenor 2y ago

   https://news.ycombinator.com/item?id=36871528 Hah. Yes. It looks like they only show up in models with 6.7B parameters or more. The problem can start at 125M. Small enough to test on a whim. So train a model that exhibits t
4. 184.▲
   by Grosvenor 2y ago

   I guess the next step is to see if you're getting those mega activations as he describes. A/B test the two models and compare? Would be interesting to see if these activations only show up on larger models, or they're some re
5. 185.▲
   by Grosvenor 2y ago

   Regular softmax (and attention) has an error in it. softmax should be exp()/1+∑exp() Notice the 1 added to the denominator. The difference is at the negative limit, softmax can be 0, instead of some epsilon. The same could be done by a
6. 186.▲
   by Grosvenor 2y ago

   It turns out boredom sets in after about a week. Come talk to me!
7. 187.▲
   by Grosvenor 2y ago

   They should open up trapping season on them. Apparently their fur is used in fashion. I could use a new winter hat. https://en.wikipedia.org/wiki/Nutria_fur
8. 188.▲
   by Grosvenor 2y ago

   Cool. But doesn't include the training data. So can't reproduce it. :-(
9. 189.▲
   by Grosvenor 2y ago

   SEEKING WORK - remote, email in profile. Location: West coast, remote only. Technologies/techniques: Machine learning/AI, LLM's/RAG/KB's, Transformers, GenAI, MCMC, MCTS, Bayesnets, Neural architecture search,
10. 190.▲
    by Grosvenor 2y ago

    SEEKING WORK - remote, email in profile. Location: West coast, remote only. I'm a data scientist who's worked with companies you know. I'm looking for the next project. I just lost a large project with an airline to a large c