2 ms·
Right? Then you get some strange ones where parts of whole words are generated by different experts. Makes me think that there’s room for improvement in the e
by dartos 2y ago
Right?
Then you get some strange ones where parts of whole words are generated by different experts.
Makes me think that there’s room for improvement in the expert selection machinery, but I don’t know enough about it to speculate.
- zo1 2y agoHonestly, looks like someone throwing spaghetti on a wall a billion times and seeing what sticks, then training the throwing arm to somehow minimize something. I get that LLM magic is kinda magic and is doing some cool stuff, but this looks like it's just chaos and statistical untangling that happens to minimize some random fitness function X-levels down the line.