4 ms·
There's no free lunch™, so from what I can tell there's some pathway loss here. E.g. some Jacobi trajectories definitionally exclude higher temperature paths. W
by dvt 2y ago
There's no free lunch™, so from what I can tell there's some pathway loss here. E.g. some Jacobi trajectories definitionally exclude higher temperature paths. Which might actually be a positive given data retrieval (but a negative if we want to maximize for creativity?).
- wrsh07 2y agoThere are better and worse algorithms. I'm not sure "there is no free lunch" always applies in a particularly meaningful way. Some things aren't on the pareto frontier.
- factormeta 2y agoKinda like the aiff -> mp3 conversion process. A lot of data is lost, but we human can really tell the too much of a difference?
- wrsh07 2y agoThere's no reason to think the current next token prediction models are optimal for predicting sentences (they aren't!) > An algorithm may outperform another on a problem when neither is specialized to the problem https://en.m.wikipedia.org/wiki/No_free_lunch_in_search_and_optimization https://en.m.wikipedia.org/wiki/No_free_lunch_in_search_and_...
- stkdump 2y agoI would go even further and say there isn't any indication that we are even close to what is possible. My subjective feeling is that with the current rate of progress it is entirely possible that we will have GPT-4 level performance locally on smartphone hardware within 3-10 years (unless companies decide again that they don't want to give this kind of power away)
- naasking 2y agoProbably. Advancements in ML algorithms, like this one, have been outpacing advancements in hardware for awhile now, so both are converging on making ML faster and ubiquitous.