4 ms·
No, this paper was edited yesterday. The original (you can verify on arxiv) contained this incredible section: "6.10 Optimised Routing and Pruning Operations (O
by qeternity 2y ago
No, this paper was edited yesterday. The original (you can verify on arxiv) contained this incredible section: "6.10 Optimised Routing and Pruning Operations (ORPO)"
The actual ORPO paper is "Odds Ratio Preference Optimisation" and it has nothing to do with pruning. This goes way beyond native language preference.
- cubefox 2y agoWow, so significant parts of the paper could still be LLM confabulation.
- espadrine 2y agoIt takes no time at all to find other major mistakes. For instance, the Mixtral diagram § 6.6.1 shows a single router that selects separate 32-layer transformers. Instead, Mixtral has one router per layer (inside of each block), and it doesn’t select a transformer block: it selects a feedforward.