3 ms·
>Press F to doubt. I just tried their best model, Liquid-40B, and it gives some good responses on some questions, but also terrible ones more often than you'd
by moralestapia 2y ago
>Press F to doubt.
I just tried their best model, Liquid-40B, and it gives some good responses on some questions, but also terrible ones more often than you'd wish (GPT-2 level, try it and you'll see).
It's also quite easy to make it become stuck on a loop.
No idea how they scored so high in those benchmarks. Maybe they overfitted on MMLUPro? Lol.
Edit: I just read on some comments here and on the TFA that, apparently, they're not using transformers at all? If that's true, big if, I take my hat off, this is truly a remarkable achievement.
- wret-ai-high 2y agono transformers, from their blog post: "Specifically, our analysis informs model building by improving three key aspects: token-mixing structure (how the operator mixes embeddings in the input sequence), channel-mixing structure (how it mixes channel dimensions), and featurization, responsible for modulating computation based on the input context." https://www.liquid.ai/liquid-foundation-models https://www.liquid.ai/liquid-foundation-models