4 ms·

The paper has a “preliminary scaling law” diagram. The shape of the graph is the same, but with 20% fewer FLOPS. The real breakthrough is that Hyena apparently

by bckr 4y ago

The paper has a “preliminary scaling law” diagram. The shape of the graph is the same, but with 20% fewer FLOPS. The real breakthrough is that Hyena apparently has an unlimited context window.

flangola7 4y ago
>The real breakthrough is that Hyena apparently has an unlimited context window. It's extrapolated volition time (ﾟ∀ﾟ)
- te0006 4y ago
  Is it? Removing the context window limit is big, no doubt, but inference still takes time (and compute).
  - bckr 4y ago
    I think GP is talking about the ability of an AI to make decisions with reference to context from the past and therefore have a “will extended over time”
    - flangola7 4y ago
      Not quite https://intelligence.org/files/CEV.pdf https://intelligence.org/files/CEV.pdf