3 ms·
A nice estimate! Since „you can compress knowledge, but not factual knowledge” https://x.com/bojie_li/status/2049314403208896521 https://x.com/bojie_li/status/2
by stared 5mo ago
A nice estimate! Since „you can compress knowledge, but not factual knowledge” https://x.com/bojie_li/status/2049314403208896521 https://x.com/bojie_li/status/2049314403208896521, it is likely we can actualy measure its size.
- stared 5mo agoI tried to run it, but estimate is 24–33T parameters, vide https://gist.github.com/stared/a86d7380937e6d0ab7920014866ace4c https://gist.github.com/stared/a86d7380937e6d0ab7920014866ac.... It seems to be a huge overshot, vide Hy3 model, which this model claims to be 2.4T, while it is 295B.