2 ms·

They have pretty much admitted that GPT4 is a bunch of 3.5s in a trenchcoat.

• whimsicalism 2y ago
  They have not. You probably read "MoE" and some pop article about what that means without having any clue.
  • matsemann 2y ago
    If you know better it would be nice of you to provide the correct information, and not just refute things.
    • whimsicalism 2y ago
      gpt-4 is a sparse MoE model with ~1.2T params. this is all public knowledge and immediately precludes the two previous commentators assertions