3 ms·

Can anybody give an educated guess based on the published pricing or reading between the lines of the report? How big is this model and what did they do differ

• espadrine 4y ago
  My educated guess is that they use a MoE-style model similar to the Switch transformer[0], and combine a similar encoding as that of Kosmos-1[1] (with an “image” latch token, and a ViT-style transformer to process images). As a result, the parameter count is likely bigger, but since not all of them are involved in a forward pass, it is not as meaningful. [0]: https://arxiv.org/pdf/2302.14045.pdf https://arxiv.org/pdf/2302.14045.pdf [1]: https://arxiv.org/pdf/2101.03961.pdf https://arxiv.org/pdf/2101.03961.pdf
• pixel_tracing 4y ago
  You could copy the abstract now and ask GPT4 to explain itself in the style of ELI5