3 ms·
The MatFormer is clearly called out as a different aspect of the model design. PLE is much more likely to be a reference to the Per-Layer Embeddings paper that
by yorwba 1y ago
The MatFormer is clearly called out as a different aspect of the model design.
PLE is much more likely to be a reference to the Per-Layer Embeddings paper that will be published in the future once it doesn't give away any secret sauce anymore.
- andy12_ 1y agoI thought the same, but Per-Layer Embeddings as a name doesn't make sense in any context, and MatFormer does exactly what the blogpost says PLE does. I just think it's more probable that the blogpost was written by several authors and that noone bothered to check the final result.