lambdaviking

3 ms·

1. 1.▲
   by lambdaviking 3y ago

   > How do the results of your work inform these predictions that stack-more-layers token-prediction scaling will keep unlocking new abilities? The short answer is that scaling up alone probably won't fix these shortcomings. Better de
2. 2.▲
   by lambdaviking 3y ago

   Good question! A key use for this type of analysis is that it acts as an upper bound on transformers, rather just demonstrating "transformers are like logic". That is, it allows us to fairly easily identify problems that transform
3. 3.▲
   by lambdaviking 3y ago

   Our recent paper proves that transformers can be translated to sentences in first-order logic with majority-vote quantifiers (FOM). FOM is a symbolic language that can capture computation inside transformers!
4. 4.▲
   A Logic for Expressing Transformers(arxiv.org)

   2 points by lambdaviking 3y ago|6 comments