3 ms·
GPT2 is known to be unable to track and bind variables, scaling purely associative models beyond the trivial examples is going to be difficult or more likely im
by Avi-D-coder 6y ago
GPT2 is known to be unable to track and bind variables, scaling purely associative models beyond the trivial examples is going to be difficult or more likely impossible.
This will end up being a better tabnine. Models like GPT2 are still just approximating intelligence, they are not rationally cognizing.
- gradys 6y agoCurious what analysis you're referencing here. While I don't doubt people have shown that various transformer models have certain limitations, I'm pretty bullish on transformer models in general. Here's a post exploring the application of transformers to symbolic mathematics for instance: https://medium.com/analytics-vidhya/solving-differential-equations-with-transformers-21648d3a1695 https://medium.com/analytics-vidhya/solving-differential-equ...