This video from Yann LeCun gives a great summary on where things stand. https://www.youtube.com/watch?v=pd0JmT6rYcI https://www.youtube.com/watch?v=pd0JmT6rYcI

This video from Yann LeCun gives a great summary on where things stand. https://www.youtube.com/watch?v=pd0JmT6rYcI https://www.youtube.com/watch?v=pd0JmT6rYcI He is of the opinion the current generation transformers architecture is flawed and it will take a new generation of models to get close to the hype.