They're still extremely effective as a fuzzy search engine and translator (e.g., I can write out a function in plain English and get something that would have t

They're still extremely effective as a fuzzy search engine and translator (e.g., I can write out a function in plain English and get something that would have taken me a minute in 5 seconds). Additionally (more importantly), the most magical part is still the architecture: there seems to be no end to the expressivity of transformers as long as you can pour in more compute. This can be extremely powerful and lead to more generalizable reasoning given the right learning objective (e.g., pose tasks as an RL Markov Decision Processes where state=text, actions=tokens, rewards=performance of generated code/math/language).