3 ms·
I didn't know anyone was even suspicious of the reversal "curse". I thought it went viral more because people were surprised anyone was surprised and the tweet
by godelski 3y ago
I didn't know anyone was even suspicious of the reversal "curse". I thought it went viral more because people were surprised anyone was surprised and the tweet was highly sensationalized. It feels more like the __expected__ results considering both speech patter bias __and__ that "causal" attention is sequentially biased. Hell, we saw the same things in RNNs, we see it in classic autoregressive models, and so on. But it definitely isn't how humans encode information because if you had all 3 pieces of knowledge (you know who Tom Cruse is, you know who Mary Pfieffer is, and you know the relation between Mary and Tom is mom/son) then your ability to recall is (nearly) invariant to the ordering. Hell, we're so robust you can caveman speak like "Mary Pfieffer son who!" and still get the answer or caveman yoda speak "Who Mary Pfieffer son is?" Some languages even have these orderings but we're impressively robust (enough that I think we trick ourselves a lot when subtlety comes into play. AKA overfitting).
So I find it weird to call this "a feature" and also act like this is surprising.
But can I also take a minute to just say I really hate GPT experiments? They're performed on a stochastic model, with proprietary weights, proprietary training data, proprietary training methods, and above all is constantly changing and at a rather fast pace. It makes for a very non-scientific process as you can't decouple a lot of important factors and reproduction is a crap shoot. It is not a very good way to go about studying "how __LLMs__ work" and is rather "how does GPT work at this particular moment in time and aggregating all these unknowns?" There's some bitter sweetness because I do like that GPT is free but it feels like a major edge that they have is that the community just does a lot of free research for them and in ways where they can better interpret results than the people who performed the experiments in the first place. I really do believe that academic works shouldn't be focusing on proprietary and dynamic methods. It's fine to include them in results (and preferably added post review to avoid identity spoilage (or we openly acknowledge that double blind is a joke)) but I'd rather most researchers focusing on the general concepts and with the ability to dive down the rabbit hole than playing a wack-a-mole game.
Also, I'd totally love it if more research papers were instead blog posts. Kudos to anyone posting their research on blogs, academic or not (I don't care about your creds, your real creds are your work). Papers are about communicating with fellow scientists, right? Why do we need journals and conferences these days?