4 ms·
By time, they’re talking about the writing style of a specific time period. Feels like a click bait title. Of course language model weights encode different wr
by alephnan 3y ago
By time, they’re talking about the writing style of a specific time period.
Feels like a click bait title. Of course language model weights encode different writing styles. The fact that you can lift out a vector to stylize writing is also more interesting, but that’s also nothing newly discovered here. It should be obvious that this is possible given that you can prompt ChatGPT to change its writing style.
- cush 3y agoWhy would it pertain only to writing style?
- n2d4 3y agoBesides what the sibling comment said, what's most interesting (imo) is that you can manipulate the vectors like that. The fact that you can average the vectors for January and March, and get better results for February, is pretty surprising to me.
- macleginn 3y agoThis also generalises: https://arxiv.org/abs/2302.04863 https://arxiv.org/abs/2302.04863
- jimbobthrowawy 3y agoGeneralizing vectors in generative models seems like an incredibly useful thing to know about, if you want to use them more effectively. Blew my mind when I saw someone demonstrate doing vector math on a GAN a couple years back to move an "input image" around the space of outputs. Maybe this could be useful for singling out post-LLM text and generating output that excludes it.
- k__ 3y agoInteresting that writing style works, but other reflective actions don't. Like, "only use the the 2000 most common words of the English language" or "the response should be 500 words long".
- n2d4 3y agoIt does work on other reflective actions, parent is just wrong; in the paper, they specifically run the experiment on a dataset of political affiliation over time
- mycall 3y agoFrom the title, I was thinking "of course the neural network of the LLM is a [cause-effect] sequence of words" thus time is encoded in each connection.