4 ms·
LLMLingua uses a well-trained small language model after alignment, such as GPT2-small or LLaMA-7B, to detect the unimportant tokens in the prompt and enable in
by TarqDirtyToMe 3y ago
LLMLingua uses a well-trained small language model after alignment, such as GPT2-small or LLaMA-7B, to detect the unimportant tokens in the prompt and enable inference with the compressed prompt in black-box LLMs, achieving up to 20x compression with minimal performance loss.
- cyanydeez 3y ago“Why waste time say lot word when few word do trick” -Kevin Malone
- deleted 3y ago[deleted]
- arthurcolle 3y agoPerfection. Key insight. "Few Word [is] All Need" (with a robust enough foundation model) Linked for the culture: https://www.youtube.com/watch?v=bctjSvn-OC8&t=4s https://www.youtube.com/watch?v=bctjSvn-OC8&t=4s Sleep big last night
- cyanydeez 3y ago"Kevin, are you saying 'See the World' or Sea World?" -- Jim
- behnamoh 3y agoCame here to mention this. Whenever I hear "alignment" I immediately say "No way am I going to use that shit". Seriously, there's alignment and then there's censorship—the AI creators are using the former when they actually mean the latter. This needs to stop.
- TarqDirtyToMe 3y agoMy understanding is that in an academic context you’ll hear alignment anytime a model is tuned to accomplish a certain task, not just to steer its political affiliation and idea of ethics I don’t think this models use of alignment implies any sort of censorship, it’s just being tuned to accomplish the task of outputting only important tokens for the target llm
- behnamoh 3y ago[flagged]
- TarqDirtyToMe 3y agoI’m really not all that familiar with the space so I could be mistaking. The definition of ai alignment on Wikipedia says an aligned model one that “advances intended objectives”. In the paper, “distribution alignment” is one of methods used to improve the results of compression so intent is preserved: > To narrow the gap between the distribution of the LLM and that of the small language model used for prompt compression, here we align the two distributions via instruction tuning So in any case for this paper alignment seems to be used in very specific way that doesn’t seem related to censorship Edit: would to love to hear from someone who has a better understanding of the paper to clarify. I am operating from the position of layman here
- ryanklee 3y agoIt is common, standard usage precisely in this context.
- smeagull 3y agoIn my experience it means the AI will waste tokens apologizing for it's short comings and ignoring task prompts in favour of it's alignment.
- TarqDirtyToMe 3y agoThis does not seem relevant to the alignment discussed in the paper. It seems to be explicitly out of scope: > The potential harmful, false or biased responses using the compressed prompts would likely be unchanged. Thus using LLMLingua has no inherent benefits or risks when it comes to those types of responsible AI issues.
- nathan_compton 3y agoIt amazes me that this amazing new technology comes out and there is a group of people who are like "NO, NOT IF IT CAN'T TELL RACIST JOKES!" I agree that like "tone" alignment is silly and pointless for models in the public domain, but if I were a big company who wanted to keep customers I'd align my models this way. It isn't censorship, its marketing.
- sroussey 3y agoWhat would happen if instead of the long prompt, you just sent the mean of the embeddings of the prompt tokens?