amanrs

3 ms·

1. 1.▲
   by amanrs 2y ago

   This is harder than it looks. First "token-healing" doesn't work. Consider the case "app" where the most likely options are "ap|praisal" or "apple|sauce". You can't just sample all tokens th
2. 2.▲
   by amanrs 2y ago

   It edits the file based on the "plan" laid out by a smarter language model
3. 3.▲
   Editing Files at 1000 tokens/s with llama-70B(cursor.com)

   10 points by amanrs 2y ago|4 comments
4. 4.▲
   by amanrs 3y ago

   The key misconception about many quantization methods is that lower precision = better speed. I believe GPT-Q is not much faster than bf16 from skimming the AWQ paper - https://arxiv.org/pdf/2306.00978.pdf It's 3x
5. 5.▲
   I built and shut down a popular text-to-image site in 2 weeks(amansanger.com)

   3 points by amanrs 4y ago|2 comments