4 ms·

Google's TurboQuant AI-compression algorithm can reduce LLM memory usage by 6x

9 points by geoffbp 7mo ago

jqpabc123 7mo ago
This cheat sheet is necessary because, as we say all the time, LLMs don’t actually know anything; they can do a good impression of knowing things through the use of vectors In their words, it's not real intelligence --- it's a facsimile thereof that can be convincing to some. This can be useful in some limited cases --- but also risky and dangerous if relied on for decision making.
- hasperdi 7mo ago
  It's "artificial" intelligence
  - jqpabc123 7mo ago
    It's not "intelligence" --- real or artificial. It's "fake intelligence" --- it simulates human understanding by using statistical patterns without actually having any real semantic capability. Fluency without wisdom is the most dangerous form of brilliance. https://www.linkedin.com/pulse/semantic-abyss-we-building-ai-thats-brilliantly-clueless-paul-morley-7exnc/ https://www.linkedin.com/pulse/semantic-abyss-we-building-ai...
LuxBennu 7mo ago
[dead]
psidium 7mo ago
Is it correct to rephrase PolarQuant as “embeddings vectors but as complex numbers” since polar vectors were represented with i complex numbers in other fields of applications? Would this open a new field of transformations using complex number arithmetics? Edit: I believe I have my math concepts all wrong. Vectors are currently represented with xyz coordinates, so that can be our complex number equivalent. This paper saves memory size by ‘simply’ transforming this “complex number” into its polar form (radius/size and angle). Since you use less bytes on angle + radius you then always refer to that polar form instead of the regular vector/complex number part. If I have my concepts correct this idea is simple and genius. Edit 2: I may be missing the forest for the trees here. I’ll try to learn more from the actual sources if I can.
amai 7mo ago
The PolarQuant paper: https://arxiv.org/abs/2502.02617 https://arxiv.org/abs/2502.02617 The TurboQuant paper: https://arxiv.org/abs/2504.19874 https://arxiv.org/abs/2504.19874 These elaborate ideas are not adequately represented in the article.

9 points by geoffbp 7mo ago

This cheat sheet is necessary because, as we say all the time, LLMs don’t actually know anything; they can do a good impression of knowing things through the use of vectors In their words, it's not real intelligence --- it's a facsimile thereof that can be convincing to some. This can be useful in some limited cases --- but also risky and dangerous if relied on for decision making.

hasperdi 7mo ago
It's "artificial" intelligence
- jqpabc123 7mo ago
  It's not "intelligence" --- real or artificial. It's "fake intelligence" --- it simulates human understanding by using statistical patterns without actually having any real semantic capability. Fluency without wisdom is the most dangerous form of brilliance. https://www.linkedin.com/pulse/semantic-abyss-we-building-ai-thats-brilliantly-clueless-paul-morley-7exnc/ https://www.linkedin.com/pulse/semantic-abyss-we-building-ai...

LuxBennu 7mo ago

[dead]

psidium 7mo ago

Is it correct to rephrase PolarQuant as “embeddings vectors but as complex numbers” since polar vectors were represented with i complex numbers in other fields of applications? Would this open a new field of transformations using complex number arithmetics? Edit: I believe I have my math concepts all wrong. Vectors are currently represented with xyz coordinates, so that can be our complex number equivalent. This paper saves memory size by ‘simply’ transforming this “complex number” into its polar form (radius/size and angle). Since you use less bytes on angle + radius you then always refer to that polar form instead of the regular vector/complex number part. If I have my concepts correct this idea is simple and genius. Edit 2: I may be missing the forest for the trees here. I’ll try to learn more from the actual sources if I can.

amai 7mo ago

The PolarQuant paper: https://arxiv.org/abs/2502.02617 https://arxiv.org/abs/2502.02617 The TurboQuant paper: https://arxiv.org/abs/2504.19874 https://arxiv.org/abs/2504.19874 These elaborate ideas are not adequately represented in the article.