naasking

17 ms·

211.
by naasking 6mo ago
Yes, Vulkan is currently faster due to some ROCm regressions: https://github.com/ROCm/ROCm/issues/5805#issuecomment-414161... ROCm should be faster in the end, if they ever fix those issues.
212.
by naasking 6mo ago
From what I understand, ROCm is a lot buggier and has some performance regressions on a lot of GPUs in the 7.x series. Vulkan performance for LLMs is apparently not far behind ROCm and is far more stable and predictable at this time.
213.
by naasking 7mo ago
Great! I hope the era of 1-bit LLMs really gets going.
214.
by naasking 7mo ago
Similar in spirit but different in execution as far as I can tell.
215.
by naasking 7mo ago
No organization can ever rival a real government like the US due to the latter's monopolization on the use of force.
216.
by naasking 7mo ago
You can only truly stop competition by government intervention.
217.
by naasking 7mo ago
Open source vs. Microsoft is a great example.
218.
by naasking 7mo ago
> Thiel's book has influenced so many entrepreneurs into believing Monopolies are Good. Haven't read his book, but the idea that monopolies are good isn't typically made in a vacuum, it's made relative to alternatives
219.
by naasking 7mo ago
> and they will spend infinite amounts of circular fake money to ensure hardware remains prohibitively expensive forever. That's ridiculous, "infinite money" isn't a thing. They will spend as much as they can not beca
220.
by naasking 7mo ago
> it's not clear to me based on the description how this could all be done efficiently. Depends how you define efficiency. The power use of this rig is a lot less than the large data centers that serve trillion parameter models. The
221.
by naasking 7mo ago
> But none of this helps you solve harder problems, or distinguish between a simple solution which is wrong, and a more complex solution which is correct. It does because hallucinations and low confidence share characteristics in the emb
222.
by naasking 7mo ago
Agents need the ability to code but also to objectively and accurately evaluate whether changes resulted in real improvements. This requires skills with metrics and statistics. If they can make those reliable then self-improvement is basica
223.
by naasking 7mo ago
Private companies are embedded in every healthcare system in the world, even public ones.
224.
by naasking 7mo ago
> Learned rotations for INT4 are cool! Seems similar to SpinQuant? https://arxiv.org/abs/2405.16406 Indeed, but much better! More accurate, less time and space overhead, beats AWQ on almost every bench. I hope it be
225.
by naasking 7mo ago
Newer quantization approaches are even better, 4-bits gets you no meaningful loss relative to FP16: https://github.com/z-lab/paroquant Hopefully Microsoft keeps pushing BitNet too, so only "1.58" bits are nee
226.
by naasking 7mo ago
Eventually, yes. ParoQuant is hopefully the future here, 4-bit weights with no real degradation from FP16: https://github.com/z-lab/paroquant
227.
by naasking 7mo ago
> not be so worried if that side has people you don't like on it. I think the point is that they don't like Sony music because they are so often on the wrong side, this time included.
228.
by naasking 7mo ago
> They all make sense to me if we're trying to judge whether these tools are AGI, no? As long as the mean and median human scores are clearly communicated, the scoring is fine. I think the human scores above would surprise people at
229.
by naasking 7mo ago
You could have a system where everyone is directly elected while keeping checks and balances, if voting were restricted , eg. maybe everyone can vote for a president/prime minister, but only non-teachers can vote for an education mi
230.
by naasking 7mo ago
This sounds great! TurboQuant does KV cache compression using quantization via rotations, and ParoQuant [1] does weight compression using quantization via rotations! So we can get 4-bit weights that match bf16 precision, the KV cache goes d
231.
by naasking 7mo ago
> Yet, the majority of new apps and services that I see are all AI ecosystem stuff. The same was true of all this computer science stuff too. We built parsers, compilers, calculators, ftp and http, all cool stuff that just builds up our
232.
by naasking 7mo ago
> The semantics imposed on the bit strings does not exist anywhere in the arithmetic operations, Correct, the semantics is actually in the network of relations between the nodes. That has been one of the major lessons of LLMs, and it val
233.
by naasking 7mo ago
Interesting idea, but I hope people just start switching to ParoQuant and eliminate basically all quantization errors relative to fp16/bf16 even going down to 4-bits: https://github.com/z-lab/paroquant
234.
by naasking 7mo ago
> 2. it makes good business sense. What expected ROI are you basing this on? If it made good business sense on its own, it wouldn't be required by law.
235.
by naasking 7mo ago
Presumably creating a different class for parameter lists allows you to extend it with operations that aren't natural to tuples, like named arguments.
236.
by naasking 7mo ago
> just the Python process collecting and parsing the metrics of all programs consumed 30-40% of the processing power of the lower end boxes. Just write the parsing loop in something faster like C or Rust, instead of the whole thing.
237.
by naasking 7mo ago
> The human genome contains around 1.5GB of information and DeepSeek v3 weighs in at around 800GB, so it's a bit apples-to-oranges. The apples-to-apples comparison is comparing the human genome to the code behind a particular LLM. T
238.
by naasking 7mo ago
Seems like they're relying on the loss as a measure, at least for now.
239.
by naasking 7mo ago
Great project. On the matter of data efficiency and regularization, I'd love to see someone try scaling GrokAlign!
240.
by naasking 7mo ago
> Algorithm implementations are programmatic manifestations of mathematical models and, as such, are not what they model by definition. This is false for constructs of information, ie. a "manifested model" of a sorted list is

More ›