Hugsun

11 ms·

91.
by Hugsun 2y ago
I'm not in any disagreement with your overall sentiment, but don't all installations on macos require and admin password? It's been a while since I used it.
92.
by Hugsun 2y ago
Indeed. I'm reminded of the time a childhood friend essentially discovered the inverted spectrum argument[1]. That is, we can't know if my qualia when perceiving the color red doesn't match yours of the color blue. We were un
93.
by Hugsun 2y ago
> The experiments that showed this were confusing correlation with causation. What nonsense. https://www.nature.com/articles/s41893-019-0323-1 From the paper: "All studies measured the effects of exposure to ar
94.
by Hugsun 2y ago
Combatting the byproducts of the entire industrial world is a huge undertaking. We should not expect statistically significant contributions to the fight, until the most promising technologies have been scaled up massively. This does not me
95.
by Hugsun 2y ago
It learns to recognize which objects correspond to which sounds, and vice versa. It doesn't really understand linguistic concepts like grammar, sentences, logic, etc. It doesn't really have much in common with LLMs. If the ideas w
96.
by Hugsun 2y ago
I've often wondered if there aren't some structures in the brain, that have been selected for since the advent of language, that are good at learning up languages.
97.
by Hugsun 2y ago
A big problem with the conclusions of this article is the assumptions around possible extrapolations. We don't know if a meaningfully superintelligent entity can exist. We don't understand the ingredients of intelligence that well
98.
by Hugsun 2y ago
My guess is that it's much lower. I'm having a hard time finding a LLM output logit visualizer online, but IIRC, around half of tokens are predicted with >90% confidence. There are regularly more difficult tokens that need to b
99.
by Hugsun 2y ago
> I feel this thought of AGI even possible stems from the deep , very deep , pervasive imagination of the human brain as a computer. But it's not. In other words, no matter how complex a program you write, it's still a Turing m
100.
by Hugsun 2y ago
You shouldn't have included those quotes if you didn't want people responding to them.
101.
by Hugsun 2y ago
At first I was frustrated that the abstract talks about machines without properly defining them. There are obviously gigantic machines, bigger than humans. Then I read a bit more. It's a pretty funny read. I found the "Unboundedne
102.
by Hugsun 2y ago
> Their cost per Max chip is probably $200-300 which beats the 4090 by a massive margin... That's true. I was talking about end user pricing. > ...each chip can do more than a 4090 because it also has a CPU onboard. That's a
103.
by Hugsun 2y ago
> the narrative itself is a circular proposition I only use the simile to make the text more approachable to leymen. I don't ever conclude that because embeddings are like thoughts, thoughts are like embeddings---or LLMs are intelli
104.
by Hugsun 2y ago
Thank you, and well said. > 2. forced to continually output tokens without... I reference one paper that explores this. It lets a model output identical ellipsis tokens that are then discarded before the final output. It worked pretty we
105.
by Hugsun 2y ago
Most people can't do everything you can do (depending on what you mean by everything). They are still intelligent. Many animals are considered intelligent, like crows and dogs. I show, in the article, at least one case where an LLM per
106.
by Hugsun 2y ago
No. It could think on it's own, like MuZero does. It becomes intelligent by learning from prior human work, like every intelligent being.
107.
by Hugsun 2y ago
How do you explain the positive cases in the article? You are right that they are bad at reflecting on their own knowledge. I do explain in the article why it's not reasonable to expect an LLM to have a strong intuition about the speci
108.
by Hugsun 2y ago
The benchmarks are very interesting. Unfortunately, the writing benchmarks seem to be poorly constructed. It looks like there are tasks no model can achieve and others that almost all models pass, i.e. every model gets around 9.0.
109.
by Hugsun 2y ago
How is any of this good? Apple serves its captive users inferior models without giving them a choice. I don't see how that is winning the AI market either.
110.
by Hugsun 2y ago
I would like to see their method compared quantitatively to the best llama.cpp methods. IQ3_S has a similar bpw and pretty high quality. I wonder if they didn't stretch the truth using the phrase "without loss in accuracy".
111.
by Hugsun 2y ago
Perhaps. It will be exciting to see if/how that happens. It does seem relatively far off still. At least some years.
112.
by Hugsun 2y ago
The reasons are very not obvious to me. Could you elaborate?
113.
by Hugsun 2y ago
There is no way they would secretly train loras in the background of their user's phones. The benefits are small compared to the many potential problems. They describe some LoRA training infrastructure which is likely using the same c
114.
by Hugsun 2y ago
I didn't see the presentation but judging by your description, this is achievable using in-context learning.
115.
by Hugsun 2y ago
You get considerably more ML FLOPS per dollar in a 4090 than any mac. It seems like the base M2 MAX is at roughly the same price point. It does grant you more RAM. Quadro and Tesla cards might be a different story. I would still like to see
116.
by Hugsun 2y ago
Did you happen to use this tool to make this post?
117.
by Hugsun 2y ago
That makes sense. I imagine that there are significant performance tradeoffs but those are probably worth it in many cases. I would be somewhat surprised if Mamba can be made usefully fast with NumPy, but it would be a pleasant surprise.
118.
by Hugsun 2y ago
Is there a benefit to implementing it in numpy over pytorch or tf?
119.
by Hugsun 2y ago
Thanks for the article. Very informative.
120.
by Hugsun 2y ago
That is exactly what I did in the section I'm referring to. I asked both suggestive and non-suggestive questions. It needed the suggestion to get the right answer. Just asking it to elaborate didn't change it's answer.

More ›