numeri

9 ms·

121.
by numeri 3y ago
This post isn't a scientific investigation. It's someone playing around with a black-box model with few to little controls – which is unfortunately the only thing we can do when experimenting with GPT-4, which does excuse this par
122.
by numeri 3y ago
FlashAttention is an amazing improvement over the previous state of the art. The others are still highly experimental, but seem like they'll at least contribute significant knowledge to whatever ends up surpassing the Transformer, (ass
123.
by numeri 3y ago
How do your views on rural independence fit with the fact that most rural areas in the United States require massive amounts of monetary support from the more population dense (and progressive) states? [1] I would argue that this idea of ru
124.
by numeri 3y ago
One of the most significant quantization papers of the last year [1] found precisely that these outliers only start occuring with LLMs at 6.7B parameters and above. One of the most important keys to the success of deep learning in the last
125.
by numeri 3y ago
In the Qualcomm paper cited, they explain/hypothesize that Transformers learn to attend to these low-meaning tokens when they want to avoid adding too much extra info to the residual stream. So it's not an issue that the models
126.
by numeri 3y ago
In addition to the evaluation issues, it looks like several of their test sets have significant overlap with the test sets [1]. Especially for a compression-based technique, having exact duplicates is going to help a lot. [1] https:/&
127.
by numeri 3y ago
Neither of those have anything to do with what I quoted. Parts I have issue with are the idea that the best structure for analyzing language are messages, which contain intentions, which can be recursive, or that the fundamental space of me
128.
by numeri 3y ago
This reeks of the classic "physicist/mathematician/computer scientist encounters new field and is convinced she/he has solved it", this time applied to linguistics and the deeper meaning and structure of language: &
129.
by numeri 3y ago
Long term, it's the teenagers who are really in play. The events that impact each generation as they transition into adulthood will impact their political views for the rest of their lives.
130.
by numeri 3y ago
Mostly because society (at least in the US) does not yet have the protections in place to stop corporations and government agency from abusing automated surveillance. Who has permission to perform mass facial recognition? What are the limit
131.
by numeri 3y ago
This is reminiscent of the Hegelian dialectic. I'll probably misrepresent it, but as I understand it, it's the idea that society evolves slowly but surely through a constant three step process: - Society swings to one extreme
132.
by numeri 3y ago
I've opted out the last three times while traveling between Europe and the United States, but each time the worker has given me grief about it. One badgered me repeatedly about why I wanted to opt out, and then interrupted my answer to
133.
by numeri 3y ago
I'd say for research purposes. HackerNews seems to tend to mostly represent the LLM consumer viewpoint, but these waves of models being released are honestly more interesting from a research than a user point of view. As a LLM user, yo
134.
by numeri 3y ago
The closest English equivalent I can think of is the first sound in the word "huge" if it's pronounced with a lot of exaggeration. Even that's not a perfect match.
135.
by numeri 3y ago
This technique can be added on to any encoder–decoder Transformer model post-training, so the added training difficulties you mention don't apply. It honestly is a very interesting approach to me – the main issue I see (which they disc
136.
by numeri 3y ago
This paper is honestly fairly weak, if you look at it closely. The theoretical justifications rely heavily on approximating attention with linear attention, and really just show that linear attention could be written as (A + A_delta)x, then
137.
by numeri 3y ago
I agree with you (although I think one should be upset about this, and you seemed to imply that you disagree with the "systemic racism people"). This would directly penalize endangered languages and dialects, which is a tragic los
138.
by numeri 4y ago
I've seen this hyped a lot on Reddit, and its main contributing base seems to be on a Discord channel, but have yet to see any scholarly discussion behind it. Is there a reason the creator hasn't written or tried to publish a pape
139.
by numeri 4y ago
In what ways would you say its awkward? I'm really curious about its performance in languages that might not have shown up in its training data much. I will say, I've tried using it with a dialect, and it's good when it choos
140.
by numeri 4y ago
If I understand correctly, ChatGPT Plus only gives you access to GPT-4 in the online chat interface, not through the API.
141.
by numeri 4y ago
I personally think this is important completely aside from any cultural propriety/censoring, as it's the one of the only parts of AI ethics that we can currently directly work on and test. If we can't get models not to say ra
142.
by numeri 4y ago
> The concept of factuality doesn't exist in the model at all. This is an example of a whole range of beliefs about LLMs that are very common (even in the field itself), because they were obviously true for small models, but that mi
143.
by numeri 4y ago
So, tax fraud?
144.
by numeri 4y ago
I'd read it, if so!
145.
by numeri 4y ago
You seem to have described my home schooling experience to a T. It really was a wonderful way to learn and interact with expert adults in my community. I had a lot of good role models and mentors that I wouldn't have had otherwise.
146.
by numeri 4y ago
If you think the distribution of disinformation is uniform across both sides, you've also been brainwashed good. There is plenty of peer-reviewed research on this, but here are two that I found after brief searching: https:/&#x2F
147.
by numeri 4y ago
But then you open yourself up entirely to being manipulated by the website owners. The Russian government has enough resources to artificially push their SEO way higher than it should be, and it's objectively lower quality informatio
148.
by numeri 4y ago
This is the real reason they received so few donations. I've only ever donated to them through Brave's Attention Tokens [1], so I never realized they use BitPay, but it really is intrusive. 1. I wonder if they'll keep this en
149.
by numeri 5y ago
This is fairly useless nitpicking in this context – he's talking about a non-native English speaker, who has learned English to the point that her accent is nearly indistinguishable from some native English accent. This is a very com
150.
by numeri 5y ago
The previously existing solution?

More ›