visarga

37 ms·

661.
by visarga 1y ago
I am always skeptical of RNN approaches but this paper is just sparsifying the input, it is not compressing any size input to a fixed memory. I am hopeful maybe this is a big break. 11x inference speedup with no degradation from an algorith
662.
by visarga 1y ago
> trademark law Presumably an AI should know about the trademarks, they are part of the world too. There is no point shielding LLMs from trademarks in the wild. A text editor can also infringe trademarks, depending how you use it. AI is
663.
by visarga 1y ago
The point about training data stands. We usually only think of scaling compute, but we need to scale data as well, maybe even faster than compute. But we exhausted the source of high quality organic text, and it doesn't grow exponentia
664.
by visarga 1y ago
1:1 lending is not unlimited copying, I don't advocate sharing copyrighted data like that.
665.
by visarga 1y ago
Once a book is sold publishers would do anything to limit its use. No scanning, no online lending, no training LLMs on it, no reusing of protected style or abstractions for fan fiction. If the author is not to be found, the book itself beco
666.
by visarga 1y ago
> could run cosign diff on your vector db or maybe even "cosine similarity"
667.
by visarga 1y ago
Oh what you don't understand is that LLMs also use embeddings inside, it's how they represent tokens. It's just that you don't get to see the embeddings, they are inner workings.
668.
by visarga 1y ago
I think it is more informative to simply visualize a word cloud or even to show top-k results for a query. Something like https://projector.tensorflow.org/ just type a word in, select UMAP projection.
669.
by visarga 1y ago
> LLMs reduce a very high-dimensional vector space into a very low-dimensional vector space. What do you mean? There is an embedding size that is maintained constant from the first layer to the last. Embedding lookup, N x transformer lay
670.
by visarga 1y ago
Your approach is much more intuitive. I was coming back to say why didn't they show an embedding with categorical/scalar features?
671.
by visarga 1y ago
You can search the closest matching words or expressions in a dictionary. It is trivial to understand where an embedding points to.
672.
by visarga 1y ago
Simple, don't read anything in original, pass everything through your unslopifiying AI. That is the end game.
673.
by visarga 1y ago
Well said, the problem with recursion is that it constructs its own context as it goes, rewrites its rules, and you cannot predict it statically, without forward execution. It's why we have the halting problem. Recursion is irreducible
674.
by visarga 1y ago
> To me, blogs and other articles feel like sales pitches, whereas comments are full of raw emotion and seem more honest. I end up seeking out discussions over buttoned up long-form articles. Me too. That is why sometimes I take the raw
675.
by visarga 1y ago
Sometimes they hack the back navigation function and present their own clone of Google Discover feed. If you are not careful you might end up in a different feed.
676.
by visarga 1y ago
Chomsky's innate grammar misses the larger process - is it not more likely that languages that can't be learned by babies don't survive? Learnability might be the outcome of language evolution. The brain did not have time to
677.
by visarga 1y ago
This is why SOTA LLMs can't manage to maintain a lead of more than a few months. There are half a million datasets on HuggingFace. Models are social, they learn from each other, learn from humans, and work together with humans and othe
678.
by visarga 1y ago
> We're really good at cumulative iteration. Humans are turbo optimized for communities, basically. This is why brainstorming is so effective… But usually only in a group. There is an entire theory in cognitive psychology about cumu
679.
by visarga 1y ago
> The ease of training AI to solve a task is proportional to how verifiable the task is. All tasks that are possible to solve and easy to verify will be solved by AI. I've been saying this for years. Especially related to prediction
680.
by visarga 1y ago
Yes, it is covered technically. But practically nobody knows what is infringing in the non-literal infringement case. It all depends on the judge and context. Was this idea sufficiently original or was it a necessity, or a generic pattern?
681.
by visarga 1y ago
> why should I bother to read it and provide feedback? I like to discuss a topic with a LLM and generate an article at the end. It is more structured and better worded but still reflects my own ideas. I only post these articles in a priv
682.
by visarga 1y ago
You can train a LLM on completely clean data, creative commons and legally licensed text, and at inference time someone will just put a whole article or chapter in the model and has full access to regenerate it however they like.
683.
by visarga 1y ago
> The counterintuitive part of automation is that it removes parts of the economy rather than making the economy bigger So you believe in zero sum economy? I think new capabilities lead to demand expansion, they mobilize latent demand th
684.
by visarga 1y ago
The paper was a very nice read, and they did many creative things. It's a pity this model won't be directly accessible, only integrated in some apps.
685.
by visarga 1y ago
I am more and more convinced that social networks are causing the increased political instability and breakdown of communication. In the past media was centralized. So the powers that be focused on it and controlled the message. We had a ki
686.
by visarga 1y ago
> I’m certainly not convinced that they’re the future I want. But what I’m most certain of is that we have choices about what our future should look like, and how we choose to use machines to build it. While I must admit we have some cho
687.
by visarga 1y ago
This works great for software, math and games where you can have cheap validation. But what about messy real world tasks? I think hindsight learning from chat logs could fit the bill. What do I mean? Imagine a long conversation. It is hard
688.
by visarga 1y ago
> At the risk of overgeneralizing, more and more in modern life it feels as though we are all surrounded by people who are supposed to do their jobs right who don't Meta observation - human society works by abstraction - leaky, and
689.
by visarga 1y ago
1.6x multiplier is low, we usually need to apply 5x
690.
by visarga 1y ago
Yes, until serious adoption I am reserved too, both on SSMs and diffusion based LLMs.

More ›