4 ms·

I use LLM embeddings all the time at my day job (and I work in a very, very low-tech "tech" job, basically an IT department). It's not even remotely difficult a

I use LLM embeddings all the time at my day job (and I work in a very, very low-tech "tech" job, basically an IT department). It's not even remotely difficult and works incredibly well for sifting through absurd amounts of data to filter it down to something that is easy to human-review. https://platform.openai.com/docs/guides/embeddings https://platform.openai.com/docs/guides/embeddings https://www.youtube.com/watch?v=vVGTegRvXg8 https://www.youtube.com/watch?v=vVGTegRvXg8 I think you may be commenting "confidently" on technologies that you're not very familiar with. It is something that we all struggle with sometimes.

dragontamer 2y ago
> I think you may be commenting "confidently" on technologies that you're not very familiar with. It is something that we all struggle with sometimes. Your snarkiness aside, what I've said earlier I stand by. LLMs are a network of neurons / self-learning units that point towards the "next" word to be predicted". This is fundamental to the architecture of this entire system. I recognize that there's some kind of parsing and understanding from tokens -> hidden layers of the network. That's just how the math works. Perhaps you should educate yourself upon neural networks and the fundamental math at play here before caustically criticizing everyone who contributes to a discussion. I admit that I don't know what an embedding is but I know it's something that is created by a neural network's output layer (or the post-processing of the output layer). This is once again, fundamental to the math of the neural nets at play here. So anything I know about output layers applies to whatever an embedding is. --------- Not everyone in this field is working off of just using the damn thing. Some of us have experience in the layout and underlying theory of neural nets. And I'm confident in what I've said.
- reaperman 2y ago
  I’m sorry that I came across so different from how I meant to. I was intending to evoke a tone more similar to one I had recently here[0]. That is my fault and I should work on my communication. > I admit that I don't know what an embedding is but I know it's something that is created by a neural network's output layer (or the post-processing of the output layer). The embedding layer comprises the first layers of a transformer model, definitely not the “output layer”. It is typically trivially easy to isolate the output of the embedding layer because it generates the “actual” input to the LLM. The LLM has a certain number of input parameters (typically 784) and some length of text usually won’t fit precisely into 784 parameters (it will be longer or shorter than that). All the embedding layer does is map arbitrary length text to a single 784-dimension vector, which actually can be passed to the input of the transformer. Ever since the paper “Attention is all you need”, the embedding layers have always been the first layers of a transformer to act on the text input. Most of the output is also sent back to the embedder, but this is only so that it can be used as input again to the “attention” layers. 0: https://news.ycombinator.com/item?id=40878317 https://news.ycombinator.com/item?id=40878317

dragontamer 2y ago

> I think you may be commenting "confidently" on technologies that you're not very familiar with. It is something that we all struggle with sometimes. Your snarkiness aside, what I've said earlier I stand by. LLMs are a network of neurons / self-learning units that point towards the "next" word to be predicted". This is fundamental to the architecture of this entire system. I recognize that there's some kind of parsing and understanding from tokens -> hidden layers of the network. That's just how the math works. Perhaps you should educate yourself upon neural networks and the fundamental math at play here before caustically criticizing everyone who contributes to a discussion. I admit that I don't know what an embedding is but I know it's something that is created by a neural network's output layer (or the post-processing of the output layer). This is once again, fundamental to the math of the neural nets at play here. So anything I know about output layers applies to whatever an embedding is. --------- Not everyone in this field is working off of just using the damn thing. Some of us have experience in the layout and underlying theory of neural nets. And I'm confident in what I've said.

reaperman 2y ago
I’m sorry that I came across so different from how I meant to. I was intending to evoke a tone more similar to one I had recently here[0]. That is my fault and I should work on my communication. > I admit that I don't know what an embedding is but I know it's something that is created by a neural network's output layer (or the post-processing of the output layer). The embedding layer comprises the first layers of a transformer model, definitely not the “output layer”. It is typically trivially easy to isolate the output of the embedding layer because it generates the “actual” input to the LLM. The LLM has a certain number of input parameters (typically 784) and some length of text usually won’t fit precisely into 784 parameters (it will be longer or shorter than that). All the embedding layer does is map arbitrary length text to a single 784-dimension vector, which actually can be passed to the input of the transformer. Ever since the paper “Attention is all you need”, the embedding layers have always been the first layers of a transformer to act on the text input. Most of the output is also sent back to the embedder, but this is only so that it can be used as input again to the “attention” layers. 0: https://news.ycombinator.com/item?id=40878317 https://news.ycombinator.com/item?id=40878317