3 ms·LLMs will talk to each other using embeddings.by riku_iki 3y agoLLMs will talk to each other using embeddings.Rhapso 3y agoFTR LLM embeddeds are a lot less efficient than ASCII for communication and are likely to stay that way.riku_iki 3y agoabsolutely not. Transformer layers already communicate using embeddings, and ASCII would be absolutely less efficient there.Rhapso 3y agoAnd how many bits are in an embedded vector?riku_iki 3y ago12k for gpt3.riku_iki 3y agoIt is not bits, but weightsRhapso 3y agoSo somehow ascii is less information dense than 12k 32-bit floats per token?