3 ms·

> Because self-attention can be replaced with FFT for a loss in accuracy and a reduction in kWh [1], I suspect that the Quantum Fourier Transform can also be su

> Because self-attention can be replaced with FFT for a loss in accuracy and a reduction in kWh [1], I suspect that the Quantum Fourier Transform can also be substituted for attention in LLMs. Couldn't figure out where you are quoting this from. > Can the QFT Quantum Fourier Transform (and IQFT Inverse Quantum Fourier Transform) also be substituted for self-attention in LLMs No. The quantum Fourier transform is just a particular factorization of the QFT as run on a quantum computer. It's not any faster if you run it on a classical computer. And to run (part of) LLMs would be more expensive on a quantum computer (because using arbitrary classical data with a quantum computer is expensive).

westurner 8mo ago
My mistake. That's actually a quote of myself, from an also tangential comment re: "Transformer is a holographic associative memory" (2025) https://news.ycombinator.com/item?id=43029899 https://news.ycombinator.com/item?id=43029899 .. https://westurner.github.io/hnlog/#comment-43029899 https://westurner.github.io/hnlog/#comment-43029899 There's more to that argument though. Is quantum logic more appropriate for universal function approximation than LLMs (self-attention,), which must not do better than next word prediction unless asked (due to copyright)? If quantum probabilistic logic is appropriate for all physical things, then quantum probabilistic logic is probably better at simulating physical things. If LLMs, like [classical Fourier] convolution, are an approximation and they don't do quantum logic, then they cannot be sufficient at simulating physical things. But we won't know until we have enough coherent qubits and we determine how to quantum embed these wave states. (And I have some notes on this; involving stars in rectangular lattices and nitrogenated lignin and solitons.) Or, it's possible to reason about what will be possible given sufficient QC to host an artificial neural network. How to quantum embed a trained LLM into qubit registers (or qubit storage) and use programmable/reconfigurable quantum circuits to lookup embeddings and do only feed-forward better than convolution? But QFT and IQFT solve the discrete inverse logarithm problem. There's probably a place for quantum statistical mechanics in LLMs. Probably also counterfactuals including Constructor Theory counterfactuals.

westurner 8mo ago

My mistake. That's actually a quote of myself, from an also tangential comment re: "Transformer is a holographic associative memory" (2025) https://news.ycombinator.com/item?id=43029899 https://news.ycombinator.com/item?id=43029899 .. https://westurner.github.io/hnlog/#comment-43029899 https://westurner.github.io/hnlog/#comment-43029899 There's more to that argument though. Is quantum logic more appropriate for universal function approximation than LLMs (self-attention,), which must not do better than next word prediction unless asked (due to copyright)? If quantum probabilistic logic is appropriate for all physical things, then quantum probabilistic logic is probably better at simulating physical things. If LLMs, like [classical Fourier] convolution, are an approximation and they don't do quantum logic, then they cannot be sufficient at simulating physical things. But we won't know until we have enough coherent qubits and we determine how to quantum embed these wave states. (And I have some notes on this; involving stars in rectangular lattices and nitrogenated lignin and solitons.) Or, it's possible to reason about what will be possible given sufficient QC to host an artificial neural network. How to quantum embed a trained LLM into qubit registers (or qubit storage) and use programmable/reconfigurable quantum circuits to lookup embeddings and do only feed-forward better than convolution? But QFT and IQFT solve the discrete inverse logarithm problem. There's probably a place for quantum statistical mechanics in LLMs. Probably also counterfactuals including Constructor Theory counterfactuals.