2 ms·

A basic transformer architecture performs only a bounded amount of computation per generated token, so it can never emulate a machine computing sufficiently har

A basic transformer architecture performs only a bounded amount of computation per generated token, so it can never emulate a machine computing sufficiently hard problems.

EVa5I7bHFq9mnYK 2y ago
Yes, because it's feed forward. It must have loops to be a Turing machine.
- phkahler 2y ago
  It does. The output is fed back in.
  - jhanschoo 2y ago
    It indeed does, but it must generate a token per loop, and can thereby solve some linearly complex problems, but it cannot solve harder problems.