14 ms·

it doesn't seem to support variable length for input and output, does it? The paper seems to use EOS padding to create fixed length input/output. so is there

• sidkshatriya 2y ago
  Yes this is confusing. I too am not sure but here is a guess: In a normal transformer the incremental message is a single token while in LLADA the incremental message would be an additional fixed size block of tokens. If you do inference in the semi-autoregressive strategy outlined in the paper you could start by adding block after block. When a block is fully unmasked it essentially becomes part of the prompt. You would stop the "turn" as soon as there is some sort "end of reply" token decoded in your block. This token could appear anywhere in the block. Questions: (1) Is there such thing as a "end of reply" token ? (I'm not taking about End of Sentence <EOS> which is obviously different) (2) What about the junk tokens that appear after this "end of reply" tokens. I could image the unmasking transformer generating some stuff for those tokens.
  • refulgentis 2y ago
    > (1) Is there such thing as a "end of reply" token ? (I'm not taking about End of Sentence <EOS> which is obviously different) EOS as end of sentence? IIRC its end of sequence, which may also answer the question IIUC.
    • sidkshatriya 2y ago
      <EOS> is end of sequence ! that makes sense, thanks for the correction. So, if we simply wait for <EOS> to be sampled in a block, that could be the place to stop. My answer above should be the same (modulo this correction) -- simply keep obtaining the blocks in a semi-autoregressive fashion.