4 ms·
Makes sense. I wonder if it would actually make more sense to just plug a function that generates the mask because in general it's just: fn complete(input, m
by killthebuddha 3y ago
Makes sense. I wonder if it would actually make more sense to just plug a function that generates the mask because in general it's just:
fn complete(input, masker):
completion = input
while (not done yet):
mask = masker(completion)
completion = complete(complete, mask)
end
return completion
end
is that accurate?
- rckrd 3y agoThere's probably a better API that wraps generate, but there's a bit more work than the logit mask. You have to go one token at a time, otherwise the masking becomes combinatoric rather than linear (two tokens at a time -- need to generate all two token pairs, etc.). But otherwise, that's what the code does! https://github.com/r2d4/rellm/blob/main/rellm/rellm.py#L21 https://github.com/r2d4/rellm/blob/main/rellm/rellm.py#L21