3 ms·

I don't think you get what I was saying. Specifically ChatGPT, the current interface, is hilariously bad at lipograms. Nothing about other models, obviously you

I don't think you get what I was saying. Specifically ChatGPT, the current interface, is hilariously bad at lipograms. Nothing about other models, obviously you can build a model to model just about anything you like. I appreciate the link though, it will be fun to play with the huggingface example. Edit: also that example isn't exactly sophisticated. It literally just retries until it finds an example that doesn't contain the banned letters, bit of a brute force method. Would be more interesting to build a new model from scratch that just completely omits the letters.

Der_Einzige 4y ago
Retries what exactly? It filters before the decoding step. I think you misread the code and the paper. I wouldn't know though, I only wrote it and all that...
- jaggederest 4y ago
  Lol living up to the name "hellisotherpeople" I guess. https://huggingface.co/spaces/Hellisotherpeople/Gadsby/blob/main/Text-Generation.py https://huggingface.co/spaces/Hellisotherpeople/Gadsby/blob/... next_token_candidates = torch.multinomial(probs, num_samples=int(number_of_tokens_to_sample)) ## 10000 random samples word_list = [] for candidate_string in next_token_candidates: for candidate in candidate_string: resulting_string = tokenizer.decode(candidate) #skip_special_tokens=True, clean_up_tokenization_spaces=True) ###Constrained text generation starts HERE ##Lipogram - No naughty strings used if mode == "lipogram" or mode == "e-prime": if all(nauty_string not in resulting_string for nauty_string in naughty_strings): ## This returns at the first naughty strings return resulting_string That, to me, reads as "grab tokens as long as they don't contain the bad string(s)", and the code that uses it just repeats that. It doesn't seem, to me, to eliminate tokens at the model level, but I could be wrong, I'm no expert. I'd expect the model itself not to generate candidates that contained the "bad" string. Merry Christmas.

Der_Einzige 4y ago

Retries what exactly? It filters before the decoding step. I think you misread the code and the paper. I wouldn't know though, I only wrote it and all that...

jaggederest 4y ago
Lol living up to the name "hellisotherpeople" I guess. https://huggingface.co/spaces/Hellisotherpeople/Gadsby/blob/main/Text-Generation.py https://huggingface.co/spaces/Hellisotherpeople/Gadsby/blob/... next_token_candidates = torch.multinomial(probs, num_samples=int(number_of_tokens_to_sample)) ## 10000 random samples word_list = [] for candidate_string in next_token_candidates: for candidate in candidate_string: resulting_string = tokenizer.decode(candidate) #skip_special_tokens=True, clean_up_tokenization_spaces=True) ###Constrained text generation starts HERE ##Lipogram - No naughty strings used if mode == "lipogram" or mode == "e-prime": if all(nauty_string not in resulting_string for nauty_string in naughty_strings): ## This returns at the first naughty strings return resulting_string That, to me, reads as "grab tokens as long as they don't contain the bad string(s)", and the code that uses it just repeats that. It doesn't seem, to me, to eliminate tokens at the model level, but I could be wrong, I'm no expert. I'd expect the model itself not to generate candidates that contained the "bad" string. Merry Christmas.