4 ms·
Here are two papers linked from Inception's site: 1. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution - https://arxiv.org/abs/231
by freeqaz 1y ago
Here are two papers linked from Inception's site:
1. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
- https://arxiv.org/abs/2310.16834 https://arxiv.org/abs/2310.16834
2. Simple and Effective Masked Diffusion Language Models - https://arxiv.org/abs/2406.07524 https://arxiv.org/abs/2406.07524
- AlexCoventry 1y agoThanks, yes, I was thinking specifically of "Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution". They actually consider two noise distributions: one with uniform sampling for each noised token position, and one with a terminal masking (the Q^{uniform} and Q^{absorb}.) However, the terminal-masking system is clearly superior in their benchmarks. https://arxiv.org/pdf/2310.16834#page=6 https://arxiv.org/pdf/2310.16834#page=6