7 ms·
Reflected Diffusion Models
- ttul 3y agoReflected Diffusion Models are a new class of diffusion models that respect general boundary constraints and correctly train for thresholded sampling. They bridge the gap between theoretical frameworks and practical implementation details, leading to improved performance and image quality in generative modeling tasks. Diffusion models are generative models that work by perturbing data using a stochastic process, then reversing this process to generate new data. However, when discretized into discrete steps (as in most practical implementations), the sampling process can negatively affect data quality. To alleviate this issue, a technique called thresholding is employed, resulting in better samples but breaking the theoretical framework. Reflected Diffusion Models use a reflected stochastic differential equation (rSDE) to correctly model the thresholding process. This allows the models to train appropriately while respecting boundary constraints, like images with pixel values in the [0, 255] range. The benefits of Reflected Diffusion Models include improved perceptual quality, correct handling of boundary constraints, better image generation with guidance, and general applicability to different shapes of data domains. This has the potential to expand the applications of diffusion models in fields such as image, language, and molecule generation.
- ttul 3y agoAs is so often the case in neural network design, preventing numbers from blowing up is the cause of so many necessary, but poorly understood math hacks. It seems these researchers have found a potentially better way to control the sampling process so that diffusion models can generate outputs that are more correct. I often wonder if Midjourney, behind the scenes, has already discovered something like this in their v5 release, which is substantially better at modelling things like hands.
- toxik 3y agoI hate to ask but is this ChatGPT?
- dr_dshiv 3y agoOnly if you don’t find it useful
- visarga 3y agochatGPT "of the gaps", it is chatGPT only when it is not useful
- refulgentis 3y agoIt is ChatGPT and it's not useful. It purposefully hides while it attempts to illuminate. It masquerades as summarizing the paper, but it's a random grab of whatever OP decided to jam into the chatbox that fits in the context window, (if you check, its mostly regurgitating the abstract). However, it presents itself as a summary as if the entire paper was read. I'm _really_ into AI stuff but the flippant nature its used at on HN has me scared. People here are generally well-educated, considered in their decisions, and are well acquainted with tech. If HN can't avoid glaring issues with it, who can? The top example that came to mind was someone laboriously sharing a really nice data-driven comment, someone asks source? they say ChatGPT, and acted like we were trolling for saying it hallucinates.
- smrtinsert 3y ago> People here are generally well-educated This isn't a given (anymore?).
- ttul 3y agoThe entire blog post fit into the context window. I did not use GPT-4 to summarize the whole paper. Either way, I found the summary informative after reading the blog post end-to-end in the more human manner anyway. I don't think GPT-4 missed anything crucial from the blog.
- refulgentis 3y agoI stand corrected: only 4100 tokens! It missed a _ton_ for me, it seems to pick out unique phrases and elegantly create a "skeleton" of the original. > I did not use GPT-4 to summarize the whole paper. > I don't think GPT-4 missed anything crucial from the blog. ?
- devxpy 3y agoIs there a way to run this locally?
- toxik 3y agoThis text was your classic “I’m writing a tutorial to see how much I know”, being only intelligible to the exact set of people who already know what it’s about. A lot of important ideas glossed over, eg “we don’t know log pt” then immediately using it in the loss function. I probably sound very negative but I feel that you have some responsibility to your readers if you publish your text.
- lbotos 3y agoAre you a PHD level AI researcher? If not, I don't think you are the intended audience. I'm not, so as a reader here, I'm just happy that I didn't have to pay to get exposed to new challenging ideas. A lot of other science is sadly paywalled inside of journals. Give it a few weeks for the fast followers to dumb it down for us.
- toxik 3y agoI am, yes. Most of our field’s research is publicly available on arxiv by the way.
- yorwba 3y agoIndeed, the paper associated with the blog post is also on arXiv. But how does that relate to your complaint?
- mrtranscendence 3y agotoxik was responding to the apparent position that it's OK for the material to be presented in a confusing way, because the research has been provided freely rather than locked behind a paywall. It's apropos to point out that this isn't a great excuse.
- tmabraham 3y agoOkay, but are you a diffusion model/generative AI researcher? As a researcher in this field, I found the article fairly understandable and accessible. I suspect that was the target audience. The domain of diffusion models have a lot of terminology and concepts that are not explained here, so maybe that's why it's harder to understand?
- aantix 3y agoWe don't tolerate poorly named variable and method names in software development - why is it tolerated in mathematical notation?
- Kuinox 3y agoHistorically paper was costly, and there is no autocomplete on paper too.
- simion314 3y agoYou should try it with long names and without and let's see what is easy to understand. e^pii = -1 or theNatural....Constant... = theCircleConstant... theSqureRootOfMinusOne = minusOn or maybe you replace One with theNeutralElementForMultiplication but maybe someone is too lazy and you need to give a good clear name for what multiplication is, Sorry, but I could not help it, I seen this kind of comments multiple time, some people want to jump to the middle or end of a book/course and skip the hard work.
- qumpis 3y agoThis is other side of extreme. And these constants are widely accepted notation.
- simion314 3y ago>This is other side of extreme. And these constants are widely accepted notation. Some dude will say that "i" is also used as the vector for the horizontal axis, i is sometimes used in induction as iterator too, so clearly it should be refactored to sqrtOfMinusOne. Also + and * are overloaded in math, so maybe they should be replaced with a clear name like "realNumberAdition"
- Freebytes 3y agoWhat you demonstrated as the "bad example" is closer to what mathematicians have been doing recently, but they would use natLog, ratCir, and im as variable names instead inside of a function called gnl() and offer no comments as to what any of it means. From what I have seen, they should likely be putting their code into ChatGPT and asking it to clean it up because it does a better job of writing functions with properly named variables.
- Der_Einzige 3y agoOne missing feature that could be implemented in places like the Automatic1111 webUI, would be a "color threshold" or even a "color loss" function. I should be able to specify that for a particular region of the image, that a particular set of colors should not be generated, or I should be able to bias or prefer certain colors on certain parts of the image. I actually wrote a jupyter notebook based off of another notebook that showed that color loss was possible. My jupyter notebook extended the idea by allowing to specify the region and the particular color that the color loss would operate on. I'm surprised that despite how much tooling currently exists, tooling for specifying colors is not really common or well known. I figured that controlnet would have something for this by now, but here we are...
- enlyth 3y agoThis already exists, check out the Spatial Palette (Color) Adapter here: https://github.com/TencentARC/T2I-Adapter https://github.com/TencentARC/T2I-Adapter It works with the Mikubill ControlNet plugin for A1111.
- deleted 3y ago[deleted]
- nmca 3y agoThis was a great investigation of a trick one might not naively expect to be important & a theoretically justified fix. Nice work!
- subroutine 3y agoThis is a side note / random observation, but it seems like "An astronaut riding a horse" has become the Lorem ipsum / Lenna / Utah teapot of AI text-to-image generation. https://en.wikipedia.org/wiki/Lenna https://en.wikipedia.org/wiki/Lenna https://en.wikipedia.org/wiki/Utah_teapot https://en.wikipedia.org/wiki/Utah_teapot
- TrainedMonkey 3y agoSomething about bridging transportation technologies.
- kristjansson 3y agoIt’s funny, the original paper (DALL-E maybe?) used it as an example of something unlikely to exist in the training data, and therefore a good test of the model's compositional ability. Now that there’s so many successful generative models producing examples of horse-astronauts, producing them isn’t as obviously convincing.
- albert_e 3y agoIndeed! https://www.google.com/search?q=%22An+astronaut+riding+a+horse%22 https://www.google.com/search?q=%22An+astronaut+riding+a+hor... The internet is already flooded with images generated by these common prompts. It would be fun to have a space agency do a photoshoot of a real astronaut on a real horse ... just to complete the circle :) There was a phase decades ago about "googlenope" -- defined as a phrase that google has never seen, so it would return zero results if searched with quotes. Something similar for GenerativeAI may be worth coining?
- pavelstoev 3y agoI have been using The quick brown fox jumps over the lazy dog
- subroutine 3y agoFunny you say that, I was just trying to get this to work! Do you get good results? For some reason I can't get sd2.1 to include the "lazy dog".
- bionhoward 3y agoYo dawg I heard you like Parmenides dialogue and Euler number formula and everything is secretly connected?!?