They refer to score models, where the score is the density of the dataset at the input. Imagine you have a huuuuge landscape, with peaks and valleys. This land

They refer to score models, where the score is the density of the dataset at the input. Imagine you have a huuuuge landscape, with peaks and valleys. This landscape defines the density of the distribution. Now, imagine that you have a bunch of samples drawn from the distribution, this is the dataset. Sampling an image is a mixture of dirac delta functions (or a mixture of Gaussian with variance approaching zero). We can increase the variance to smoothen the landscape. This sort of builds an empirical estimate of the true landscape. The height of this landscape is “the score”. It turns out that we can actually compute an approximation of the gradient of the score. The gradient of the score always points towards where the score (aka density) will increase, and you essentially apply gradients to “walk” towards a high density region. This idea is actually very similar to using gradients to create poisoned inputs that are falsely predicted. I am being a bit handwavey here, we don’t just increase the variance, but this is a decent enough approximation of what’s happening and what the gradient refers to in this case. For all intents and purposes, you can think of the gradient as a vector pointing towards the direction that the density of the dataset increases.