4 ms·

Can someone explain this in a way so that an ordinary mortal computer scientist can understand it?

vbarrielle 9y ago
> Can someone explain this in a way so that an ordinary mortal computer scientist can understand it? I'll try. Instead of the common approach that tries to search for image pixels to minimize e.g. a denoising objective, they realize that they can instead search for the weights of an image generator network such that the generated image matches the objective. They argue that the structure of the network then constitutes some prior knowledge over what a natural image should look like. My (probably wrong) interpretation: since a convolutional neural network essentially works by looking for some spatial patterns at different resolutions, their optimization process boils down to finding the high and mid-resolution patterns that best match the input image, and then re-using that information to fill in the missing information (or replacing "noise" that does not match the extracted pattern).
- rdlecler1 9y ago
  What do they mean by ‘the structure of the network’ is this the network topology?
fny 9y ago
There exist neural network structures that can solve the image processing problems shown when weights are randomly applied. This means that for these problems the solution is encoded entirely in the network structure and not the weights. We could use linear regression as an analogy: y = b f(x). This is akin to saying that for some class of problems, it's the case that you can get as good of a solution from the function you pick alone (x^2, log(x)) as you could from picking a function and then computing the best coefficient. Note this isn't true for linear regression for any meaningful problems, but I feel like it's a decent analogy.
denzil_correa 9y ago
There's a great explanation of the idea here in the comments : https://news.ycombinator.com/item?id=15820994 https://news.ycombinator.com/item?id=15820994
notaboutdave 9y ago
Here's a psuedocode rundown of exactly what it does: network = new NeuralNetwork() targetOutput = readFile('./images/zebra-500x500-corrupted.jpg') input = generateNoise(500, 500) while(iterationCount < overfitThreshold) { networkOutput = network.getOutput(input) loss = getLoss(targetOutput, networkOutput) network.backPropogate(loss) iterationCount++ } writeFile('./images/zebra-500x500-denoised.jpg', networkOutput)

> Can someone explain this in a way so that an ordinary mortal computer scientist can understand it? I'll try. Instead of the common approach that tries to search for image pixels to minimize e.g. a denoising objective, they realize that they can instead search for the weights of an image generator network such that the generated image matches the objective. They argue that the structure of the network then constitutes some prior knowledge over what a natural image should look like. My (probably wrong) interpretation: since a convolutional neural network essentially works by looking for some spatial patterns at different resolutions, their optimization process boils down to finding the high and mid-resolution patterns that best match the input image, and then re-using that information to fill in the missing information (or replacing "noise" that does not match the extracted pattern).

rdlecler1 9y ago
What do they mean by ‘the structure of the network’ is this the network topology?

fny 9y ago

There exist neural network structures that can solve the image processing problems shown when weights are randomly applied. This means that for these problems the solution is encoded entirely in the network structure and not the weights. We could use linear regression as an analogy: y = b f(x). This is akin to saying that for some class of problems, it's the case that you can get as good of a solution from the function you pick alone (x^2, log(x)) as you could from picking a function and then computing the best coefficient. Note this isn't true for linear regression for any meaningful problems, but I feel like it's a decent analogy.

denzil_correa 9y ago

There's a great explanation of the idea here in the comments : https://news.ycombinator.com/item?id=15820994 https://news.ycombinator.com/item?id=15820994

notaboutdave 9y ago

Here's a psuedocode rundown of exactly what it does: network = new NeuralNetwork() targetOutput = readFile('./images/zebra-500x500-corrupted.jpg') input = generateNoise(500, 500) while(iterationCount < overfitThreshold) { networkOutput = network.getOutput(input) loss = getLoss(targetOutput, networkOutput) network.backPropogate(loss) iterationCount++ } writeFile('./images/zebra-500x500-denoised.jpg', networkOutput)