3 ms·

http://imgur.com/ypBQEWx http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activa

by 8note 11y ago

http://imgur.com/ypBQEWx http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activation curve, but it converges more slowly

rmellow 11y ago
Using syn, cos, x1, x2 with 1 six-neuron hidden layer does the trick quickly: http://imgur.com/UMv5gsH http://imgur.com/UMv5gsH No need to mess with noise or regularization :)
andrewtbham 11y ago
Yeh you're on the right track. Nice pattern emerges on this after 160 iterations. http://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=spiral&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=25&networkShape=8,4&seed=0.38071&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=true&xSquared=true&ySquared=true&cosX=false&sinX=true&cosY=false&sinY=true&collectStats=false&problem=classification http://playground.tensorflow.org/#activation=tanh&batchSize=...
- deleted 11y ago
  [deleted]
cglace 11y ago
Using all inputs and 6 layers of varying sizes. After about 500 iterations. http://i.imgur.com/x1MOpvl.jpg http://i.imgur.com/x1MOpvl.jpg
- visarga 11y ago
  Just 100 iterations, learning rate 0.03, activation tanh, regularization L2, rate 0.01. The network is 8,8,8 neurons per layer.
makeset 11y ago
> Add some noise This actually makes the dataset harder to fit to. It is not the same thing here as the "training with noise" method where random noise would be added to each batch, as an alternative means of Tikhonov regularization.
- 8note 11y ago
  wih that particular data set, it looks like it really just adds more data, and more importantly, fills in the gaps along the spirals which is where my setup was having troubles. The noise doesn't go far enough to start confusing points between different clusters, but it adds more points. That said, my knowledge of neural nets is fairly limited.

by 8note 11y ago

http://imgur.com/ypBQEWx http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activation curve, but it converges more slowly

rmellow 11y ago

Using syn, cos, x1, x2 with 1 six-neuron hidden layer does the trick quickly: http://imgur.com/UMv5gsH http://imgur.com/UMv5gsH No need to mess with noise or regularization :)

andrewtbham 11y ago

Yeh you're on the right track. Nice pattern emerges on this after 160 iterations. http://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=spiral&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=25&networkShape=8,4&seed=0.38071&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=true&xSquared=true&ySquared=true&cosX=false&sinX=true&cosY=false&sinY=true&collectStats=false&problem=classification http://playground.tensorflow.org/#activation=tanh&batchSize=...

deleted 11y ago
[deleted]

cglace 11y ago

Using all inputs and 6 layers of varying sizes. After about 500 iterations. http://i.imgur.com/x1MOpvl.jpg http://i.imgur.com/x1MOpvl.jpg

visarga 11y ago
Just 100 iterations, learning rate 0.03, activation tanh, regularization L2, rate 0.01. The network is 8,8,8 neurons per layer.

makeset 11y ago

> Add some noise This actually makes the dataset harder to fit to. It is not the same thing here as the "training with noise" method where random noise would be added to each batch, as an alternative means of Tikhonov regularization.

8note 11y ago
wih that particular data set, it looks like it really just adds more data, and more importantly, fills in the gaps along the spirals which is where my setup was having troubles. The noise doesn't go far enough to start confusing points between different clusters, but it adds more points. That said, my knowledge of neural nets is fairly limited.