4 ms·
Are the reported success metrics on the training or testing set? The website says its on the training set, which shouldn't be a valid metric of success since ne
by bradneuberg 12y ago
Are the reported success metrics on the training or testing set? The website says its on the training set, which shouldn't be a valid metric of success since neural networks can easily overfit to their training data (one of their downsides if you aren't careful).
Having the output layer be an 8-bit character representation though is very clever, rather than a softmax layer with each node being the relative probability of a given character. That probably lowers the number of free parameters you have to train, which probably speeds up training and can help prevent overfitting. I'm interested in knowing what the true success rate is with this approach as it seems clever.
Btw, what's your loss function on the output layer?
- megalodon 12y agoYes, the success metrics are measured on the training set. I just ran a measurement with a separately generated testing set consisting of 52k characters which yielded a success rate of 98.53% on letters a-z. This turns out to be almost exactly the same as the rate presented on the repo page (98.52%). Will upload the result for your perusal as soon as possible. You will have to browse the synaptic neural network library [1] for an answer to your second question. [1]: https://github.com/cazala/synaptic https://github.com/cazala/synaptic EDIT: A separate testing set is now generated in addition to the training set. I updated the success rates of the examples in the readme accordingly.