4 ms·
isn't it impressive? They trained a network to predict the next character and when the finished they trained the last layer to predict sentiment with only few e
by jorgemf 10y ago
isn't it impressive? They trained a network to predict the next character and when the finished they trained the last layer to predict sentiment with only few examples. The network was able to learn sentiment somehow in the first training phase without telling it!
For me this open my mind to new opportunities when training deep learning. For example I can do the same for images: train a network to recognize objects and later use the same network to predict sentiment or prettiness for example. And the best thing is that I don't need a lot of labeled examples for the second phase of training!
- taneq 10y agoImmediate thought - the network didn't learn sentiment in the training phase, it just clustered the data. The last phase was so quick to train because it was labeling already-clustered data, so a few data points in each cluster was enough to make it 'obvious'.
- jorgemf 10y agoActually it was only an output neuron, and for me it is the same if you call it cluster or not. It just learnt sentiment without any hint. Now I wonder if they can train it again and get the same sentiment neuron, that would mean something. Like sentiment is necessary to predict the next character? (word?)