3 ms·

Hi, I have already expected a question about Neural Networks :) When I had the idea I also toyed around with word-rnn and similar RNN libraries. The results I

Hi, I have already expected a question about Neural Networks :) When I had the idea I also toyed around with word-rnn and similar RNN libraries. The results I got were pretty good, but training was extremely resource consuming. Cuda gave a 8x boost, but still training one of the smaller sites took 20 minutes on my simple graphics card while creating the Markov chain is done in 2 minutes. I also have absolutely no experience with Machine Learning and just the setup was already quite an experience. So I stuck with what I know and stayed "the traditional way".

stared 8y ago
Interesting. Yes, training takes way more time. In some really big projects like https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/ "We first trained a multiplicative LSTM with 4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text. Training took one month across four NVIDIA Pascal GPUs, with our model processing 12,500 characters per second." For smaller ones (and smaller datasets) a few hours of GPU is considered fast (at ~1$/h it is not much!). If you want to use it online, there is https://neptune.ml/ https://neptune.ml/ (no setup, $5 free credit for computing; full disclosure: created my colleagues). In any case, I would be excited to see it on some site with code (like SO) or formulae (math or stats). Especially as I am a big fan of StackExchange and analysis (vide http://p.migdal.pl/tagoverflow/ http://p.migdal.pl/tagoverflow/ :)).
- Findus23 8y ago
  Sounds nice, but my plan was to find out how much data I can handle easily on my plain simple desktop PC. You may have seen, that you can filter by site [1]. Code quite ruined my chains as it didn't appear in blocks, but rather everywhere, so I went the easy way and filtered all code blocks. I didn't filter math as I couldn't find a proper way to do it, but you can see that it gets quite messy [2] [1] https://se-simulator.lw1.at/s/math.stackexchange.com https://se-simulator.lw1.at/s/math.stackexchange.com [2] https://se-simulator.lw1.at/q/a-e-3-how-to-span https://se-simulator.lw1.at/q/a-e-3-how-to-span
  - stared 8y ago
    Well, then taking a subset of math and leaving it overnight will work. (No need to do run it on ALL sites).

stared 8y ago

Interesting. Yes, training takes way more time. In some really big projects like https://blog.openai.com/unsupervised-sentiment-neuron/ https://blog.openai.com/unsupervised-sentiment-neuron/ "We first trained a multiplicative LSTM with 4,096 units on a corpus of 82 million Amazon reviews to predict the next character in a chunk of text. Training took one month across four NVIDIA Pascal GPUs, with our model processing 12,500 characters per second." For smaller ones (and smaller datasets) a few hours of GPU is considered fast (at ~1$/h it is not much!). If you want to use it online, there is https://neptune.ml/ https://neptune.ml/ (no setup, $5 free credit for computing; full disclosure: created my colleagues). In any case, I would be excited to see it on some site with code (like SO) or formulae (math or stats). Especially as I am a big fan of StackExchange and analysis (vide http://p.migdal.pl/tagoverflow/ http://p.migdal.pl/tagoverflow/ :)).

Findus23 8y ago
Sounds nice, but my plan was to find out how much data I can handle easily on my plain simple desktop PC. You may have seen, that you can filter by site [1]. Code quite ruined my chains as it didn't appear in blocks, but rather everywhere, so I went the easy way and filtered all code blocks. I didn't filter math as I couldn't find a proper way to do it, but you can see that it gets quite messy [2] [1] https://se-simulator.lw1.at/s/math.stackexchange.com https://se-simulator.lw1.at/s/math.stackexchange.com [2] https://se-simulator.lw1.at/q/a-e-3-how-to-span https://se-simulator.lw1.at/q/a-e-3-how-to-span
- stared 8y ago
  Well, then taking a subset of math and leaving it overnight will work. (No need to do run it on ALL sites).