2 ms·
It could just be char-rnn output inside a very simple HTML wrapper. I found that a char-rnn can produce a lot of syntactically valid CSS with lots of URLs, so C
by gwern 11y ago
It could just be char-rnn output inside a very simple HTML wrapper. I found that a char-rnn can produce a lot of syntactically valid CSS with lots of URLs, so CSS+some HTML+image URLs is entirely possible: http://www.gwern.net/AB%20testing#training-a-neural-net-to-generate-css http://www.gwern.net/AB%20testing#training-a-neural-net-to-g...
The template here looks like it would go
"<script src="jquery.js"></script>
<script src="sha1.js"></script>
<script src="links.js"></script>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); https://www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-4209567-5', 'auto');
ga('send', 'pageview');
</script>
<html>
...[until generation limit, ~1600 characters; note the absence of any valid closing tags]
The low quality of the text might reflect the demands of simultaneously modeling text+HTML+CSS in a too-small RNN or insufficient training (training a high quality char-rnn/torch-rnn to convergence on the entire Geocities corpus, or even just a few hundred megabytes, could take weeks).
- gwern 11y agoYes, it turns out to be char-rnn/torch-rnn: 'RNN was 512-node, 3-layer' https://twitter.com/aanand/status/728738421758414849 https://twitter.com/aanand/status/728738421758414849 He wasn't training on his own computer so aside from being much too small (for modeling at least 3 languages, text+HTML+CSS, I would start with 1k neurons and go up from there), it probably wasn't trained to convergence either. The 50MB size is maybe a bit on the small size but probably not the root cause of the low text quality.