5 ms·
Training a Neural Net to Generate CSS
- _lce0 11y agoit could be a nice experiment to have a basic html structure and a picture, and then attempt to generate the style until the rendered html match the image. genetic css?
- ChrisClark 11y agoThat would be really interesting to see. I bet there would be convoluted CSS hacks we never even dreamed were possible and they won't be practical at all. :)
- gwern 11y agoYes, a generative model of HTML+CSS is definitely a direction I'm nodding towards. (I discussed it briefly in the previous section about reinforcement learning in general.) I'm still hazy on the full architecture: you need an RNN to generate the CSS, you probably want to feed in a particular HTML page to target the CSS onto, the users' browsers generate the reward signal, and you can create images of the HTML+CSS combo as rendered in a web browser and bring in a convolutional network somehow... There doesn't seem to be anything quite like this in the literature that I've come across. (Actually, there's a surprising dearth of reinforcement learning in general. Very few blog posts or demos or introductory materials. It makes it hard to understand what is new about DQN or how the whole system works on a concrete coding level.)
- gburt 11y agoThe "reward signal" aka objective function seems to be the challenging part here. In the parent post's suggestion, all you'd end up with is a neural network that could /maybe/ reproduce a picture (assuming CSS was capable and the network has the approximately learnable properties necessary). It'd be more interesting to have some "quality" measure that actually meant something to evaluate outputs.
- stefs 11y agoof course you need a fitness function; why not pixel by pixel comparison of the rendered output?
- biot 11y agoIf the original image were created in a vector-based program -- in other words, something where the x,y,width,height parameters are known and stored -- you can load the generated HTML+CSS in a known reference browser and enumerate the x,y,width,height of the matching elements in the DOM. If your fitness function is a golf score, then something like: sum( abs(X_expected - X_actual) * abs(Width_expected - Width_actual) + abs(Y_expected - Y_actual) * abs(Height_expected - Height_actual) ) mapped over all elements ought to do the trick. When you hit 0, it's a perfect reproduction.
- dsfsdfd 11y agoyeah, but you need to render in a headless browser. This might take 0.1 seconds per webpage which is extremely slow when trying to use in the context of reinforcement learning. I thought about trying to do MCMC over a beam search through the rnn output, but ran out of time and patience.
- gburt 11y agoMy point was that this is boring. A fitness function that evaluated some notion of how "pretty" a page was would be cooler than being able to regenerate a screenshot's CSS (in a likely very complex form).
- deleted 11y ago[deleted]
- kiba 11y agoIf you like his work, consider donating to gwern on patreon. https://www.patreon.com/gwern?ty=h https://www.patreon.com/gwern?ty=h Note: I am obviously not gwern.
- VOYD 11y agoTraining a NN to be subjective? Cool story bro, keep telling it.
- arturhoo 11y agoIf you find this combination of Natural Computation and arts (or styling in this case), you might want to take a look on papers published on these tracks: * Evolutionary Computation for Music, Art, and Creativity - http://cilab.cs.ccu.edu.tw/ci-tf/ECMAC2015.html http://cilab.cs.ccu.edu.tw/ci-tf/ECMAC2015.html * EvoMUSART - http://www.evostar.org/2016/cfp_evomusart.php http://www.evostar.org/2016/cfp_evomusart.php * International Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging - http://expressive.richardt.name/2015/CAe/Home http://expressive.richardt.name/2015/CAe/Home * Generative Art Conference - http://www.generativeart.com/ http://www.generativeart.com/ I've seen a few papers working on CSS in the past.
- seba_dos1 11y ago"1 one parse error on *zoom: 2 !important;" That might be a parse error according to validator, but it's actually a pretty commonly used hack for old Internet Explorers - real browsers drop such invalid declaration, while IE (up to 6 I think, although 7 still accepted some other character IIRC) parsed it anyway and ignored the asterisk. What's more - this quirk is commonly used with, guess what, the "zoom" property, which is one of the common ways to trigger "hasLayout" mode in IE engine (although you'd most likely use it with value "1" instead of "2"). So this NN simply wrote a compatibility hack for IE6 :)
- richmarr 11y agoIt's reassuring to think that there'll soon be a growing population of smart systems and artificial intelligences that hate Internet Explorer too.
- whalabi 11y agoCan it center vertically?
- mizzao 11y agoProbably, if trained with data containing display: flex
- dzhiurgis 11y agoAbility to generate sense out of thin air must be nightmare for anyone trying to fight spam. Will this ever make into our inboxes?
- tempodox 11y agoLetting my imagination run wild, I'd add a speech recognition module, generating CSS from a spoken description of layout etc. After some busy work along that line, we would arrive at a generation of PLs where you don't write source code but have it generated from interpreted speech input. I'm not so sure I (as a programmer) would really want that, but it would seem a rather obvious line of development.
- noobie 11y agoOh, I was looking forward to getting into ML and NN, but given my crappy i3 CPU/integrated GPU, seems like I am not.
- Houshalter 11y agoYou can do ML. But you won't match the performance of state of the art deep learning networks. Training on CPU isn't impossible. Gwern said it was a 20x slowdown, others said it was only as bad as 10x. But for smaller models it should be feasible.
- ganarajpr 11y agoSomeone correct me if I am wrong here, but these char-rnn based outputs are great to look at and experiment with - but they dont seem to be of any practical use. All that they are capable of showing is that an RNN can "remember" things. The biggest question is how do you make any use of these things ? Is it possible to nudge the RNN in a particular direction - so that it produces something that we want ? Perhaps there is an answer to this in Alex Graves et al. Paper on Handwriting generation. A more thorough explanation or exploration in this direction would perhaps help ?. Anyone really working on generating CSS ( conditioned! ) ?
- azernik 11y agoNo experience actually trying, but I'd suspect the first step would be training a system on combinations of CSS and HTML; all the really interesting behavior comes from the interaction between the two. Then I imagine you could do interesting stuff by e.g. constraining the HTML and seeing what kind of CSS was spit out.