It's effectively doable now. Facebook has already shipped a realtime smartphone style transfer app using Caffe2go (no word on how much it hammers battery life,

It's effectively doable now. Facebook has already shipped a realtime smartphone style transfer app using Caffe2go (no word on how much it hammers battery life, though). That was for a fixed style transfer network, but a variable style transfer only needs to tack on an additional smaller network to process the style image as well as the input, so <2x as much computation/model size.