I'm interested in the more general problem; how can you compare expressive power between completely different models of computation? Are neural networks more o

I'm interested in the more general problem; how can you compare expressive power between completely different models of computation? Are neural networks more or less expressive than other forms of computation like register machines? What makes them so? And what can we do to make them more expressive while still being trainable?