Re: "I’m not even sure how one would begin to approximate a highly algorithmic process (e.g., a hash function) using a neural network" You build the circuit co

Re: "I’m not even sure how one would begin to approximate a highly algorithmic process (e.g., a hash function) using a neural network" You build the circuit corresponding to the function and map it to an NN. Can this be discovered easily via GD? Absolutely no clue (though this paper says "yes"), but is it possible to approximate it? Yes, you can nail it exactly in a polynomial number of layers (if the algorithm takes poly-space, which is a necessary condition for it to run in poly-time).