That would be a great contribution but unfortunately I think not an easy one to provide. A few issues 1. Most BLAS libraries aren't deterministic. Floating poi

That would be a great contribution but unfortunately I think not an easy one to provide. A few issues 1. Most BLAS libraries aren't deterministic. Floating point numbers aren't in general associative, (a + b) + c =/= a + (b + c). However most BLAS libraries will happily shuffle the order of operations around if it saves some CPU cycles, I've seen this issue cause huge differences in behavior between different CPUs before, although I think that was a pathological case. So if you want to do a test like this you really need to have everyone on the same page an installing a deterministic BLAS. 2. Training neural networks is stochastic I know you are talking about just the forward pass but I think it would be interesting to have these tests for training your network as well. Naively this sounds like it might be easy to fix (just provide a seed!), but gets very tricky when you want something to output the same code over multiple libraries, operating systems, hardware etc. Multithreading code, putting stuff on the GPU or even the cloud add even more difficulty.