It looks like lots of quick calls to numpy. The python ffi is really slow and is eating all the gains. A large matmul works fine, but not thousands of small one

It looks like lots of quick calls to numpy. The python ffi is really slow and is eating all the gains. A large matmul works fine, but not thousands of small ones. Hpy numpy port might fix this.