3 ms·

To be precise: An Elixir test suite written using ExUnit, heavily reliant on a Postgres container. The Postgres container is being emulated. I realise that thi

To be precise: An Elixir test suite written using ExUnit, heavily reliant on a Postgres container. The Postgres container is being emulated. I realise that this might not be due to the architecture or the M1 per se but due to the fact that we are emulating an x86 docker image. Regardless the test suite runs significantly slower, than if my computer would have been x86, which is the experience I am commenting on – a point that I also fleshed out even. However, I love browsing the web, watching movies, writing email, etc. on the computer, at which it excels. ---- I read the troll judgement as being passed towards me, if that was not the case, I apologise. Yes, internet communication can be difficult.

tharkun__ 4y ago
Ah yes. That's why I wrote that your suspicion on why it was slow is probably correct. We had the same thing. Emulated stuff either runs not better, way slower or even crashes and you gotta get a new image/package/compile it for the M1/M2. A database is a prime example of where any minor slowdown due to emulation can be disastrous. A database does lots of things in an optimal way to make it fast. For that it has to make assumptions (or know) how certain things behave, what properties they have. Emulation can screw with that. This is also why running your production database in a VM against virtualized storage (not even thinking docker/k8s etc. here - tho that's also bad in the same way - just plain old running that postgres in production in a VM provided by your vmware esx cluster) is not a good idea. It's just going to be abysmally slow and slow down at exactly the wrong moments even if it "usually" runs "just fine". If your tests rely on that postgres database in any significant way, I am not the least surprised that your test suite slows down like that. My advice would be to replace it with a native image or better yet, write less tests that rely on that postgres database and write the integration tests that aren't reliant on actual postgres behavior against an in-memory database like hsqldb and only have a very thin layer of tests running against an actual postgres, if you still need them at all. Fun exercise: take an ARM version of postgresql and run it under emulation on an x86 computer and see how your test suite behaves ;)
- madsbuch 4y ago
  These are very good suggestion to make it go faster. However, it is also a distraction. My raison d'etre is not to make our infrastructure run fast on Apple's hardware, but to build our core product – again an argument to support my experience of the hardware. I actually think it is Apple's responsibility to make these things run fast. replace it with a native image This is not possible, as we work with a specific version and the test DB needs to be the same version / have the same feature as production. The other advice would be terrible for our team. It would increase the behaviour surface significantly. Now we have huge discrepancies between what runs in production on what runs in test. Alternatively: We could also just eat the 13 seconds slowdown, as we doe now or don't use Apple M1s. Both are much better approaches and much more cost efficient.
  - tharkun__ 4y ago
    I don't agree that it's a distraction. You want to build your core product. You want your productivity to be high to do that. Part of that is being able to run tests with a fast turnaround so that you can be confident in your changes. Why would it be Apple's responsibility to make x86 optimized code run at native ARM speeds? That makes no sense to me. It's Apple's responsibility to make your upgrade path to ARM hardware as painless as they can. Thus the emulation layer so that for most cases you can just migrate your Intel Mac to the M1 and keep going as before. But you'll want to slowly replace every binary you can w/ the native ARM version if you want to get the performance gains of the M1. I think it's like saying "it's a distraction to change my software take advantage of multi core CPUs. It's Intel's job to keep making single core performance faster and faster". Intel was making things faster by going from upping the clock speeds to multi core and hype threaded CPUs. If you choose to call it a distraction you'll be missing out on the performance gains other people can take advantage of because their software can take advantage of having more than one CPU available. An easy example is older computer games that basically don't run any faster today than 15 years ago because they can only use one CPU and the clock speeds are almost the same. Other games are able to run enemy AI, pathfinding, the UI rendering thread etc. on different threads and thus take advantage of the multi-core CPUs. This is not possible, as we work with a specific version and the test DB needs to be the same version / have the same feature as production. I would like to understand this better. You say it's not possible. Whenever I hear that something is not possible, I instinctively want to dig deeper. Maybe it really isn't. Or maybe there's an assumption somewhere that turns out not to be universally true or that w/ a slight modification and perhaps a divide and conquer strategy it actually is possible. So I'll start w/ the first of the "5 whys": Why is it not possible? ;) The other advice would be terrible for our team. It would increase the behaviour surface significantly. Now we have huge discrepancies between what runs in production on what runs in test. What are those huge discrepancies? What exactly are you afraid of will run differently between what runs in test and in production? Keep in mind that I don't know anything about your test suite and what specifically you're testing. Maybe in your exact scenario you're right and it would potentially be bad. In many many situations I've found it's mainly just fear of the uncertainty and in fact there's no issue. Alternatively: We could also just eat the 13 seconds slowdown, as we doe now or don't use Apple M1s. Both are much better approaches and much more cost efficient. Like I said, I don't know your specific circumstances but these things can add up. 13 seconds here, 9 seconds there etc. and suddenly your test suite doesn't take 2 minutes to run but 10. That's a significant slowdown of a developer. Even 2 minutes wait time is an eternity if you're trying to move ahead w/ something and now you have to sit there and wait. Worst case your developer decides that 2 minutes is a long enough period of time to "quickly go check HN" and 45 minutes later he emerges from the browser window to see that the test suite failed 10 seconds after he switched to reading HN ;)

tharkun__ 4y ago

Ah yes. That's why I wrote that your suspicion on why it was slow is probably correct. We had the same thing. Emulated stuff either runs not better, way slower or even crashes and you gotta get a new image/package/compile it for the M1/M2. A database is a prime example of where any minor slowdown due to emulation can be disastrous. A database does lots of things in an optimal way to make it fast. For that it has to make assumptions (or know) how certain things behave, what properties they have. Emulation can screw with that. This is also why running your production database in a VM against virtualized storage (not even thinking docker/k8s etc. here - tho that's also bad in the same way - just plain old running that postgres in production in a VM provided by your vmware esx cluster) is not a good idea. It's just going to be abysmally slow and slow down at exactly the wrong moments even if it "usually" runs "just fine". If your tests rely on that postgres database in any significant way, I am not the least surprised that your test suite slows down like that. My advice would be to replace it with a native image or better yet, write less tests that rely on that postgres database and write the integration tests that aren't reliant on actual postgres behavior against an in-memory database like hsqldb and only have a very thin layer of tests running against an actual postgres, if you still need them at all. Fun exercise: take an ARM version of postgresql and run it under emulation on an x86 computer and see how your test suite behaves ;)

madsbuch 4y ago
These are very good suggestion to make it go faster. However, it is also a distraction. My raison d'etre is not to make our infrastructure run fast on Apple's hardware, but to build our core product – again an argument to support my experience of the hardware. I actually think it is Apple's responsibility to make these things run fast. replace it with a native image This is not possible, as we work with a specific version and the test DB needs to be the same version / have the same feature as production. The other advice would be terrible for our team. It would increase the behaviour surface significantly. Now we have huge discrepancies between what runs in production on what runs in test. Alternatively: We could also just eat the 13 seconds slowdown, as we doe now or don't use Apple M1s. Both are much better approaches and much more cost efficient.
- tharkun__ 4y ago
  I don't agree that it's a distraction. You want to build your core product. You want your productivity to be high to do that. Part of that is being able to run tests with a fast turnaround so that you can be confident in your changes. Why would it be Apple's responsibility to make x86 optimized code run at native ARM speeds? That makes no sense to me. It's Apple's responsibility to make your upgrade path to ARM hardware as painless as they can. Thus the emulation layer so that for most cases you can just migrate your Intel Mac to the M1 and keep going as before. But you'll want to slowly replace every binary you can w/ the native ARM version if you want to get the performance gains of the M1. I think it's like saying "it's a distraction to change my software take advantage of multi core CPUs. It's Intel's job to keep making single core performance faster and faster". Intel was making things faster by going from upping the clock speeds to multi core and hype threaded CPUs. If you choose to call it a distraction you'll be missing out on the performance gains other people can take advantage of because their software can take advantage of having more than one CPU available. An easy example is older computer games that basically don't run any faster today than 15 years ago because they can only use one CPU and the clock speeds are almost the same. Other games are able to run enemy AI, pathfinding, the UI rendering thread etc. on different threads and thus take advantage of the multi-core CPUs. This is not possible, as we work with a specific version and the test DB needs to be the same version / have the same feature as production. I would like to understand this better. You say it's not possible. Whenever I hear that something is not possible, I instinctively want to dig deeper. Maybe it really isn't. Or maybe there's an assumption somewhere that turns out not to be universally true or that w/ a slight modification and perhaps a divide and conquer strategy it actually is possible. So I'll start w/ the first of the "5 whys": Why is it not possible? ;) The other advice would be terrible for our team. It would increase the behaviour surface significantly. Now we have huge discrepancies between what runs in production on what runs in test. What are those huge discrepancies? What exactly are you afraid of will run differently between what runs in test and in production? Keep in mind that I don't know anything about your test suite and what specifically you're testing. Maybe in your exact scenario you're right and it would potentially be bad. In many many situations I've found it's mainly just fear of the uncertainty and in fact there's no issue. Alternatively: We could also just eat the 13 seconds slowdown, as we doe now or don't use Apple M1s. Both are much better approaches and much more cost efficient. Like I said, I don't know your specific circumstances but these things can add up. 13 seconds here, 9 seconds there etc. and suddenly your test suite doesn't take 2 minutes to run but 10. That's a significant slowdown of a developer. Even 2 minutes wait time is an eternity if you're trying to move ahead w/ something and now you have to sit there and wait. Worst case your developer decides that 2 minutes is a long enough period of time to "quickly go check HN" and 45 minutes later he emerges from the browser window to see that the test suite failed 10 seconds after he switched to reading HN ;)