7 ms·
> Getting your application aligned with the NUMA model makes way more difference in performance than anything else. Choose a better data format (binary format
by fnord123 6y ago
> Getting your application aligned with the NUMA model makes way more difference in performance than anything else.
Choose a better data format (binary format rather than json).
Compress data.
Remove repeated work (lift out of loops).
Get all the data from the DB up front.
Index.
Cache.
Parallelise.
cpusets (aligned with the NUMA model)
- TeMPOraL 6y ago> Remove repeated work (lift out of loops). > Get all the data from the DB up front. Between those two, I'd add: Move computation to the data where appropriate. Let the data layer (particularly the database itself) do the heavy lifting.
- UK-Al05 6y agoConsidering the databases often end up being the bottleneck. It may not always be wise. So its a small amount of data, but large computation it might be tempted to do it on the application. But its a large amount of data, I/O maybe a problem in doing it on the app.
- TeMPOraL 6y agoThat's why I hedged my bets with "where appropriate" there :). You're of course right, and it could be that DB being a compute bottleneck is the most common case - but I've seen or been involved in projects where the bottleneck was DB I/O and processing unnecessary queries, because the application was doing most of the calculations.
- fnord123 6y agoI would also add 'use specialised hardware (simd, gpgpu)'