4 ms·

Some of us write software for a living. You give me two more cores, I'll use them.

zasdffaa 4y ago
I'm a dev too, my machine is a dual core haswell. It depends very much what you're doing of course (compiling linux kernel repeatedly perhaps) but if you have a little you can get a heck of a lot out of it with experience, although you surely will hit a wall sometime. But 32 cores not enough is just unimaginable for me.
- devmor 4y ago
  Can't speak for the guy you're replying to, but I regularly saturate my 16-core CPU as a web developer. I often work on environments that need 20+ services locally virtualized to properly simulate the environment they'll be used in.
  - zasdffaa 4y ago
    Worked in a place where the local architecture astronaut had taken over and we were subject to his insane designs that were virtualisation and microservices everywhere. A monolithic lump on a server would have been better, cheaper, faster in every way and needed many fewer cores. But your case, ok, understood. BTW The biggest slowdown IME is not from lack of cores but lack of RAM. Odd that no-one's mentioned that. Or using a server with larger numbers of memory channels to keep their cores fed. I/O > cycles more often than not.
- jiggawatts 4y ago
  So in $dayjob I have used an 8-core (16-thread) CPU to 100% regularly, as well as fully utilising 64-GB of RAM. I would be able to use 16 cores, 32, or more. Random examples: Trawling through 70 GB of web server log files going back years with ad-hoc analytics. I tried ingesting this into a cloud service but it was taking forever, and cost a non-trivial amount of money. It was also dog slow compared to just doing it locally with SQL Server Clustered Columnstore. Compiling Chromium. I'm not even a browser developer or anything, but there was a "global" change that I had to preview ahead of time, so I made a custom Chrome build that had the change embedded in it. I also made a matching build without the change to enable A/B comparison testing. Server base image task sequence development. You would think a server is ideal for this, but no! It's very I/O heavy, and doing it locally in Hyper-V using an NVMe drive can make this 4-5x as productive as any shared platform with throttled disks. There are zero cloud services that can even approach a Samsung Pro 980 SSD for reasonable costs. Multi-server lab tests. Compared to cloud services, it's faster and easier to build a local pool of machines for testing how they interact. With 64 GB of memory, I can (and have) easily run 20+ machines. Compared to cloud platforms, I can do things like thin-cloning to build (and then delete and rebuild) these nearly instantly -- about 5 seconds or less in some cases. Even the relatively heavyweight Windows Server 2022 VMs boots in just 1.4 seconds! Some build jobs require nearly 100 GB of content to be deployed, and the upload to the cloud is a problem in some customer environments. E.g.: due to security restrictions. Doing it locally is easier and faster. Photography and video editing. I do both as a hobby. Lightroom will cheerfully use whatever number of CPUs (and GPUs!) you have to process images. My recent hardware upgrade made a very noticeable difference to the workflow. Similarly, photogrammetry and other advanced 3D modelling workflows can use up virtually any amount of compute. Password cracking. I occasionally need to reverse a password hash for work, typically to fix lost access to some ancient piece of network gear. Sometimes it's for verifying password strength where the plain-text password is not known. I could distribute this across a pool of cloud VMs, but my lab subscriptions have hard limits and this is sensitive data. Automated searches through code for a breaking change. E.g.: scripting the checkout, build, test, etc... in a loop. (Git bisect doesn't solve every problem) VS Code now has ripgrep integrated, so "find in files" will use every CPU. I have about 100 GB of files on my laptop from various old projects, so this is a very helpful feature for digging up some code snippet that I only vaguely remember writing. It can find a match for a regular expression at 700 MB/s using 16 threads at 100%. Defragmenting moderately large (50-500 GB) databases. I have some customers that have built up horrendously fragmented databases over decades, but can't afford the IOPS and/or downtime to fully rebuild all of the tables and indexes. I recently utilised the aforementioned Samsung SSD to defragment an 80 GB database in just minutes, that would have taken days or weeks on the production server. Again, transferring this out to a cloud is a no-go, and the problem was that typical server storage is much slower than a typical laptop. Locally I could get 200K IOPS sustained. SQL Server Developer Edition is the same as Enterprise, and will utilise all cores to rebuild indexes in parallel. Etc, etc... If you know what you are doing, you can make the CPUs go "brr...".
  - zasdffaa 4y ago
    Cloud stuff is never going to make sense, local machine it is. I do see what you're saying and here it seems like a good fit for a massive machine. No argument here. NB. "Trawling through 70 GB of web server log files" unless you partition your search you're going to be doing a lot of IO due to having only 64-GB of RAM. Chucking in extra mem to keep it all ram-resident will beat cores any day.
    - jiggawatts 4y ago
      You would be... surprised. The query engine in Microsoft SQL Server is absurdly advanced, and simply beats the pants off just about everything else for complex, general-case queries. There are other engines that are better tuned for special cases, but if you've got "stuff" that's within the capacity of a single box, it's really hard to beat MSSQL. Let's expand on my example above: Clustered ColumnStore.[1] It's basically PowerBI or Vertica embedded within the traditional MSSQL engine. This extension enables the ability to have a per-column (vertical) on-disk storage format instead of the traditional row-store format most databases have. You can have it as a secondary acceleration index, or as a clustered (primary) index. In the latter case, it'll compress your raw table data. For my log files it was 33-to-1 compression (3% of the original file size). Second, even if the data doesn't fit into memory, it'll utilise the columnar storage to simply ignore (skip) columns it doesn't need for the query. I was computing performance statistics, so it was just reading in some numbers instead of the text. A further 100-to-1 optimisation can be gained here. It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores. All of this I/O is fully asynchronous with a huge queue depth. Expect to see 200K IOPS on a laptop, easily. Once in memory, it uses Intel AVX SIMD instructions to chew through the data 8 rows per clock tick per core. With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second. What's neat about it is that it's fully general: Columnstore can be mixed -- in a single query -- with in-memory tables, on-disk tables, traditional row-store, and external tables streamed in over the network. The engine will just... "figure this out". It'll build indexes on the fly if it has to, ideally in memory, but failing that... it'll stream disk-to-disk as required. I've seen "sort spills" reach 600K IOPS on my laptop. If I have to scan through years of logs and be able to join it against additional tables for "enriching" it, then it's hard to beat. Sure, there's a scale where the are dedicated log analytics databases, but they have their quirks, limitations, and costs. I take your incredulity that others can fully utilise their computers as a sign that general knowledge of just what is possible is poor. People have pointed out that some simple shell scripts can outperform Hadoop clusters up to a surprising scale. Similarly, knowledge of the capabilities of traditional database engines like MSSQL is oddly lacking, even amongst developers. Not to mention more esoteric ways of making your computer do your bidding, such as Mathematica, Julia, GPU codes, or whatever. Your computer is a power tool for your squishy, manual-labour meat brain! A lever for the mind. Learn to utilise it better and you'll be better at thinking. [1] https://docs.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview?view=sql-server-ver15 https://docs.microsoft.com/en-us/sql/relational-databases/in...
- zhdc1 4y ago
  Not the same use case, but we regularly saturate 48 cores doing simple - and heavily parallelized - text analysis and language processing work. So much so that I’m now trying to figure out whether we can somehow cheaply move to a dual Epyc 7742 setup.
  - zasdffaa 4y ago
    Whenever I read stuff like this my inner optimiser starts wondering where the CPU is going and why - massive GC cycles, inefficient regexp stuff, poor algo choice, bad cache locality, etc etc. At the risk of ill manners but i must ask, have you looked at all this because I've got in some cases orders of magnitude speedup by popping the hood and instrumenting.
    - _zoltan_ 4y ago
      It's usually cheaper to buy more hardware than to burn engineering hours except at the very low scale (1-2 engineer) and at FAANG scale.
      - zasdffaa 4y ago
        No. It depends. You weigh it up. And a little (cheap) nous can often beat a truckload of hardware.
- aunty_helen 4y ago
  ai. Our models train in a reasonable time (or we push them onto gpu) but doing the pipelines and etl when developing is a big time sink. We could sit there and optimize the approach, but the time that would take and the complexity added, it would be cheaper to just buy more powerful computers. For example, on reasonably new 8 core 64gb ram, 7gbs HDD machine, I optimized a calculation for one of the features down from 2-3 days to about 15 minutes. Throwing cores or ram or disk at this algo before would’ve got nowhere. Now htop glows like a Christmas tree when it’s running. This was costly to do but it was the only way. We could talk about python being slow and blah blah blah use C but once again, whenever we do tricky stuff with code that then gets shared amongst a bunch of python devs I need to think about the holistic cost including maintenance, ease of changes, bug fixing, resource that actually know c, client appetite, what the production load is going to look like etc And finally, at some point a client will just come along with another million rows of data

I'm a dev too, my machine is a dual core haswell. It depends very much what you're doing of course (compiling linux kernel repeatedly perhaps) but if you have a little you can get a heck of a lot out of it with experience, although you surely will hit a wall sometime. But 32 cores not enough is just unimaginable for me.

devmor 4y ago
Can't speak for the guy you're replying to, but I regularly saturate my 16-core CPU as a web developer. I often work on environments that need 20+ services locally virtualized to properly simulate the environment they'll be used in.
- zasdffaa 4y ago
  Worked in a place where the local architecture astronaut had taken over and we were subject to his insane designs that were virtualisation and microservices everywhere. A monolithic lump on a server would have been better, cheaper, faster in every way and needed many fewer cores. But your case, ok, understood. BTW The biggest slowdown IME is not from lack of cores but lack of RAM. Odd that no-one's mentioned that. Or using a server with larger numbers of memory channels to keep their cores fed. I/O > cycles more often than not.
jiggawatts 4y ago
So in $dayjob I have used an 8-core (16-thread) CPU to 100% regularly, as well as fully utilising 64-GB of RAM. I would be able to use 16 cores, 32, or more. Random examples: Trawling through 70 GB of web server log files going back years with ad-hoc analytics. I tried ingesting this into a cloud service but it was taking forever, and cost a non-trivial amount of money. It was also dog slow compared to just doing it locally with SQL Server Clustered Columnstore. Compiling Chromium. I'm not even a browser developer or anything, but there was a "global" change that I had to preview ahead of time, so I made a custom Chrome build that had the change embedded in it. I also made a matching build without the change to enable A/B comparison testing. Server base image task sequence development. You would think a server is ideal for this, but no! It's very I/O heavy, and doing it locally in Hyper-V using an NVMe drive can make this 4-5x as productive as any shared platform with throttled disks. There are zero cloud services that can even approach a Samsung Pro 980 SSD for reasonable costs. Multi-server lab tests. Compared to cloud services, it's faster and easier to build a local pool of machines for testing how they interact. With 64 GB of memory, I can (and have) easily run 20+ machines. Compared to cloud platforms, I can do things like thin-cloning to build (and then delete and rebuild) these nearly instantly -- about 5 seconds or less in some cases. Even the relatively heavyweight Windows Server 2022 VMs boots in just 1.4 seconds! Some build jobs require nearly 100 GB of content to be deployed, and the upload to the cloud is a problem in some customer environments. E.g.: due to security restrictions. Doing it locally is easier and faster. Photography and video editing. I do both as a hobby. Lightroom will cheerfully use whatever number of CPUs (and GPUs!) you have to process images. My recent hardware upgrade made a very noticeable difference to the workflow. Similarly, photogrammetry and other advanced 3D modelling workflows can use up virtually any amount of compute. Password cracking. I occasionally need to reverse a password hash for work, typically to fix lost access to some ancient piece of network gear. Sometimes it's for verifying password strength where the plain-text password is not known. I could distribute this across a pool of cloud VMs, but my lab subscriptions have hard limits and this is sensitive data. Automated searches through code for a breaking change. E.g.: scripting the checkout, build, test, etc... in a loop. (Git bisect doesn't solve every problem) VS Code now has ripgrep integrated, so "find in files" will use every CPU. I have about 100 GB of files on my laptop from various old projects, so this is a very helpful feature for digging up some code snippet that I only vaguely remember writing. It can find a match for a regular expression at 700 MB/s using 16 threads at 100%. Defragmenting moderately large (50-500 GB) databases. I have some customers that have built up horrendously fragmented databases over decades, but can't afford the IOPS and/or downtime to fully rebuild all of the tables and indexes. I recently utilised the aforementioned Samsung SSD to defragment an 80 GB database in just minutes, that would have taken days or weeks on the production server. Again, transferring this out to a cloud is a no-go, and the problem was that typical server storage is much slower than a typical laptop. Locally I could get 200K IOPS sustained. SQL Server Developer Edition is the same as Enterprise, and will utilise all cores to rebuild indexes in parallel. Etc, etc... If you know what you are doing, you can make the CPUs go "brr...".
- zasdffaa 4y ago
  Cloud stuff is never going to make sense, local machine it is. I do see what you're saying and here it seems like a good fit for a massive machine. No argument here. NB. "Trawling through 70 GB of web server log files" unless you partition your search you're going to be doing a lot of IO due to having only 64-GB of RAM. Chucking in extra mem to keep it all ram-resident will beat cores any day.
  - jiggawatts 4y ago
    You would be... surprised. The query engine in Microsoft SQL Server is absurdly advanced, and simply beats the pants off just about everything else for complex, general-case queries. There are other engines that are better tuned for special cases, but if you've got "stuff" that's within the capacity of a single box, it's really hard to beat MSSQL. Let's expand on my example above: Clustered ColumnStore.[1] It's basically PowerBI or Vertica embedded within the traditional MSSQL engine. This extension enables the ability to have a per-column (vertical) on-disk storage format instead of the traditional row-store format most databases have. You can have it as a secondary acceleration index, or as a clustered (primary) index. In the latter case, it'll compress your raw table data. For my log files it was 33-to-1 compression (3% of the original file size). Second, even if the data doesn't fit into memory, it'll utilise the columnar storage to simply ignore (skip) columns it doesn't need for the query. I was computing performance statistics, so it was just reading in some numbers instead of the text. A further 100-to-1 optimisation can be gained here. It then parallelises this across all cores/hyperthreads, so I get 100% utilisation on 16 cores. All of this I/O is fully asynchronous with a huge queue depth. Expect to see 200K IOPS on a laptop, easily. Once in memory, it uses Intel AVX SIMD instructions to chew through the data 8 rows per clock tick per core. With 8 cores going at once, this is 64 rows per clock, or up to 234 billion rows processed per second. What's neat about it is that it's fully general: Columnstore can be mixed -- in a single query -- with in-memory tables, on-disk tables, traditional row-store, and external tables streamed in over the network. The engine will just... "figure this out". It'll build indexes on the fly if it has to, ideally in memory, but failing that... it'll stream disk-to-disk as required. I've seen "sort spills" reach 600K IOPS on my laptop. If I have to scan through years of logs and be able to join it against additional tables for "enriching" it, then it's hard to beat. Sure, there's a scale where the are dedicated log analytics databases, but they have their quirks, limitations, and costs. I take your incredulity that others can fully utilise their computers as a sign that general knowledge of just what is possible is poor. People have pointed out that some simple shell scripts can outperform Hadoop clusters up to a surprising scale. Similarly, knowledge of the capabilities of traditional database engines like MSSQL is oddly lacking, even amongst developers. Not to mention more esoteric ways of making your computer do your bidding, such as Mathematica, Julia, GPU codes, or whatever. Your computer is a power tool for your squishy, manual-labour meat brain! A lever for the mind. Learn to utilise it better and you'll be better at thinking. [1] https://docs.microsoft.com/en-us/sql/relational-databases/indexes/columnstore-indexes-overview?view=sql-server-ver15 https://docs.microsoft.com/en-us/sql/relational-databases/in...
zhdc1 4y ago
Not the same use case, but we regularly saturate 48 cores doing simple - and heavily parallelized - text analysis and language processing work. So much so that I’m now trying to figure out whether we can somehow cheaply move to a dual Epyc 7742 setup.
- zasdffaa 4y ago
  Whenever I read stuff like this my inner optimiser starts wondering where the CPU is going and why - massive GC cycles, inefficient regexp stuff, poor algo choice, bad cache locality, etc etc. At the risk of ill manners but i must ask, have you looked at all this because I've got in some cases orders of magnitude speedup by popping the hood and instrumenting.
  - _zoltan_ 4y ago
    It's usually cheaper to buy more hardware than to burn engineering hours except at the very low scale (1-2 engineer) and at FAANG scale.
    - zasdffaa 4y ago
      No. It depends. You weigh it up. And a little (cheap) nous can often beat a truckload of hardware.
aunty_helen 4y ago
ai. Our models train in a reasonable time (or we push them onto gpu) but doing the pipelines and etl when developing is a big time sink. We could sit there and optimize the approach, but the time that would take and the complexity added, it would be cheaper to just buy more powerful computers. For example, on reasonably new 8 core 64gb ram, 7gbs HDD machine, I optimized a calculation for one of the features down from 2-3 days to about 15 minutes. Throwing cores or ram or disk at this algo before would’ve got nowhere. Now htop glows like a Christmas tree when it’s running. This was costly to do but it was the only way. We could talk about python being slow and blah blah blah use C but once again, whenever we do tricky stuff with code that then gets shared amongst a bunch of python devs I need to think about the holistic cost including maintenance, ease of changes, bug fixing, resource that actually know c, client appetite, what the production load is going to look like etc And finally, at some point a client will just come along with another million rows of data