8 ms·

Question: I looked up the price of an 18 core Xeon processor, with a clock speed under 3.0 GHz, and got $4000+. So, why be willing to pay so much per core

by graycat 11y ago

Question: I looked up the price of an 18 core Xeon processor, with a clock speed under 3.0 GHz, and got $4000+. So, why be willing to pay so much per core for such slow cores? At first I guessed that having lots of cores per processor would reduce the number of processors needed and, then, maybe for some software reduce licensing costs, but the IBM Power processors likely are going to be running Linux. So, the goal is to reduce licensing costs for Oracle? Something else? Or licensing costs has nothing to do with it? As I plan my server farm for my startup, what am I missing about the value of paying $4000+ for 18 relatively slow cores?

yshalabi 11y ago
These processors have a lot of advanced features for big-time computing users (warehouse scale, super computers, etc). You can't just look at frequency or cachesize or number of cores for this chip. There are reliability features, quality of service features, and overall capabilities that consumer grade processors don't have. Also I don't recommend you try to outfit your startups server farm. I dont know your needs, but you should find someone who can figure out what you really need.
- graycat 11y ago
  Thanks! Initially all I "need" is a relatively high end of a consumer grade mid-tower case, e.g., an AMD 8 core processor, 32 GB of ECC main memory, several hard disks, on a current motherboard for less than $150 or so running Windows Server or maybe, initially, just Windows 7 Pro. Keep that on average half busy 24 x 7 for a month, and I will be able to afford more. If I get, say, two wire rack shelf units 18 x 48 x 72", fill them with a good router and mid tower cases, and keep all that on average half busy 24 x 7, then I will consider a colocation facility or the cloud. A few miles from me is a nicely big, fully serious colocation facility that offers dual 10 GbE Internet connections, etc. No joke: One of those shelf units has room for about 12 mid tower cases. Keep two such shelf units busy, and that will max out what I'm willing to pursue without a lot more server farm expertise than I have (or want to get) and also make cash not a problem. So, one hope is that I will be able to pay some experts, two or three days at a time, to hold my hand into more -- reliable electrical power, HVAC, floor space, cabling, racks, servers for the racks, internals of the servers for the racks, e.g., maybe Xeon, automation for software installation, system management, system monitoring, farm performance analysis, fail-over, virtual machine exploitation, security, test systems, development systems, organized code repository and testing, all relevant documentation, training, recruiting, HR, legal, real estate leasing, janitorial, physical security, etc. Or, right, just use a cloud! But for the high end Xeon processors, I'm looking ahead. E.g., a motherboard with two Xeons with 18 cores each, all in just one full tower case, or two for failover, or three considering testing, might provide enough computing for my startup all the way to exit so that I could avoid taking seriously what I'd have to do with 100 people and 50,000 square feet of server farm, dual optical fiber connections to an Internet backbone point of presence, etc. So, that's why looking into high end Xeons now is not totally wasted time. Thanks for the help!
vessenes 11y ago
Those cores are going to be sold to you largely on their memory bandwidth benefits for the top end of the Xeon range. If you need memory bandwidth, you will gladly pay for them, and look seriously at IBM (something we just did). If you don't know that you need memory bandwidth, just skip them. Also, I would suggest you check out OVH or Hetzner for a new startup -- it's (sadly) very unlikely that you will need enough servers for long enough to make buying your own a good plan.
- graycat 11y ago
  Thanks! > Those cores are going to be sold to you largely on their memory bandwidth benefits for the top end of the Xeon range. That is, (A) bandwidth, bytes moved per second or (B) total permitted memory size, say, 1/2 TB? My guess is that memory sizes of 1/2 TB require registered memory, that is, a register in the memory to simplify timing which can be challenging for such large memories, but the use of the register is an intermediate stop on the way to/from the processor and its cache(s) so, really, reduces bytes per second that might be achieved with, say, the simpler, consumer 1600 MHz, DDR3? Of course, other issues could include number of electronically independent channels to/from memory, address interleaved memory, etc.? > look seriously at IBM I looked at the article; IBM seems to be trying to sell hardware (again!). Okay. So far my software is all written for Windows, and my guess is that Windows (7 Pro or Server) doesn't run on IBM's Power processors? And even if Windows does run, lots of other software that runs on Windows and Intel x86 likely won't run on IBM Power?
  - vessenes 11y ago
    For where you sound like you're at, I wouldn't even worry about it. Usually when we say bandwidth, we mean bytes/second, to and from the caches and main system memory. But, really, don't worry about it -- for windows, ovh or hetzner, or if your workload varies a lot, azure or AWS are almost certainly what you want; put the time in to product development until it's so successful that you _need_ the tech help to scale.
    - graycat 11y ago
wmf 11y ago
I found a 14-core Xeon to give optimal price/performance for my usage. But I guess if that's simply not fast enough to run your app there's always the 18-core; the cores may be slow but the total "MIPS" is still higher.
- graycat 11y ago
  Thanks! Initially and no doubt for a long time, 14 cores will be plenty for my startup. I've done my software development on one core at 1.8 GHz. Going live, I'm considering 4 cores at about 3.2 GHz or 8 cores at 4.0 GHz -- for total cost down in the very low rent district. Or, the month my startup keeps 14 cores, or 8 cores, or even 4 cores, busy will be when I order a new, high end Corvette and go shopping for a really nice house! My question was not really about being "fast enough" but just what the heck is the cost per core per clock Hz? E.g., $160 buys an 8 core AMD processor at 4.0 GHz, and a high end Xeon costs much, much more per core per clock Hz. I can see that an Intel processor would save on electrical power, but at least initially that will be, all other things considered, small potatoes.
i336_ 11y ago
Your questions prompted a bit of curious research on what you're pursuing. Your post history is very rewarding to pour over, and I'm glad I did. You definitely should publish a book and set the record straight once and for all. I think it would bring you a lot of closure. And you should make more noise about the mathematical models you've developed so you get more recognition. I really hope this search/probability venture of yours pans out, and I'm quite interested to learn more about it. In fact, I'd feel very privileged if I could hitchhike along for a while as a curious passerby - not directly involved, more just observing and learning. I did a little digging, but I can't be certain since you're rather good at the anonymity thing - is this your current email address? Obfuscated for privacy: put list ('73'x||'69'x||'67'x||'6d'x||'61'x||'77'x||'61'x||'69'x|| '74'x||'65'x||'40'x||'6f'x||'70'x||'74'x||'6f'x||'6e'x|| '6c'x||'69'x||'6e'x||'65'x||'2e'x||'6e'x||'65'x||'74'x); I think the best thing that would answer your questions would be direct testing, both for the experience of trying out a bunch of different hardware, and also to work out how to get the best bang for your buck with the workload you're using. Throwing your existing code on various different S3 configurations would probably be your best bet to start with. On hardware, though, Xeon processors incorporate extra instructions appropriate to high-performance computing, whereas desktop/consumer-class CPUs include hardware acceleration and chipset support for theft prevention, basic media acceleration, wireless projector connectivity, etc. In a server environment, workloads tend to need scalability more than processor power - it's generally significantly faster to run 100 tasks in parallel on slow hardware than in sequence on fast hardware. A desktop context is usually the opposite though - running a small number of applications, each of which generally needs to run as quickly as possible so the system feels snappy. It also generally boils down to manufacturing. You don't get 24-core desktop-oriented i7 chips yet, because you can't yet pack 24 4GHz+ cores into a CPU die. Oh, and that datacenter you mentioned, with dual 10GbE - is that $30k a month, or more?
- graycat 11y ago
  > Oh, and that datacenter you mentioned, with dual 10GbE - is that $30k a month, or more? I don't recall the details of their pricing. The colocation facility is at the old IBM Wappingers Falls, Myers Corners lab complex. There were several nice buildings, one with a lot of well done raised floor. IIRC, they were getting revenue from having a fiber roughly down the Hudson River for the 70 miles or so to Wall Street. I don't plan to go for a $4000+ Xeon processor soon, but I wanted to understand what the value is and see what I was missing. Or can get an eight core AMD processor at 4.0 GHz for about $160. And, say, an 18 core Xeon at 3.3 GHz for $4000. So, the AMD has price per core per GH of 160 / ( 8 * 4.0 ) = 5 and the Xeon has 4000 / ( 18 * 3.2 ) = 69.44 for a ratio of 69.44 / 5 = 13.89 That's 1389%, and that's a biggie. For my startup, the software for at least early production appears to be ready. Now I'm loading some initial data and generally, e.g., for my server farm, doing some planning. Then I will do some testing, alpha, beta and possibly some revisions. Then I will go live and go for publicity, users, ads, revenue, earnings, and growth. The basic core applied math and software really should work as I intend. The main question is, will lots of users like the site? If so, then the project should become a big thing. From my software and server farm architecture and software timings, one 8 core AMD processor at 4.0 GHz should be able to support on average, 24 x 7, one new user a second. Then I should be able to get, ballpark, $207,360 dollars a month in revenue. In that case, growth should be fast, and I should consider Xeon processors if they have some significant advantages. E.g., some builders offer two Xeon processors, 18 cores each, on a motherboard in a full tower case with lots of room for hard disks. So, that would be 36 cores, and four of the 8 core AMD processors would be 32 cores in four cases. Then, sure, for more, get some standard racks and put in servers designed for racks. I have a lot of flexibility in how many processors and motherboards I use because the software and server farm architecture is highly scalable just via simple sharding. The architecture has just five boxes, Web server, Web session state server, SQL Server, and two specialized servers full of applied math. Each of these five can run in their own server(s), or, for a good start, all five can run in one server.

by graycat 11y ago

yshalabi 11y ago

These processors have a lot of advanced features for big-time computing users (warehouse scale, super computers, etc). You can't just look at frequency or cachesize or number of cores for this chip. There are reliability features, quality of service features, and overall capabilities that consumer grade processors don't have. Also I don't recommend you try to outfit your startups server farm. I dont know your needs, but you should find someone who can figure out what you really need.

graycat 11y ago
Thanks! Initially all I "need" is a relatively high end of a consumer grade mid-tower case, e.g., an AMD 8 core processor, 32 GB of ECC main memory, several hard disks, on a current motherboard for less than $150 or so running Windows Server or maybe, initially, just Windows 7 Pro. Keep that on average half busy 24 x 7 for a month, and I will be able to afford more. If I get, say, two wire rack shelf units 18 x 48 x 72", fill them with a good router and mid tower cases, and keep all that on average half busy 24 x 7, then I will consider a colocation facility or the cloud. A few miles from me is a nicely big, fully serious colocation facility that offers dual 10 GbE Internet connections, etc. No joke: One of those shelf units has room for about 12 mid tower cases. Keep two such shelf units busy, and that will max out what I'm willing to pursue without a lot more server farm expertise than I have (or want to get) and also make cash not a problem. So, one hope is that I will be able to pay some experts, two or three days at a time, to hold my hand into more -- reliable electrical power, HVAC, floor space, cabling, racks, servers for the racks, internals of the servers for the racks, e.g., maybe Xeon, automation for software installation, system management, system monitoring, farm performance analysis, fail-over, virtual machine exploitation, security, test systems, development systems, organized code repository and testing, all relevant documentation, training, recruiting, HR, legal, real estate leasing, janitorial, physical security, etc. Or, right, just use a cloud! But for the high end Xeon processors, I'm looking ahead. E.g., a motherboard with two Xeons with 18 cores each, all in just one full tower case, or two for failover, or three considering testing, might provide enough computing for my startup all the way to exit so that I could avoid taking seriously what I'd have to do with 100 people and 50,000 square feet of server farm, dual optical fiber connections to an Internet backbone point of presence, etc. So, that's why looking into high end Xeons now is not totally wasted time. Thanks for the help!

vessenes 11y ago

Those cores are going to be sold to you largely on their memory bandwidth benefits for the top end of the Xeon range. If you need memory bandwidth, you will gladly pay for them, and look seriously at IBM (something we just did). If you don't know that you need memory bandwidth, just skip them. Also, I would suggest you check out OVH or Hetzner for a new startup -- it's (sadly) very unlikely that you will need enough servers for long enough to make buying your own a good plan.

graycat 11y ago
Thanks! > Those cores are going to be sold to you largely on their memory bandwidth benefits for the top end of the Xeon range. That is, (A) bandwidth, bytes moved per second or (B) total permitted memory size, say, 1/2 TB? My guess is that memory sizes of 1/2 TB require registered memory, that is, a register in the memory to simplify timing which can be challenging for such large memories, but the use of the register is an intermediate stop on the way to/from the processor and its cache(s) so, really, reduces bytes per second that might be achieved with, say, the simpler, consumer 1600 MHz, DDR3? Of course, other issues could include number of electronically independent channels to/from memory, address interleaved memory, etc.? > look seriously at IBM I looked at the article; IBM seems to be trying to sell hardware (again!). Okay. So far my software is all written for Windows, and my guess is that Windows (7 Pro or Server) doesn't run on IBM's Power processors? And even if Windows does run, lots of other software that runs on Windows and Intel x86 likely won't run on IBM Power?
- vessenes 11y ago
  For where you sound like you're at, I wouldn't even worry about it. Usually when we say bandwidth, we mean bytes/second, to and from the caches and main system memory. But, really, don't worry about it -- for windows, ovh or hetzner, or if your workload varies a lot, azure or AWS are almost certainly what you want; put the time in to product development until it's so successful that you _need_ the tech help to scale.
  - graycat 11y ago

wmf 11y ago

I found a 14-core Xeon to give optimal price/performance for my usage. But I guess if that's simply not fast enough to run your app there's always the 18-core; the cores may be slow but the total "MIPS" is still higher.

graycat 11y ago
Thanks! Initially and no doubt for a long time, 14 cores will be plenty for my startup. I've done my software development on one core at 1.8 GHz. Going live, I'm considering 4 cores at about 3.2 GHz or 8 cores at 4.0 GHz -- for total cost down in the very low rent district. Or, the month my startup keeps 14 cores, or 8 cores, or even 4 cores, busy will be when I order a new, high end Corvette and go shopping for a really nice house! My question was not really about being "fast enough" but just what the heck is the cost per core per clock Hz? E.g., $160 buys an 8 core AMD processor at 4.0 GHz, and a high end Xeon costs much, much more per core per clock Hz. I can see that an Intel processor would save on electrical power, but at least initially that will be, all other things considered, small potatoes.

i336_ 11y ago

Your questions prompted a bit of curious research on what you're pursuing. Your post history is very rewarding to pour over, and I'm glad I did. You definitely should publish a book and set the record straight once and for all. I think it would bring you a lot of closure. And you should make more noise about the mathematical models you've developed so you get more recognition. I really hope this search/probability venture of yours pans out, and I'm quite interested to learn more about it. In fact, I'd feel very privileged if I could hitchhike along for a while as a curious passerby - not directly involved, more just observing and learning. I did a little digging, but I can't be certain since you're rather good at the anonymity thing - is this your current email address? Obfuscated for privacy: put list ('73'x||'69'x||'67'x||'6d'x||'61'x||'77'x||'61'x||'69'x|| '74'x||'65'x||'40'x||'6f'x||'70'x||'74'x||'6f'x||'6e'x|| '6c'x||'69'x||'6e'x||'65'x||'2e'x||'6e'x||'65'x||'74'x); I think the best thing that would answer your questions would be direct testing, both for the experience of trying out a bunch of different hardware, and also to work out how to get the best bang for your buck with the workload you're using. Throwing your existing code on various different S3 configurations would probably be your best bet to start with. On hardware, though, Xeon processors incorporate extra instructions appropriate to high-performance computing, whereas desktop/consumer-class CPUs include hardware acceleration and chipset support for theft prevention, basic media acceleration, wireless projector connectivity, etc. In a server environment, workloads tend to need scalability more than processor power - it's generally significantly faster to run 100 tasks in parallel on slow hardware than in sequence on fast hardware. A desktop context is usually the opposite though - running a small number of applications, each of which generally needs to run as quickly as possible so the system feels snappy. It also generally boils down to manufacturing. You don't get 24-core desktop-oriented i7 chips yet, because you can't yet pack 24 4GHz+ cores into a CPU die. Oh, and that datacenter you mentioned, with dual 10GbE - is that $30k a month, or more?

graycat 11y ago
> Oh, and that datacenter you mentioned, with dual 10GbE - is that $30k a month, or more? I don't recall the details of their pricing. The colocation facility is at the old IBM Wappingers Falls, Myers Corners lab complex. There were several nice buildings, one with a lot of well done raised floor. IIRC, they were getting revenue from having a fiber roughly down the Hudson River for the 70 miles or so to Wall Street. I don't plan to go for a $4000+ Xeon processor soon, but I wanted to understand what the value is and see what I was missing. Or can get an eight core AMD processor at 4.0 GHz for about $160. And, say, an 18 core Xeon at 3.3 GHz for $4000. So, the AMD has price per core per GH of 160 / ( 8 * 4.0 ) = 5 and the Xeon has 4000 / ( 18 * 3.2 ) = 69.44 for a ratio of 69.44 / 5 = 13.89 That's 1389%, and that's a biggie. For my startup, the software for at least early production appears to be ready. Now I'm loading some initial data and generally, e.g., for my server farm, doing some planning. Then I will do some testing, alpha, beta and possibly some revisions. Then I will go live and go for publicity, users, ads, revenue, earnings, and growth. The basic core applied math and software really should work as I intend. The main question is, will lots of users like the site? If so, then the project should become a big thing. From my software and server farm architecture and software timings, one 8 core AMD processor at 4.0 GHz should be able to support on average, 24 x 7, one new user a second. Then I should be able to get, ballpark, $207,360 dollars a month in revenue. In that case, growth should be fast, and I should consider Xeon processors if they have some significant advantages. E.g., some builders offer two Xeon processors, 18 cores each, on a motherboard in a full tower case with lots of room for hard disks. So, that would be 36 cores, and four of the 8 core AMD processors would be 32 cores in four cases. Then, sure, for more, get some standard racks and put in servers designed for racks. I have a lot of flexibility in how many processors and motherboards I use because the software and server farm architecture is highly scalable just via simple sharding. The architecture has just five boxes, Web server, Web session state server, SQL Server, and two specialized servers full of applied math. Each of these five can run in their own server(s), or, for a good start, all five can run in one server.