4 ms·

You have since edited your comment, so... > That's not even that hard to figure out! A 48TB Memcached / Redis, which is far more useful than a 8TB Memcached /

You have since edited your comment, so... > That's not even that hard to figure out! A 48TB Memcached / Redis, which is far more useful than a 8TB Memcached / Redis box. No... dozens of terabytes of Optane would be just as good and much much cheaper. The person designing the system would have to prove that a few nanoseconds of latency difference makes any material difference to the company's profits in order to justify the more expensive machine. Otherwise, it's a huge waste of company money, hurting the business. Also keep in mind that Redis is only going to be using a single core of that machine. A total waste of huge amounts of CPU cores just to have a lot of RAM, when there are equally good solutions that cost much less. You surely must see why I'm skeptical. > That's the point: spend more money on hardware and then don't spend much engineering time thinking about optimization. It's bad business practice to buy the most expensive thing possible instead of engineering a solution that is the right price. If the more expensive solution saves money in the long term, sure... but your example doesn't show this.

dragontamer 6y ago
> No... dozens of terabytes of Optane would be just as good and much much cheaper. PCIe Optane is orders of magnitudes slower than DDR4. In both bandwidth and latency. The only Optane that keeps up to DDR4 (kinda-sorta) is the Optane DIMMs which are exclusive to Xeon Golds / Platinum systems.
- coder543 6y ago
  Not really... especially if this is an in-memory database attached to a network, as implied. It’s only slower if someone can observe the difference, which I don’t think they would be able to in this design. I’m a strong proponent of using fewer, larger machines and services, instead of incurring the overhead involved in spreading things out into a million microservices on a million machines. But there is a balance to be achieved, and beyond a certain point... synthetic improvements in performance don’t show up in the real world. Queuing up a few database requests concurrently to make up for the overhead of literally hundreds of nanoseconds of latency is trivial, especially when Optane can service those requests concurrently, unlike a spinning hard drive. Applications running on other machines won’t be able to tell a difference. But, agree to disagree. There are probably applications where these mega machines are useful, but I don’t personally find this to be a compelling example. I readily admit that I could be wrong... but neither of us have numbers in front of us showing a compelling reason for a company to spend unbelievable amounts of money on a single machine. My experiences (limited compared to many, I’m sure) tell me this isn’t the winning scenario, though.
  - dragontamer 6y ago
    > I readily admit that I could be wrong... but neither of us have numbers in front of us showing a compelling reason for a company to spend unbelievable amounts of money on a single machine. $500k on a machine isn't a lot of money compared to engineers. Even if you buy 4 of them for test / staging / 2xProduction, its not a lot compared to the amount spent on programming.
    - coder543 6y ago
      One thing being expensive doesn’t make another thing not expensive. It’s possible for them to both be independently expensive, and I’m saying that unless you can prove that the performance difference makes any difference to company profits, it is literally a huge waste of company money to buy those expensive machines. A lot of applications will actually perform worse in NUMA environments, so you’re spending more money to get worse performance. Reality isn’t as simple as “throw unlimited money at Intel to save engineering time.” Intel wishes it was. Engineering effort will be expended either way. It is worth finding the right solution, rather than the most expensive solution. Especially since that most expensive solution is likely to come with major problems.

dragontamer 6y ago

> No... dozens of terabytes of Optane would be just as good and much much cheaper. PCIe Optane is orders of magnitudes slower than DDR4. In both bandwidth and latency. The only Optane that keeps up to DDR4 (kinda-sorta) is the Optane DIMMs which are exclusive to Xeon Golds / Platinum systems.

coder543 6y ago
Not really... especially if this is an in-memory database attached to a network, as implied. It’s only slower if someone can observe the difference, which I don’t think they would be able to in this design. I’m a strong proponent of using fewer, larger machines and services, instead of incurring the overhead involved in spreading things out into a million microservices on a million machines. But there is a balance to be achieved, and beyond a certain point... synthetic improvements in performance don’t show up in the real world. Queuing up a few database requests concurrently to make up for the overhead of literally hundreds of nanoseconds of latency is trivial, especially when Optane can service those requests concurrently, unlike a spinning hard drive. Applications running on other machines won’t be able to tell a difference. But, agree to disagree. There are probably applications where these mega machines are useful, but I don’t personally find this to be a compelling example. I readily admit that I could be wrong... but neither of us have numbers in front of us showing a compelling reason for a company to spend unbelievable amounts of money on a single machine. My experiences (limited compared to many, I’m sure) tell me this isn’t the winning scenario, though.
- dragontamer 6y ago
  > I readily admit that I could be wrong... but neither of us have numbers in front of us showing a compelling reason for a company to spend unbelievable amounts of money on a single machine. $500k on a machine isn't a lot of money compared to engineers. Even if you buy 4 of them for test / staging / 2xProduction, its not a lot compared to the amount spent on programming.
  - coder543 6y ago
    One thing being expensive doesn’t make another thing not expensive. It’s possible for them to both be independently expensive, and I’m saying that unless you can prove that the performance difference makes any difference to company profits, it is literally a huge waste of company money to buy those expensive machines. A lot of applications will actually perform worse in NUMA environments, so you’re spending more money to get worse performance. Reality isn’t as simple as “throw unlimited money at Intel to save engineering time.” Intel wishes it was. Engineering effort will be expended either way. It is worth finding the right solution, rather than the most expensive solution. Especially since that most expensive solution is likely to come with major problems.