6 ms·

> The easiest solution to not having enough RAM is to throw money at the problem. People underestimate how much memory modern computers can actually support if

> The easiest solution to not having enough RAM is to throw money at the problem. People underestimate how much memory modern computers can actually support if you max them out. http://www.itu.dk/people/jovt/fitinram/ http://www.itu.dk/people/jovt/fitinram/

wongarsu 7y ago
And not just on super fancy computers, 128GB of RAM is like $500 and is supported by many low-to-mid-range desktop systems.
- kevin_thibedeau 7y ago
  Without ECC that's a risky way to compute.
  - ben-schaaf 7y ago
    IIRC ryzen in general can run with ecc, though motherboard support is a little shaky
  - wongarsu 7y ago
    Eyeballing the numbers [0][1] I would expect one bitflip between once every year and once every 100 years on 128GB of memory. I'm willing to take those odds. Of course with some planning you can get an AMD system with ECC support on the mainboard; ECC RAM is about the same price as consumer RAM. 0: https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf 1: https://www.cs.virginia.edu/~gurumurthi/papers/asplos15.pdf https://www.cs.virginia.edu/~gurumurthi/papers/asplos15.pdf
    - nostrademons 7y ago
      When Craig Silverstein left Google somebody asked him "What was the biggest mistake you made while at Google?" His response was "Not using ECC memory." The problem is that both the consequences and debugging time of a single random bitflip are potentially unbounded. That time that GMail lost 10% of their accounts and had to restore from tape was because of a single bitflip (due to a software bug, not a cosmic ray). Google Search lost months of engineering time in the early days from cosmic rays, back when they really could've used those engineers on other stuff. The odds may be low, but they do happen when you have multiple computers, and the consequences are high. Not really odds I'd want to take, when ECC RAM isn't that much more expensive than non-ECC RAM.
      - jandrese 7y ago
        Since Google was building their own servers it seems like they might have been able to sidestep the ECC tax too. Normally ECC costs so much more because it's the "serious user" solution so vendors feel free to mark it up.
      - hinkley 7y ago
        I kinda wonder if we’ll come out the other side of this DRY era and rediscover redundant systems. Turning really important decisions into a single bit in memory is overoptimization. Especially when all of the less important bits are consuming gigabytes.
      - gowld 7y ago
        How does ECC RAM protect against software bugs?
        nostrademons 7y ago
        To be clear, that sentence was in there to illustrate the effect that single-bit errors could cause. That incident happened after Google had already long since switched to ECC and obviously wouldn't have been prevented by it. However, we have a large number of other processes (testing, type systems, formal verification, code reviews, release processes, etc.) to protect against software bugs. There is no protection against cosmic rays. You don't want to be in a situation where all of the defect-mitigation work that the last 50 years of computer science has accomplished is rendered useless by a random freak occurrence. (The bug in question was actually in a migration script, and made it into production because people thought that migration scripts were one-off throwaways that didn't need the same amount of testing, code review, verification, and general carefulness that the production code does. Lesson learned. The postmortem for it actually had the lesson of "Treat your migration code as permanent, and apply all the same standards of maintainability and reliability of it that you do to production code.")
        bcaa7f3a8bbc 7y ago
        It's also why ZFS requires the use of ECC memory in the official documentation - ZFS spent great efforts building redundancy and error-checking capabilities as part of the filesystem, especially for guarding against silent data corruption, even at the expense of performance. But it would be useless and greatly decrease the benefits of these features if the memory can fail silently. Also, hitting by a beam of cosmic ray is not the only way that the bits in RAM can be flipped, dynamic RAM has inherent instabilities like row hammering, or can fail early due to manufacturing defects.
    - flukus 7y ago
      > Eyeballing the numbers [0][1] I would expect one bitflip between once every year and once every 100 years on 128GB of memory. I'm willing to take those odds. Throw in the likely hood of the flipped bit being consequential and the odds look even better. To potentially do major damage the flipped bit would have to be in an area of memory of something being executed and it has to flip after being read and before being executed, which probably ads at least 2 more orders of magnitude. Even for general bitrot of data has to be in memory and get flipped between reads and writes. Those odds are vanishingly small compared to programmer error.
  - awalton 7y ago
    The low to midrange Xeons with this kind of memory capacity have ECC. If you're not buying for your particular specifications, you're doing it wrong anyway... There are plenty of workloads tolerant to ECC errors (e.g. just about any kind of simulation).
TuringNYC 7y ago
I once had a job interview where they gave me a problem and expected some answer of "use a spark cluster, da da" and I said -- well given the problem specs and upper bounds you've given, I'd throw a $200 stick of RAM at it and be done. They wanted some answer involving using spark engineers you'd effectively pay for at $200/hr for many weeks. I didn't get the job. OK, but seriously speaking, if the upper bound is beyond 786gb RAM or whatever the current max is, I might want to use dask distributed. https://distributed.dask.org/en/latest/ https://distributed.dask.org/en/latest/ edit: wrote mb instead of gb
- nicoburns 7y ago
  > 786mb RAM Did you mean gb? In any case you can actually do at least 24TB these days: https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-18-tb-and-24-tb-instances/ https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-...
  - foota 7y ago
    That's ludicrous.
- ClumsyPilot 7y ago
  Bet their job description was like:"flexible", "open minded people" and "best tool for the job". But then they dont usually like when you demonstrate anything but the answer they want to a highly contrived problem.
  - mark-r 7y ago
    Yes, technical interviewing is hard.
- mumblemumble 7y ago
  It's slightly mystifying. The only company I've worked at that did "big data" _really_ well just plugged a few TB of RAM into some sharded databases and got on with life. Usually when I tell that story, I get a lot of objections about how that solution won't scale and they must not have really had big data from people who are, truth be told, used to working with data on a fraction of the scale that this company did. That said, it's not a turnkey solution. This company also was more meticulous about data engineering than others, and that certainly had its own cost.
usefulcat 7y ago
Sure, unless it's a laptop.
- eropple 7y ago
  I often wonder how much of the computational universe in “high tech” is indeed blinded by what we can stick in a laptop. That, and the idea of a polyphasic merge sort isn’t taught in ye olde boot camps, I guess.
  - mark-r 7y ago
    I love a good merge sort, it's just built for chunking. I've used it to sort data that just barely fit on disk.
- rumanator 7y ago
  > Sure, unless it's a laptop. That's a self-imposed constraint, not a problem constraint. Computational resources are dirt cheap nowadays. Everyone can get free time in global scale clusters. The only reason anykne is stuck with a laptop to run number/data crunching tasks is because they want to.
user5994461 7y ago
That's a bit less true with the rise of the cloud, replacing physical machines. Instances are charged in proportion to the amount of memory, making TB of RAM noticeably expensive.
6gvONxR4sf7o 7y ago
Wow it's less than $7/hour on AWS for 1TB of RAM on demand. $27/hour for 4TB. If you reserve instance (for 3 years) you can even go to 24TB.
- bocklund 7y ago
  Not that it's entirely equivalent (man hours of maintaining the local hardware/software vs man hours of AWS), but you could buy ~22.5TB of RAM outright for the price of using the 1TB AWS instance for $7/hour, assuming a current price of $2.75/GB of RAM. If you only need the 1 TB (5% cost), you could put difference ($58,000) towards hardware and an engineer at quarter time to maintain the hardware. Bonus: you get to keep the hardware.
  - pojzon 7y ago
    Unfortunately, big corporations think its "not scalable" solution and prefer to overpay 60k to outsource to AWS...
    - 6gvONxR4sf7o 7y ago
      For the "big data" stuff under discussion, you're probably spinning up an instance for some training then turning it off again. $7/hour for an hour or two a day, is about $4k/year. Maybe you retrain weekly and it's only $500/year. At that kind of price point, who cares? As the person making these algorithms, I'd hate to have to wait to order new machines for me to run something new, or to schedule time on the cluster or whatever, instead of just throwing an elastic amount of money at it.
      - pojzon 7y ago
        I can see the same argument being used for the stuff you could run locally up to 500k in infrastructure per month. People are so used to managed servers that they do not even see a possibility to use something else. Now everyone wants to be cloud agnostic (looking at conferences) so they want to eat the pie and have it too. While companies with mixed on premis/cloud setups pay the least based on all the consulting Ive done in this regard.
Animats 7y ago
As a nightmare in the opposite direction, I've been writing a new pathfinding system for Second Life, from the user side. Where you get a maximum of 64KB per program. KB. Not MB. And that's for code, stack, and data, using the Mono engine. You can have many programs, and they can pass messages around, so you can do jobs that won't fit in one program. It's like coding for a cluster of Arduinos. I've had to pack data into integers with shift operations. Come up with an efficient maze solving algorithm that only needs 2 bits per cell. Divide tasks into multi-step pipelines. Monitor memory usage and divide the data into smaller chunks that will fit. Plus it's soft real time and has to run properly under overload conditions when it can't get enough CPU time. It's kind of neat to see autonomous characters running around the virtual world at running speed. This was believed to be impossible, but I got it working out of sheer stubbornness. It was far too much work. (Second Life has a built-in pathfinding system. It's too buggy to use, and they refuse to fix it. This is a workaround.) (Why such tiny programs? Because they are not only persistent, for years if necessary, but are copied from one machine to another, state and all, as objects move around.)
- Zelizz 7y ago
  That's unreasonably cool. Do you think it consumes more resources than if they just gave you more than 64K to work with, or do you think you were pushed to optimize further because of the limitations?
  - Animats 7y ago
    It uses about 750KB total, spread over a dozen or so tiny programs. Probably 2x the memory and more CPU time because it's broken into so many pieces, both in time and code. And at least 2x-3x the development time. It pushed me to work on minimum-memory maze solving. It costs a lot to test a cell (this involves ray-casting in the simulated world) so something like A*, which examines most of the cells, is out. An algorithm in Wikipedia turned out not to work; some anon had snuck in a reference to an obscure paper of their own. Had to fix that. What I'm doing is "head for the goal, when you hit something, follow the wall, if it will get closer, head for the goal again". Wall following is both ways simultaneously, so you don't take too long on the long path when a short path is available. After getting a path, the path is tightened up to take out the jaggies. This is not optimal but is usually reasonably good. The rest of it is more or less routine, and a pain to break into sections. The programs have to communicate with JSON, over a weak IPC system with bad scheduling. I always liked the 3D "metaverse" concept. Second Life, which has about 30,000 to 50,000 users on line at any one time, comparable to GTA Online, is the biggest virtual world around. Everybody bigger is sharded, but all SL users are in one world. The technology needs a refresh, but every competitor who's tried to build a big virtual world has been unable to get many users. So, you're stuck with outdated tech if you want to get something used in a virtual world. I think they're still in 32 bit mode on the servers, even.

wongarsu 7y ago

And not just on super fancy computers, 128GB of RAM is like $500 and is supported by many low-to-mid-range desktop systems.

kevin_thibedeau 7y ago
Without ECC that's a risky way to compute.
- ben-schaaf 7y ago
  IIRC ryzen in general can run with ecc, though motherboard support is a little shaky
- wongarsu 7y ago
  Eyeballing the numbers [0][1] I would expect one bitflip between once every year and once every 100 years on 128GB of memory. I'm willing to take those odds. Of course with some planning you can get an AMD system with ECC support on the mainboard; ECC RAM is about the same price as consumer RAM. 0: https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf 1: https://www.cs.virginia.edu/~gurumurthi/papers/asplos15.pdf https://www.cs.virginia.edu/~gurumurthi/papers/asplos15.pdf
  - nostrademons 7y ago
    When Craig Silverstein left Google somebody asked him "What was the biggest mistake you made while at Google?" His response was "Not using ECC memory." The problem is that both the consequences and debugging time of a single random bitflip are potentially unbounded. That time that GMail lost 10% of their accounts and had to restore from tape was because of a single bitflip (due to a software bug, not a cosmic ray). Google Search lost months of engineering time in the early days from cosmic rays, back when they really could've used those engineers on other stuff. The odds may be low, but they do happen when you have multiple computers, and the consequences are high. Not really odds I'd want to take, when ECC RAM isn't that much more expensive than non-ECC RAM.
    - jandrese 7y ago
      Since Google was building their own servers it seems like they might have been able to sidestep the ECC tax too. Normally ECC costs so much more because it's the "serious user" solution so vendors feel free to mark it up.
    - hinkley 7y ago
      I kinda wonder if we’ll come out the other side of this DRY era and rediscover redundant systems. Turning really important decisions into a single bit in memory is overoptimization. Especially when all of the less important bits are consuming gigabytes.
    - gowld 7y ago
      How does ECC RAM protect against software bugs?
      - nostrademons 7y ago
        To be clear, that sentence was in there to illustrate the effect that single-bit errors could cause. That incident happened after Google had already long since switched to ECC and obviously wouldn't have been prevented by it. However, we have a large number of other processes (testing, type systems, formal verification, code reviews, release processes, etc.) to protect against software bugs. There is no protection against cosmic rays. You don't want to be in a situation where all of the defect-mitigation work that the last 50 years of computer science has accomplished is rendered useless by a random freak occurrence. (The bug in question was actually in a migration script, and made it into production because people thought that migration scripts were one-off throwaways that didn't need the same amount of testing, code review, verification, and general carefulness that the production code does. Lesson learned. The postmortem for it actually had the lesson of "Treat your migration code as permanent, and apply all the same standards of maintainability and reliability of it that you do to production code.")
        bcaa7f3a8bbc 7y ago
        It's also why ZFS requires the use of ECC memory in the official documentation - ZFS spent great efforts building redundancy and error-checking capabilities as part of the filesystem, especially for guarding against silent data corruption, even at the expense of performance. But it would be useless and greatly decrease the benefits of these features if the memory can fail silently. Also, hitting by a beam of cosmic ray is not the only way that the bits in RAM can be flipped, dynamic RAM has inherent instabilities like row hammering, or can fail early due to manufacturing defects.
  - flukus 7y ago
    > Eyeballing the numbers [0][1] I would expect one bitflip between once every year and once every 100 years on 128GB of memory. I'm willing to take those odds. Throw in the likely hood of the flipped bit being consequential and the odds look even better. To potentially do major damage the flipped bit would have to be in an area of memory of something being executed and it has to flip after being read and before being executed, which probably ads at least 2 more orders of magnitude. Even for general bitrot of data has to be in memory and get flipped between reads and writes. Those odds are vanishingly small compared to programmer error.
- awalton 7y ago
  The low to midrange Xeons with this kind of memory capacity have ECC. If you're not buying for your particular specifications, you're doing it wrong anyway... There are plenty of workloads tolerant to ECC errors (e.g. just about any kind of simulation).

TuringNYC 7y ago

I once had a job interview where they gave me a problem and expected some answer of "use a spark cluster, da da" and I said -- well given the problem specs and upper bounds you've given, I'd throw a $200 stick of RAM at it and be done. They wanted some answer involving using spark engineers you'd effectively pay for at $200/hr for many weeks. I didn't get the job. OK, but seriously speaking, if the upper bound is beyond 786gb RAM or whatever the current max is, I might want to use dask distributed. https://distributed.dask.org/en/latest/ https://distributed.dask.org/en/latest/ edit: wrote mb instead of gb

nicoburns 7y ago
> 786mb RAM Did you mean gb? In any case you can actually do at least 24TB these days: https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-18-tb-and-24-tb-instances/ https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-...
- foota 7y ago
  That's ludicrous.
ClumsyPilot 7y ago
Bet their job description was like:"flexible", "open minded people" and "best tool for the job". But then they dont usually like when you demonstrate anything but the answer they want to a highly contrived problem.
- mark-r 7y ago
  Yes, technical interviewing is hard.
mumblemumble 7y ago
It's slightly mystifying. The only company I've worked at that did "big data" _really_ well just plugged a few TB of RAM into some sharded databases and got on with life. Usually when I tell that story, I get a lot of objections about how that solution won't scale and they must not have really had big data from people who are, truth be told, used to working with data on a fraction of the scale that this company did. That said, it's not a turnkey solution. This company also was more meticulous about data engineering than others, and that certainly had its own cost.

usefulcat 7y ago

Sure, unless it's a laptop.

eropple 7y ago
I often wonder how much of the computational universe in “high tech” is indeed blinded by what we can stick in a laptop. That, and the idea of a polyphasic merge sort isn’t taught in ye olde boot camps, I guess.
- mark-r 7y ago
  I love a good merge sort, it's just built for chunking. I've used it to sort data that just barely fit on disk.
rumanator 7y ago
> Sure, unless it's a laptop. That's a self-imposed constraint, not a problem constraint. Computational resources are dirt cheap nowadays. Everyone can get free time in global scale clusters. The only reason anykne is stuck with a laptop to run number/data crunching tasks is because they want to.

user5994461 7y ago

That's a bit less true with the rise of the cloud, replacing physical machines. Instances are charged in proportion to the amount of memory, making TB of RAM noticeably expensive.

6gvONxR4sf7o 7y ago

Wow it's less than $7/hour on AWS for 1TB of RAM on demand. $27/hour for 4TB. If you reserve instance (for 3 years) you can even go to 24TB.

bocklund 7y ago
Not that it's entirely equivalent (man hours of maintaining the local hardware/software vs man hours of AWS), but you could buy ~22.5TB of RAM outright for the price of using the 1TB AWS instance for $7/hour, assuming a current price of $2.75/GB of RAM. If you only need the 1 TB (5% cost), you could put difference ($58,000) towards hardware and an engineer at quarter time to maintain the hardware. Bonus: you get to keep the hardware.
- pojzon 7y ago
  Unfortunately, big corporations think its "not scalable" solution and prefer to overpay 60k to outsource to AWS...
  - 6gvONxR4sf7o 7y ago
    For the "big data" stuff under discussion, you're probably spinning up an instance for some training then turning it off again. $7/hour for an hour or two a day, is about $4k/year. Maybe you retrain weekly and it's only $500/year. At that kind of price point, who cares? As the person making these algorithms, I'd hate to have to wait to order new machines for me to run something new, or to schedule time on the cluster or whatever, instead of just throwing an elastic amount of money at it.
    - pojzon 7y ago
      I can see the same argument being used for the stuff you could run locally up to 500k in infrastructure per month. People are so used to managed servers that they do not even see a possibility to use something else. Now everyone wants to be cloud agnostic (looking at conferences) so they want to eat the pie and have it too. While companies with mixed on premis/cloud setups pay the least based on all the consulting Ive done in this regard.

Animats 7y ago

As a nightmare in the opposite direction, I've been writing a new pathfinding system for Second Life, from the user side. Where you get a maximum of 64KB per program. KB. Not MB. And that's for code, stack, and data, using the Mono engine. You can have many programs, and they can pass messages around, so you can do jobs that won't fit in one program. It's like coding for a cluster of Arduinos. I've had to pack data into integers with shift operations. Come up with an efficient maze solving algorithm that only needs 2 bits per cell. Divide tasks into multi-step pipelines. Monitor memory usage and divide the data into smaller chunks that will fit. Plus it's soft real time and has to run properly under overload conditions when it can't get enough CPU time. It's kind of neat to see autonomous characters running around the virtual world at running speed. This was believed to be impossible, but I got it working out of sheer stubbornness. It was far too much work. (Second Life has a built-in pathfinding system. It's too buggy to use, and they refuse to fix it. This is a workaround.) (Why such tiny programs? Because they are not only persistent, for years if necessary, but are copied from one machine to another, state and all, as objects move around.)

Zelizz 7y ago
That's unreasonably cool. Do you think it consumes more resources than if they just gave you more than 64K to work with, or do you think you were pushed to optimize further because of the limitations?
- Animats 7y ago
  It uses about 750KB total, spread over a dozen or so tiny programs. Probably 2x the memory and more CPU time because it's broken into so many pieces, both in time and code. And at least 2x-3x the development time. It pushed me to work on minimum-memory maze solving. It costs a lot to test a cell (this involves ray-casting in the simulated world) so something like A*, which examines most of the cells, is out. An algorithm in Wikipedia turned out not to work; some anon had snuck in a reference to an obscure paper of their own. Had to fix that. What I'm doing is "head for the goal, when you hit something, follow the wall, if it will get closer, head for the goal again". Wall following is both ways simultaneously, so you don't take too long on the long path when a short path is available. After getting a path, the path is tightened up to take out the jaggies. This is not optimal but is usually reasonably good. The rest of it is more or less routine, and a pain to break into sections. The programs have to communicate with JSON, over a weak IPC system with bad scheduling. I always liked the 3D "metaverse" concept. Second Life, which has about 30,000 to 50,000 users on line at any one time, comparable to GTA Online, is the biggest virtual world around. Everybody bigger is sharded, but all SL users are in one world. The technology needs a refresh, but every competitor who's tried to build a big virtual world has been unable to get many users. So, you're stuck with outdated tech if you want to get something used in a virtual world. I think they're still in 32 bit mode on the servers, even.