8 ms·

Calculating Pi: My Attempt at Breaking the Pi World Record

138 points by olvy0 7y ago

mNovak 7y ago
Curious now to know how much would this cost to compute in AWS?
- dannyw 7y ago
  Probably 5x as much. Maybe 2-3x as much using spot instances. With AWS, you pay (dearly) for the ability to scale at a moments notice. You do not save money for predictable workloads. While not applicable to this case, bandwidth costs are absurdly marked up expensive as well; like by a factor of 10x.
- paranoidrobot 7y ago
  I'm trying to do some estimates using the AWS Pricing calculator. You can't go with a single instance and a ton of EBS storage, because it caps out at 16TB of disk, and 19Gbit[0] of EBS bandwidth, even on an instance with 100Gbit networking. So, depending on how you can allocate storage, you're probably going to need some kind of clustered filesystem like GlusterFS It's also not clear how well the application can spread it's writes - if it's all focussed on writing one file at a time, we need the most throughput to a single node at a time. Storage: Option 1: "GlusterFS: Hope it spreads writes" 20x c5n.9xlarge (36x vcpu/96GB RAM/50Gbit NIC / 9.5Gbit EBS) + 16TB st1 HDD storage each = $43k Option 2: "GlusterFS: more EBS IO" 20x c5n.9xlarge (72x vcpu/192GB RAM/100Gbit NIC / 19Gbit EBS) + 16TB gp2 SSD storage each = $89k Option 3: "GlusterFS: hey local storage is faster/cheaper" 6x i3en.24xlarge (96x vcpu/768GB RAM/100Gbit NIC/ no EBS) = $47k I was wondering about insane ideas like using mdadm in RAID0 over NFS Mounts presented by (say) 281 t3.2xlarge instances each with 1x 1TB EBS volume. That comes out at around 62k for the storage instances. Compute: I don't know how important CPU vs Disk IO bandwidth is. The instance the author is using has 4x 15 cores (60 cores total). The most I can get with standard EC2 instances is 64 cores, but that has 25Gbit network, and the next down from that is 48 cores. 1x i3en.24xlarge (96x vcpu/768GB RAM/100Gbit NIC/ no EBS) = $7.9k There are bare metal instances with more cores/memory and up to 100Gbit networking[1], but I can't find any pricing on them. All up, I think $51-55k/month using standard instances would probaably do the job. [0] There are the bare metal instances mentioned in the link below that get up to 28Gbit EBS per instance, but again no details on pricing. [1] https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-18-tb-and-24-tb-instances/ https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-...
  - jsjohnst 7y ago
    > You can't go with a single instance and a ton of EBS storage, because it caps out at 16TB of disk I just used the calculator to price out a single instance without issue. Just type in nineteen 16TB EBS volumes (you’d create an LVM volume group for them if launched). I used to have EC2 instances (albeit not by choice, I inherited the bad architecture) with 42TB total of EBS volumes using LVM without issue.
    - paranoidrobot 7y ago
      Ah, my mistake. I didn't realise they'd upped the maximum volume size from 1TB to 16TB, so thought the calculator was telling me it was capped at 16x EBS volumes per instance. The new calculator isn't helping things here[1] telling me that I can only assign 16TB to an instance. So, given that - then the issue becomes is 25Gbit NIC / 19Gbit EBS bandwidth enough IO to at least equal the needs of the task. On paper the total bandwidth of the author's disk controller was 24Gbit, but that will depend on how the output is spread and whether the EBS limit includes any overheads that aren't present in DAS. Interestingly, if the requirements are mostly sequential, you can get better performance/$ going with throughput-optimised HDDs rather than gp2 SSD. Applying striping in either case will ensure you saturate the per-instance EBS bandwidth limits. So, single instance calculations: r5.24xlarge (96vCPU/768GB RAM/25Gbit NIC/19Gbit EBS) = $4.4k Storage: 18x 16TB gp2 SSD (250MB/sec / 16K IOPS) = $31k 18x 16TB st1 HDD (500MB/sec / 500 IOPS) = $14.5k [1] https://calculator.aws/ https://calculator.aws/
      - jsjohnst 7y ago
        Those are per month costs, right? Considering it took multiple months to calculate the 50T digits of Pi, huge difference in cost.
- thesandlord 7y ago
  Using the setup for the 31.4 trillion digits that Emma calculated with GCP (previous world record): https://cloud.google.com/blog/products/compute/calculating-31-4-trillion-digits-of-archimedes-constant-on-google-cloud https://cloud.google.com/blog/products/compute/calculating-3... GCP Pricing Page: https://cloud.google.com/products/calculator#id=2eca3cef-7460-4422-8dff-b267dae5a768 https://cloud.google.com/products/calculator#id=2eca3cef-746... So ~$200 - $250k Could probably save a good bit with committed use discounts. Spot / Preemptible instances would not work, in fact before Emma did this calculation a lot of people thought this kind of thing wasn't possible on public cloud because of perceived instabilities in a multi-tenant system.
  - mNovak 7y ago
    Does AWS still give $100k credit to YC startups? I think a couple should team up..
- tyingq 7y ago
  OVH would be relatively cheap. They have 72TB storage servers for $485.99/month.
thinkloop 7y ago
> I also saw that it cost them around $200,000, which is very expensive. I’m aiming to stay below 5% of that overall amount. He may have stayed below $10K in hardware, but there is no way that includes the electricity needed to run the machines 24/7 for half a year.
- dkdk8283 7y ago
  Assuming 3kw and a PUE of 1.5 for cooling comes out to approx 20,000 kwh of power. Assuming a high-ish rate of $.13/kwh it comes out to around 2.5k for 6m. Not too bad.
saagarjha 7y ago
Have the results been verified yet?
- aaronbwebber 7y ago
  It appears they have, the y-cruncher program he used does verification. Alexander Yee, who wrote the y-cruncher program (which has been used for the previous 4 world record calculations of digits of Pi), has accepted it and posted it on their site, along with screenshots of the output of the program. http://www.numberworld.org/y-cruncher/ http://www.numberworld.org/y-cruncher/
  - hamiltont 7y ago
    One cool note per my reading here - prior record holder Emma Haruka Iwao was working for GCP as a developer advocate, and GCP makes her 31.4 trillion digits available via cloud images. Neat :-) For those of us with less insane needs, they also exposed an API to grab the specific digits of interest - https://pi.delivery/ https://pi.delivery/
- deleted 7y ago
  [deleted]
- not2b 7y ago
  Output in hex is easier to check, because of https://en.wikipedia.org/wiki/Bailey%E2%80%93Borwein%E2%80%93Plouffe_formula https://en.wikipedia.org/wiki/Bailey%E2%80%93Borwein%E2%80%9...
gesman 7y ago
>> As luck would have it, a transformer next to my house blew and the power went out yet again. This means I had to restart from the last checkpoint yet again, losing another 2 weeks worth of work. Ouch!!!
- nixpulvis 7y ago
  Man, it fucking sucks. I remember a time back freshman year of collage... It was an extra credit assignment. Who could factor the largest number (or something like this) in the least time before the due date. My friend and I devised an algorithm, and launched it on my computer. I honestly don't remember anything other than the fact the god damn power transformer downtown blew and fucked up our tests because of a large scale power outage. I assume we'd have not even come in top #5 for that assignment (who knows), but it's just so frustrating. Basically, OSs should have better fail recovery mechanics than are the default.
  - unlinked_dll 7y ago
    Same thing happened to me in college. I actually saw the transformer blow, it's kinda freaky (and that ozone smell is horrid, you only get to smell it when you weld or really screw the pooch). Except I wasn't working on anything academically meaningful, more spiritual. I was the only one that could fit a keg in my trunk. Thankfully it was a day party in the spring.
    - psaux 7y ago
      Or, the ozone from a Tesla coil. :) In eighth grade I built one, and I actually still like the smell to this day.
      - bArray 7y ago
        > I actually still like the smell to this day. Air-conditioning units with ionizers generate ozone, it's why they smell sweet. P.S. Ozone is meant to be pretty bad for you.
  - 7y ago
deleted 7y ago
[deleted]
droithomme 7y ago
tldr: He's some guy who believes he broke the world record and maybe he did.
- saagarjha 7y ago
  He almost certainly broke the record, and his word choice was intentionally modest.
saltyfamiliar 7y ago
What an irresponsible use of energy. At least mine Bitcoin...
- stjo 7y ago
  Of all the “irresponsible” uses of energy, this is the one I endorse the most.
  - goldenkey 7y ago
    It's not irresponsible if the dude needed heat in his home. Far and few people realize that all electronics are heaters that happen to do computation in the process of conversion of electricity to heat. Sure, it's not convective if there aren't fans to disperse the heat.. but it's a beefed up computer so I am sure he had good PC fans. In any case, heat that is created without convection fans to spread it quickly, will still diffuse into the home albeit at a slower pace. It has to go somewhere..
    - saagarjha 7y ago
      > Far and few people realize that all electronics are heaters that happen to do computation in the process of conversion of electricity to heat. Surely anyone who's used a computer with a fan in it knows this…
      - goldenkey 7y ago
        That's easy to say but pretty much everyone I've talked to, albeit the few oddities, thought that dedicated space heaters were way different than X-Box getting hot, or computer getting hot. Think of the cognitive dissonance to switch to all LED bulbs in a cold climate, because you're told it's more efficient, meanwhile using electric space heaters at the same time... It happens more often than you think, given the politics. Of course, there are some heating methods that are cheaper than electricity - like natural gas. So one would have to factor that in, along with power plant emissions vs home emissions. It's a complex entrenchment. It'd be interesting to produce a video of street interviews just to see what the average person thinks about electronics getting hot. It would make for a good case study of the intersection between common sense/basic physics knowledge/and energy/eco politics.
bn7t 7y ago
Is there a link somewhere where he published the result?
- ficklepickle 7y ago
  I found it, but clicking it caused my computer to run out of available memory. Edit: on a more serious note, a site[0] that tracks these records says: > Downloading of digits is no longer available due to the massive bandwidth requirements. Your best bet is to directly contact one of the record holders and see if they still have a copy of the digits. [0] http://www.numberworld.org/digits/Pi http://www.numberworld.org/digits/Pi
  - LeifCarrotson 7y ago
    The compressed representation requires 44 TB of disk space. Assuming the author has a typical home internet connection with about 5 Mbps upload rate, the transfer would take 2 years longer than it took to actually run the calculation in the first place!
    - philshem 7y ago
      Here’s more info about the race between calculating and downloading digits of pi https://opendata.stackexchange.com/a/4024/1511 https://opendata.stackexchange.com/a/4024/1511
mikorym 7y ago
For those that are interested in reciting digits of Pi (I would guess you can either memorise the digits or calculate as you go) the current record is held by a South African: https://www.youtube.com/watch?v=_mGjJMVKWcU https://www.youtube.com/watch?v=_mGjJMVKWcU
- las_balas_tres 7y ago
  He only won the record for the fastest recall of pi. 1500 digits in 4 minutes. Daniel Tammet recited 22514 in 5 hours. https://en.wikipedia.org/wiki/Daniel_Tammet https://en.wikipedia.org/wiki/Daniel_Tammet
  - begemotz 7y ago
    the Guinness world record is over 70k digits set by Rajveer Meena: https://www.guinnessworldrecords.com/world-records/most-pi-places-memorised https://www.guinnessworldrecords.com/world-records/most-pi-p...
    - imglorp 7y ago
      Such an awe inspiring achievement, to store that much structured data accurately. What else is the brain capable of doing?
      - Ma8ee 7y ago
        Structured?
        imglorp 7y ago
        Lossless, ordered set of symbols with no errors. It's much different than remembering a story or a travel route or an image.
        Ma8ee 7y ago
        Yes, but structured is in general in opposition to random, which the digits of pi seem to be. No one knows of any structure in the digits of pi.
matsemann 7y ago
But > Did you win the Putnam? [0] [0]: https://news.ycombinator.com/item?id=35076 https://news.ycombinator.com/item?id=35076
simias 7y ago
I appreciate the author sharing his experience but I must admit that it was less exciting that I expected it to be. He basically used off-the-shelf hardware, ran an off-the-shelf program and then waited for a while. There's no obvious innovation, anybody who cares enough to break that record and has ~$20k lying around can do it. If anything it got me a lot more curious about this y-cruncher program and all the fun optimizations it must implement.
- Someone 7y ago
  http://www.numberworld.org/y-cruncher/ http://www.numberworld.org/y-cruncher/ (Mostly closed source)
- darkwater 7y ago
  The previous record, which was basically just corporate promotion [1], did use the exact same program. For a sysadmin/hardware nerd this post was really interesting, although I would have probably have appreciated even more details (quick example, how many disks failed partially/totally during the process?). [1] https://cloud.google.com/blog/products/compute/calculating-31-4-trillion-digits-of-archimedes-constant-on-google-cloud https://cloud.google.com/blog/products/compute/calculating-3...
- blazespin 7y ago
  I know it isn't probably polite, but would really love to know the bill of materials. I have to say though, I think it's very cool if he came on budget. I'm very interested to see what he can do for STEM research. These cloud vendors are way too expensive compared to what he can do off ebay.
- krzepah 7y ago
  I understand your disappointment but I'd like to point out this is a really nice achievement on it's portfolio as a sys admin
  - prox 7y ago
    It also is a great side project; And having tested his system he donated it for STEM research. There’s lots to commend in this project.
kozak 7y ago
By "compressed Pi digits", does the author mean simply an efficient encoding of decimal digits in binary, or there is some way to compress Pi digits further? I thought they were incompressible like random.
- bArray 7y ago
  If they're stored as ASCII characters, there will be tonnes of compression that could be done.
- Someone 7y ago
  Typically, that means storing two digits in a byte (halves storage size compared to a text string), 9 in each 32 bits (gains another 11%), 19 in each 64 bits (gains another 5%), or something similar (at this scale, I would guess it uses at least the ‘19 digits in each 64 bits’) Idea is to not use “one digit per byte”, but to keep addressing individual digits cheap. ”I thought they were incompressible like random.” They’re easily compressed, if you accept taking this program and it’s configuration file as a compressed version. (And yes, the output of a pseudo-random number generator compresses extremely well, too)
  - lifthrasiir 7y ago
    > at this scale, I would guess it uses at least the ‘19 digits in each 64 bits’ Your guess is correct, y-cruncher uses that exact format [1]. [1] https://github.com/Mysticial/DigitViewer https://github.com/Mysticial/DigitViewer
  - maweki 7y ago
    That's the notion of kolmogorov complexity. The smallest program that can generate this output. Pi and I guess any other algebraic number, no matter how randomly distributed its digits are, are not that complex.
    - sp332 7y ago
      Pi is transcendental, not algebraic.
tromp 7y ago
Reminds me of my attempt to compute the number of Go positions, which similarly needed many TB of disk space and ended up generating 30 petabytes of disk IO. https://news.ycombinator.com/item?id=9167781 https://news.ycombinator.com/item?id=9167781 Number of legal 18x18 Go positions computed. One more to go https://news.ycombinator.com/item?id=10950875 https://news.ycombinator.com/item?id=10950875 Number of legal Go positions computed Not much point in breaking that record, as 19x19 is the largest (and standard size) Go board.
fizixer 7y ago
19-digit-per-64-bit compression storage requires about 19TiB [0]. 1-digit-per-byte requires about 45TiB [1]. Can anyone explain how it requires 38TiB for final output? [0] https://www.google.com/search?q=8+*+%285e13+%2F+19%29+%2F+1048576+%2F+1048576 https://www.google.com/search?q=8+*+%285e13+%2F+19%29+%2F+10... [1] https://www.google.com/search?q=5e13+%2F+1048576+%2F+1048576 https://www.google.com/search?q=5e13+%2F+1048576+%2F+1048576
- redcalx 7y ago
  If you cross byte boundaries, i.e. compactly pack the bits such that there are no wasted bits; then you have 4 bits per decimal digit, 50T digits becomes 200T bits == 25TB. Technically there is still some slack in there because each 4 bit block represents 16 values, and we only use 10 of those.
Pigo 7y ago
I got curious about the BOINC platform he mentioned. It reminds me of the SETI@home screensaver I used to run many years ago. I guess I'd just forgotten about it at some point, but I used to enjoy watching the data it was processing. It's pretty cool that there's another platform out there that lets you contribute to other programs.
- prox 7y ago
  The boinc wiki mentions seti@home is part of this platform. It probably grew out of this initiative.
layer8 7y ago
I wonder how many bit flips are to be expected on HDDs or in RAM for that amount of data.
kbob 7y ago
Did he calculate the value in decimal? The record would probably be a lot easier to achieve if he'd gone with, say, base 14. Everybody does decimal, even though there's no theoretical advantage, and no practical advantage since nobody actually needs trillions of digits of pi.

138 points by olvy0 7y ago

mNovak 7y ago

Curious now to know how much would this cost to compute in AWS?

dannyw 7y ago
Probably 5x as much. Maybe 2-3x as much using spot instances. With AWS, you pay (dearly) for the ability to scale at a moments notice. You do not save money for predictable workloads. While not applicable to this case, bandwidth costs are absurdly marked up expensive as well; like by a factor of 10x.
paranoidrobot 7y ago
I'm trying to do some estimates using the AWS Pricing calculator. You can't go with a single instance and a ton of EBS storage, because it caps out at 16TB of disk, and 19Gbit[0] of EBS bandwidth, even on an instance with 100Gbit networking. So, depending on how you can allocate storage, you're probably going to need some kind of clustered filesystem like GlusterFS It's also not clear how well the application can spread it's writes - if it's all focussed on writing one file at a time, we need the most throughput to a single node at a time. Storage: Option 1: "GlusterFS: Hope it spreads writes" 20x c5n.9xlarge (36x vcpu/96GB RAM/50Gbit NIC / 9.5Gbit EBS) + 16TB st1 HDD storage each = $43k Option 2: "GlusterFS: more EBS IO" 20x c5n.9xlarge (72x vcpu/192GB RAM/100Gbit NIC / 19Gbit EBS) + 16TB gp2 SSD storage each = $89k Option 3: "GlusterFS: hey local storage is faster/cheaper" 6x i3en.24xlarge (96x vcpu/768GB RAM/100Gbit NIC/ no EBS) = $47k I was wondering about insane ideas like using mdadm in RAID0 over NFS Mounts presented by (say) 281 t3.2xlarge instances each with 1x 1TB EBS volume. That comes out at around 62k for the storage instances. Compute: I don't know how important CPU vs Disk IO bandwidth is. The instance the author is using has 4x 15 cores (60 cores total). The most I can get with standard EC2 instances is 64 cores, but that has 25Gbit network, and the next down from that is 48 cores. 1x i3en.24xlarge (96x vcpu/768GB RAM/100Gbit NIC/ no EBS) = $7.9k There are bare metal instances with more cores/memory and up to 100Gbit networking[1], but I can't find any pricing on them. All up, I think $51-55k/month using standard instances would probaably do the job. [0] There are the bare metal instances mentioned in the link below that get up to 28Gbit EBS per instance, but again no details on pricing. [1] https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-18-tb-and-24-tb-instances/ https://aws.amazon.com/blogs/aws/ec2-high-memory-update-new-...
- jsjohnst 7y ago
  > You can't go with a single instance and a ton of EBS storage, because it caps out at 16TB of disk I just used the calculator to price out a single instance without issue. Just type in nineteen 16TB EBS volumes (you’d create an LVM volume group for them if launched). I used to have EC2 instances (albeit not by choice, I inherited the bad architecture) with 42TB total of EBS volumes using LVM without issue.
  - paranoidrobot 7y ago
    Ah, my mistake. I didn't realise they'd upped the maximum volume size from 1TB to 16TB, so thought the calculator was telling me it was capped at 16x EBS volumes per instance. The new calculator isn't helping things here[1] telling me that I can only assign 16TB to an instance. So, given that - then the issue becomes is 25Gbit NIC / 19Gbit EBS bandwidth enough IO to at least equal the needs of the task. On paper the total bandwidth of the author's disk controller was 24Gbit, but that will depend on how the output is spread and whether the EBS limit includes any overheads that aren't present in DAS. Interestingly, if the requirements are mostly sequential, you can get better performance/$ going with throughput-optimised HDDs rather than gp2 SSD. Applying striping in either case will ensure you saturate the per-instance EBS bandwidth limits. So, single instance calculations: r5.24xlarge (96vCPU/768GB RAM/25Gbit NIC/19Gbit EBS) = $4.4k Storage: 18x 16TB gp2 SSD (250MB/sec / 16K IOPS) = $31k 18x 16TB st1 HDD (500MB/sec / 500 IOPS) = $14.5k [1] https://calculator.aws/ https://calculator.aws/
    - jsjohnst 7y ago
      Those are per month costs, right? Considering it took multiple months to calculate the 50T digits of Pi, huge difference in cost.
thesandlord 7y ago
Using the setup for the 31.4 trillion digits that Emma calculated with GCP (previous world record): https://cloud.google.com/blog/products/compute/calculating-31-4-trillion-digits-of-archimedes-constant-on-google-cloud https://cloud.google.com/blog/products/compute/calculating-3... GCP Pricing Page: https://cloud.google.com/products/calculator#id=2eca3cef-7460-4422-8dff-b267dae5a768 https://cloud.google.com/products/calculator#id=2eca3cef-746... So ~$200 - $250k Could probably save a good bit with committed use discounts. Spot / Preemptible instances would not work, in fact before Emma did this calculation a lot of people thought this kind of thing wasn't possible on public cloud because of perceived instabilities in a multi-tenant system.
- mNovak 7y ago
  Does AWS still give $100k credit to YC startups? I think a couple should team up..
tyingq 7y ago
OVH would be relatively cheap. They have 72TB storage servers for $485.99/month.

thinkloop 7y ago

> I also saw that it cost them around $200,000, which is very expensive. I’m aiming to stay below 5% of that overall amount. He may have stayed below $10K in hardware, but there is no way that includes the electricity needed to run the machines 24/7 for half a year.

dkdk8283 7y ago
Assuming 3kw and a PUE of 1.5 for cooling comes out to approx 20,000 kwh of power. Assuming a high-ish rate of $.13/kwh it comes out to around 2.5k for 6m. Not too bad.

saagarjha 7y ago

Have the results been verified yet?

aaronbwebber 7y ago
It appears they have, the y-cruncher program he used does verification. Alexander Yee, who wrote the y-cruncher program (which has been used for the previous 4 world record calculations of digits of Pi), has accepted it and posted it on their site, along with screenshots of the output of the program. http://www.numberworld.org/y-cruncher/ http://www.numberworld.org/y-cruncher/
- hamiltont 7y ago
  One cool note per my reading here - prior record holder Emma Haruka Iwao was working for GCP as a developer advocate, and GCP makes her 31.4 trillion digits available via cloud images. Neat :-) For those of us with less insane needs, they also exposed an API to grab the specific digits of interest - https://pi.delivery/ https://pi.delivery/
deleted 7y ago
[deleted]
not2b 7y ago
Output in hex is easier to check, because of https://en.wikipedia.org/wiki/Bailey%E2%80%93Borwein%E2%80%93Plouffe_formula https://en.wikipedia.org/wiki/Bailey%E2%80%93Borwein%E2%80%9...

gesman 7y ago

>> As luck would have it, a transformer next to my house blew and the power went out yet again. This means I had to restart from the last checkpoint yet again, losing another 2 weeks worth of work. Ouch!!!

nixpulvis 7y ago
Man, it fucking sucks. I remember a time back freshman year of collage... It was an extra credit assignment. Who could factor the largest number (or something like this) in the least time before the due date. My friend and I devised an algorithm, and launched it on my computer. I honestly don't remember anything other than the fact the god damn power transformer downtown blew and fucked up our tests because of a large scale power outage. I assume we'd have not even come in top #5 for that assignment (who knows), but it's just so frustrating. Basically, OSs should have better fail recovery mechanics than are the default.
- unlinked_dll 7y ago
  Same thing happened to me in college. I actually saw the transformer blow, it's kinda freaky (and that ozone smell is horrid, you only get to smell it when you weld or really screw the pooch). Except I wasn't working on anything academically meaningful, more spiritual. I was the only one that could fit a keg in my trunk. Thankfully it was a day party in the spring.
  - psaux 7y ago
    Or, the ozone from a Tesla coil. :) In eighth grade I built one, and I actually still like the smell to this day.
    - bArray 7y ago
      > I actually still like the smell to this day. Air-conditioning units with ionizers generate ozone, it's why they smell sweet. P.S. Ozone is meant to be pretty bad for you.
- 7y ago

deleted 7y ago

[deleted]

droithomme 7y ago

tldr: He's some guy who believes he broke the world record and maybe he did.

saagarjha 7y ago
He almost certainly broke the record, and his word choice was intentionally modest.

saltyfamiliar 7y ago

What an irresponsible use of energy. At least mine Bitcoin...

stjo 7y ago
Of all the “irresponsible” uses of energy, this is the one I endorse the most.
- goldenkey 7y ago
  It's not irresponsible if the dude needed heat in his home. Far and few people realize that all electronics are heaters that happen to do computation in the process of conversion of electricity to heat. Sure, it's not convective if there aren't fans to disperse the heat.. but it's a beefed up computer so I am sure he had good PC fans. In any case, heat that is created without convection fans to spread it quickly, will still diffuse into the home albeit at a slower pace. It has to go somewhere..
  - saagarjha 7y ago
    > Far and few people realize that all electronics are heaters that happen to do computation in the process of conversion of electricity to heat. Surely anyone who's used a computer with a fan in it knows this…
    - goldenkey 7y ago
      That's easy to say but pretty much everyone I've talked to, albeit the few oddities, thought that dedicated space heaters were way different than X-Box getting hot, or computer getting hot. Think of the cognitive dissonance to switch to all LED bulbs in a cold climate, because you're told it's more efficient, meanwhile using electric space heaters at the same time... It happens more often than you think, given the politics. Of course, there are some heating methods that are cheaper than electricity - like natural gas. So one would have to factor that in, along with power plant emissions vs home emissions. It's a complex entrenchment. It'd be interesting to produce a video of street interviews just to see what the average person thinks about electronics getting hot. It would make for a good case study of the intersection between common sense/basic physics knowledge/and energy/eco politics.

bn7t 7y ago

Is there a link somewhere where he published the result?

ficklepickle 7y ago
I found it, but clicking it caused my computer to run out of available memory. Edit: on a more serious note, a site[0] that tracks these records says: > Downloading of digits is no longer available due to the massive bandwidth requirements. Your best bet is to directly contact one of the record holders and see if they still have a copy of the digits. [0] http://www.numberworld.org/digits/Pi http://www.numberworld.org/digits/Pi
- LeifCarrotson 7y ago
  The compressed representation requires 44 TB of disk space. Assuming the author has a typical home internet connection with about 5 Mbps upload rate, the transfer would take 2 years longer than it took to actually run the calculation in the first place!
  - philshem 7y ago
    Here’s more info about the race between calculating and downloading digits of pi https://opendata.stackexchange.com/a/4024/1511 https://opendata.stackexchange.com/a/4024/1511

mikorym 7y ago

For those that are interested in reciting digits of Pi (I would guess you can either memorise the digits or calculate as you go) the current record is held by a South African: https://www.youtube.com/watch?v=_mGjJMVKWcU https://www.youtube.com/watch?v=_mGjJMVKWcU

las_balas_tres 7y ago
He only won the record for the fastest recall of pi. 1500 digits in 4 minutes. Daniel Tammet recited 22514 in 5 hours. https://en.wikipedia.org/wiki/Daniel_Tammet https://en.wikipedia.org/wiki/Daniel_Tammet
- begemotz 7y ago
  the Guinness world record is over 70k digits set by Rajveer Meena: https://www.guinnessworldrecords.com/world-records/most-pi-places-memorised https://www.guinnessworldrecords.com/world-records/most-pi-p...
  - imglorp 7y ago
    Such an awe inspiring achievement, to store that much structured data accurately. What else is the brain capable of doing?
    - Ma8ee 7y ago
      Structured?
      - imglorp 7y ago
        Lossless, ordered set of symbols with no errors. It's much different than remembering a story or a travel route or an image.
        Ma8ee 7y ago
        Yes, but structured is in general in opposition to random, which the digits of pi seem to be. No one knows of any structure in the digits of pi.

matsemann 7y ago

But > Did you win the Putnam? [0] [0]: https://news.ycombinator.com/item?id=35076 https://news.ycombinator.com/item?id=35076

simias 7y ago

I appreciate the author sharing his experience but I must admit that it was less exciting that I expected it to be. He basically used off-the-shelf hardware, ran an off-the-shelf program and then waited for a while. There's no obvious innovation, anybody who cares enough to break that record and has ~$20k lying around can do it. If anything it got me a lot more curious about this y-cruncher program and all the fun optimizations it must implement.

Someone 7y ago
http://www.numberworld.org/y-cruncher/ http://www.numberworld.org/y-cruncher/ (Mostly closed source)
darkwater 7y ago
The previous record, which was basically just corporate promotion [1], did use the exact same program. For a sysadmin/hardware nerd this post was really interesting, although I would have probably have appreciated even more details (quick example, how many disks failed partially/totally during the process?). [1] https://cloud.google.com/blog/products/compute/calculating-31-4-trillion-digits-of-archimedes-constant-on-google-cloud https://cloud.google.com/blog/products/compute/calculating-3...
blazespin 7y ago
I know it isn't probably polite, but would really love to know the bill of materials. I have to say though, I think it's very cool if he came on budget. I'm very interested to see what he can do for STEM research. These cloud vendors are way too expensive compared to what he can do off ebay.
krzepah 7y ago
I understand your disappointment but I'd like to point out this is a really nice achievement on it's portfolio as a sys admin
- prox 7y ago
  It also is a great side project; And having tested his system he donated it for STEM research. There’s lots to commend in this project.

kozak 7y ago

By "compressed Pi digits", does the author mean simply an efficient encoding of decimal digits in binary, or there is some way to compress Pi digits further? I thought they were incompressible like random.

bArray 7y ago
If they're stored as ASCII characters, there will be tonnes of compression that could be done.
Someone 7y ago
Typically, that means storing two digits in a byte (halves storage size compared to a text string), 9 in each 32 bits (gains another 11%), 19 in each 64 bits (gains another 5%), or something similar (at this scale, I would guess it uses at least the ‘19 digits in each 64 bits’) Idea is to not use “one digit per byte”, but to keep addressing individual digits cheap. ”I thought they were incompressible like random.” They’re easily compressed, if you accept taking this program and it’s configuration file as a compressed version. (And yes, the output of a pseudo-random number generator compresses extremely well, too)
- lifthrasiir 7y ago
  > at this scale, I would guess it uses at least the ‘19 digits in each 64 bits’ Your guess is correct, y-cruncher uses that exact format [1]. [1] https://github.com/Mysticial/DigitViewer https://github.com/Mysticial/DigitViewer
- maweki 7y ago
  That's the notion of kolmogorov complexity. The smallest program that can generate this output. Pi and I guess any other algebraic number, no matter how randomly distributed its digits are, are not that complex.
  - sp332 7y ago
    Pi is transcendental, not algebraic.

tromp 7y ago

Reminds me of my attempt to compute the number of Go positions, which similarly needed many TB of disk space and ended up generating 30 petabytes of disk IO. https://news.ycombinator.com/item?id=9167781 https://news.ycombinator.com/item?id=9167781 Number of legal 18x18 Go positions computed. One more to go https://news.ycombinator.com/item?id=10950875 https://news.ycombinator.com/item?id=10950875 Number of legal Go positions computed Not much point in breaking that record, as 19x19 is the largest (and standard size) Go board.

fizixer 7y ago

19-digit-per-64-bit compression storage requires about 19TiB [0]. 1-digit-per-byte requires about 45TiB [1]. Can anyone explain how it requires 38TiB for final output? [0] https://www.google.com/search?q=8+*+%285e13+%2F+19%29+%2F+1048576+%2F+1048576 https://www.google.com/search?q=8+*+%285e13+%2F+19%29+%2F+10... [1] https://www.google.com/search?q=5e13+%2F+1048576+%2F+1048576 https://www.google.com/search?q=5e13+%2F+1048576+%2F+1048576

redcalx 7y ago
If you cross byte boundaries, i.e. compactly pack the bits such that there are no wasted bits; then you have 4 bits per decimal digit, 50T digits becomes 200T bits == 25TB. Technically there is still some slack in there because each 4 bit block represents 16 values, and we only use 10 of those.

Pigo 7y ago

I got curious about the BOINC platform he mentioned. It reminds me of the SETI@home screensaver I used to run many years ago. I guess I'd just forgotten about it at some point, but I used to enjoy watching the data it was processing. It's pretty cool that there's another platform out there that lets you contribute to other programs.

prox 7y ago
The boinc wiki mentions seti@home is part of this platform. It probably grew out of this initiative.

layer8 7y ago

I wonder how many bit flips are to be expected on HDDs or in RAM for that amount of data.

kbob 7y ago

Did he calculate the value in decimal? The record would probably be a lot easier to achieve if he'd gone with, say, base 14. Everybody does decimal, even though there's no theoretical advantage, and no practical advantage since nobody actually needs trillions of digits of pi.