5 ms·

The third sentence of your Medium article says; "Despite this, the Sia network is able to achieve 99.9999% uptime for files." How do you achieve this in a "not

The third sentence of your Medium article says; "Despite this, the Sia network is able to achieve 99.9999% uptime for files." How do you achieve this in a "not a highly reliable facility"?

prepend 7y ago
Redundancy, I assume. Lots of unreliable facilities and nodes can be very durable in aggregate.
- deleted 7y ago
  [deleted]
- jakewalker 7y ago
  Sounds kinda like the people who thought that bundling a bunch of bad mortgage debt together in slices could get it rated AAA.
  - __jal 7y ago
    Reminds me of a saying from the first dot.com crash. (Or at least I heard it then first.) Tying two bricks together doesn't make them float.
    - kybernetikos 7y ago
      It's a great saying, but aren't all large ships these days made out of components that don't float individually?
      - mcdevilkiller 7y ago
        No, they are mostly made of air, which floats (Partial sarcasm, as that is what makes them float)
  - feross 7y ago
    Your mind is going to be blown when you learn how TCP/IP works.
    - allendoerfer 7y ago
      Or computers for that matter. Deep down it is not really about 1s and 0s, more like thresholds in between 1 and 0. To me a big part of computer science is abstracting away unreliable details to make them seem reliable.
  - deleted 7y ago
    [deleted]
  - statictype 7y ago
    Except the people who put those sub prime mortgages in probably didn’t apply rigorous statistical analysis to their situation.
  - rpdillon 7y ago
    It does sound like that! The mistake there was to assume the failure of one mortgage was statistically independent of the failure of another, which is obviously incorrect for many failure scenarios. In this case, it it would be similar if all nodes were in, say, the same datacenter. That doesn't appear to be the case, but there may be other dimensions on which the network lacks the required diversity to support the reliability claims (disk vendor and age...? There must be others.)
  - justinmeiners 7y ago
    Not really. Just probability. If you have fully redundant services then ALL of them have to go down to have an outage. Suppose you 5 copies with 75% uptime each. The probability that all of them are down is 0.25^5 ~ 0.0009 Now of course that assumes they are uncorrelated, but since Sia nodes are distributed across the internet, that's likely as opposed to multiple servers at a few data centers like AWS. Turns out mortgage bonds tend to be correlated.
    - iampims 7y ago
      They all run the same software stack though, so despite being deployed on diverse hardware, so they can’t claim to only have independent failure modes.
      - justinmeiners 7y ago
        True. It would be interesting to analyze what others aspects are correlated.
  - zchrykng 7y ago
    I mean, the concept actually works, but you have to understand what is actually going into the bundled product. There is no reason you couldn't bundle 100 million in mortgages that had a 10% default risk and sell 10 million of that bundle as AAA. It is when you just start bundling everything and then saying the entire thing is AAA that the trouble starts.
- Aunche 7y ago
  Redundancy is kind of meaningless unless if you have sufficient diversity. If all your replicas are located on the same rack, you're screwed if there is a minor disaster like a sprinkler going off.
  - im3w1l 7y ago
    You will be able to boast of 99.9999% uptime up until that sprinkler goes of. And with luck, it could be decades until it does.
  - teknopurge 7y ago
    This is why the post is a red-herring. You cannot use storage if you cannot get to it. Talking about the price of disk space/mth is absolutely useless without talking about the bandwidth required to use it, which means, if you want to go big-boys-math, 95% bandwidth and peering agreement pricing. That $1.50 with all the network chatter between hosts for replication/repair/etc., not to mention actually _downloading_ of the data from the Sia network will take that $1.50/TB/mth to $70/TB/mth. Talking about the disk-space pricing is, IMO, disingenuous. Talk about the math for an all-in use-case of the service. I appreciate what the OP is going for, however, hand-waving around uncomfortable critical items like bandwidth cost for the Sia network is not a good look.
    - Dylan16807 7y ago
      > not to mention actually _downloading_ of the data from the Sia network will take that $1.50/TB/mth to $70/TB/mth You really need to back that math up. Here's my math: 10mbps is over 3TB per month at max utilization, so let's say it's good enough for either 2TB throughput with semi-even use, or 1TB throughput if the use is really focused on part of the day. When buying transit in bulk in the US, Google tells me that 10mbps was very roughly $8.50 a month in 2017, $4.50 a month in 2018, and $2.50 a month in 2019. Are you expecting the data to be rebuilt every single day or something?
    - deleted 7y ago
      [deleted]

The third sentence of your Medium article says; "Despite this, the Sia network is able to achieve 99.9999% uptime for files." How do you achieve this in a "not a highly reliable facility"?

prepend 7y ago

Redundancy, I assume. Lots of unreliable facilities and nodes can be very durable in aggregate.

deleted 7y ago
[deleted]
jakewalker 7y ago
Sounds kinda like the people who thought that bundling a bunch of bad mortgage debt together in slices could get it rated AAA.
- __jal 7y ago
  Reminds me of a saying from the first dot.com crash. (Or at least I heard it then first.) Tying two bricks together doesn't make them float.
  - kybernetikos 7y ago
    It's a great saying, but aren't all large ships these days made out of components that don't float individually?
    - mcdevilkiller 7y ago
      No, they are mostly made of air, which floats (Partial sarcasm, as that is what makes them float)
- feross 7y ago
  Your mind is going to be blown when you learn how TCP/IP works.
  - allendoerfer 7y ago
    Or computers for that matter. Deep down it is not really about 1s and 0s, more like thresholds in between 1 and 0. To me a big part of computer science is abstracting away unreliable details to make them seem reliable.
- deleted 7y ago
  [deleted]
- statictype 7y ago
  Except the people who put those sub prime mortgages in probably didn’t apply rigorous statistical analysis to their situation.
- rpdillon 7y ago
  It does sound like that! The mistake there was to assume the failure of one mortgage was statistically independent of the failure of another, which is obviously incorrect for many failure scenarios. In this case, it it would be similar if all nodes were in, say, the same datacenter. That doesn't appear to be the case, but there may be other dimensions on which the network lacks the required diversity to support the reliability claims (disk vendor and age...? There must be others.)
- justinmeiners 7y ago
  Not really. Just probability. If you have fully redundant services then ALL of them have to go down to have an outage. Suppose you 5 copies with 75% uptime each. The probability that all of them are down is 0.25^5 ~ 0.0009 Now of course that assumes they are uncorrelated, but since Sia nodes are distributed across the internet, that's likely as opposed to multiple servers at a few data centers like AWS. Turns out mortgage bonds tend to be correlated.
  - iampims 7y ago
    They all run the same software stack though, so despite being deployed on diverse hardware, so they can’t claim to only have independent failure modes.
    - justinmeiners 7y ago
      True. It would be interesting to analyze what others aspects are correlated.
- zchrykng 7y ago
  I mean, the concept actually works, but you have to understand what is actually going into the bundled product. There is no reason you couldn't bundle 100 million in mortgages that had a 10% default risk and sell 10 million of that bundle as AAA. It is when you just start bundling everything and then saying the entire thing is AAA that the trouble starts.
Aunche 7y ago
Redundancy is kind of meaningless unless if you have sufficient diversity. If all your replicas are located on the same rack, you're screwed if there is a minor disaster like a sprinkler going off.
- im3w1l 7y ago
  You will be able to boast of 99.9999% uptime up until that sprinkler goes of. And with luck, it could be decades until it does.
- teknopurge 7y ago
  This is why the post is a red-herring. You cannot use storage if you cannot get to it. Talking about the price of disk space/mth is absolutely useless without talking about the bandwidth required to use it, which means, if you want to go big-boys-math, 95% bandwidth and peering agreement pricing. That $1.50 with all the network chatter between hosts for replication/repair/etc., not to mention actually _downloading_ of the data from the Sia network will take that $1.50/TB/mth to $70/TB/mth. Talking about the disk-space pricing is, IMO, disingenuous. Talk about the math for an all-in use-case of the service. I appreciate what the OP is going for, however, hand-waving around uncomfortable critical items like bandwidth cost for the Sia network is not a good look.
  - Dylan16807 7y ago
    > not to mention actually _downloading_ of the data from the Sia network will take that $1.50/TB/mth to $70/TB/mth You really need to back that math up. Here's my math: 10mbps is over 3TB per month at max utilization, so let's say it's good enough for either 2TB throughput with semi-even use, or 1TB throughput if the use is really focused on part of the day. When buying transit in bulk in the US, Google tells me that 10mbps was very roughly $8.50 a month in 2017, $4.50 a month in 2018, and $2.50 a month in 2019. Are you expecting the data to be rebuilt every single day or something?
  - deleted 7y ago
    [deleted]