14 ms·

Sia – Decentralized cloud storage network

276 points by Geee 7y ago

yasp 7y ago
How do services like Sia handle illegal content?
- jameslevy 7y ago
  I believe that censorship resistance is a key value proposition for decentralized file storage. So its agnostic handling of illegal content is a feature, not a bug.
  - yazan94 7y ago
    In the spirit of blockchain and decentralization, I think you're right. But practically, explaining that to the authorities will probably not be a pleasant experience if they manage to find cp or other illegal data on your hard drive.
    - Ajedi32 7y ago
      The data is encrypted by the client, so that's theoretically impossible. Neither you nor the authorities have the ability to determine what file fragments (whole flies aren't stored; just fragments) are on your PC.
      - modeless 7y ago
        It's not theoretically impossible for law enforcement to discover the encryption keys to some illegal files on someone's computer. Then they could go around arresting people who host the chunks. Of course those people wouldn't have known about those files in particular, but they were knowingly running software that enables hosting illegal files. A judge would have to decide if that's OK.
        Ajedi32 7y ago
        Doesn't AWS also have the same problem? Someone could upload encrypted illegal content, then later law enforcement could discover the key. Why hasn't AWS gotten in trouble for that yet?
        modeless 7y ago
        No large cloud provider allows anonymous usage. They take down content on request and assist law enforcement in catching the non-anonymous people doing illegal things on their platforms.
        Ajedi32 7y ago
        Okay, bad example. AWS is a paid service so obviously they have more information about their users than most. What about Dropbox or OneDrive? Both of those services allow files to be stored and shared with no requirement to identify yourself when you're creating an account, beyond providing an email address.
        txcwpalpha 7y ago
        Both Dropbox and OneDrive scan your content to determine if it matches known illegal content (be it illegal pornography, DRM content, etc) and will remove it and remove you from their service. See https://www.microsoft.com/en-us/photodna https://www.microsoft.com/en-us/photodna If someone pre-encrypts data and then uploads that to Dropbox, it gets more complicated and AFAIK involves more of monitoring the IP addresses that things are coming from and/or being shared to.
        lmpostor 7y ago
        Same argument for Bitcoin, cash etc. You have a thousand degrees of slippery slope in what constitutes "enable". Does paying your workers in cash for tips enable tax evasion? Also, I'm not sure what scenario in which you would be able to find the encryption key, yet not have access to the wallet controlling the contracts? Even if you did, I'm not sure why hosts could not assist in cancelling contracts in a similar fashion to cloud providers taking down content on request.
      - wmf 7y ago
        Someone could upload data to you then later reveal the keys. If the authorities have the metadata and keys they can determine who is storing the data.
      - empath75 7y ago
        So basically Jeffrey Epstein gives you a CD full of encrypted files and asks you to hold on to them for him? Shouldn't be any problem with that.
  - pradn 7y ago
    I wonder about whether it's right to be renting your storage space away without knowing what it's being used for. I don't want to directly contribute toward bad things happening.
    - arbol 7y ago
      We need to move away from centralised internet. You don't have any real proof that Company XYZ aren't using your money to do bad things. You would stop participating if it turned out the Sia network was mostly used for bad things. In the same vein, you may choose not to use Amazon because it is unfairly treating warehouse workers.
    - brokenmachine 7y ago
      It's fine then because you'd only be indirectly contributing. Just like paying your taxes or buying fast food and clothes.
- Ajedi32 7y ago
  All data is fragmented and encrypted, so it's impossible to know whether a particular host is storing a fragment of illegal content or not. I suppose if you somehow _did_ find out you were hosting a fragment of illegal content you could breach the storage contract (and pay the resulting penalty) but that's very unlikely because, again, all content is encrypted.
  - alexkcd 7y ago
    Seems like this could be a common use case for pirates. Similar to how Megaupload or Fileshare worked a few years ago. Sia's encryption doesn't protect the file hosts, if the uploader posts a link + decryption keys for pirated content to a public file sharing forum. Does the take down notice go to everyone that hosts a fragment of the file? Who is liable if the uploader can't be traced?
    - Ajedi32 7y ago
      Good question. I suppose that would only work if whatever keys the pirate needed to publish to allow the data to be downloaded wouldn't _also_ give the keyholder the ability to delete that data. Otherwise the copyright holder could just issue the command to delete the file themselves. I'm not sure if Sia works that way or not; would be interesting to see.
      - alexkcd 7y ago
        Since they say their intent is to compete on price with S3 / CDNs, it seems possible to be able to download a file without having permissions to delete that file. If that were not the case, then Sia would be limited to personal backup only.
        ragle 7y ago
        It's confusing, because they refer to themselves as a potential competitor to S3 several times in the linked article, but I thought I read somewhere that conceptually what they're building is actually just the data persistence layer of a service like S3? A complete S3-like service would require a third-party tool on top of Sia. Goobox[1], for example, uses sia as a storage backend and provides an S3-compatible API[2]. In other words, right now - I think if you are interacting with Sia directly you can do whatever you want with the files you have access to. Not 100% sure about that. [1] - https://goobox.io/ https://goobox.io/ [2] - https://doc.goobox.io/#section/Why-you-should-choose-Goobox-S3-API-over-the-other-REST-APIs https://doc.goobox.io/#section/Why-you-should-choose-Goobox-...
- uwuhn 7y ago
  It doesn't. The last real application of Sia that I heard of during the previous crypto boom basically sounded like renting your HD space out to people who would potentially use it to store illegal content.
  - arcticbull 7y ago
    That's the biggest risk of permissionless / decentralized -- you learn quickly why permissioning and centralization are important.
    - hanniabu 7y ago
      Can you provide some of the reasons?
      - arcticbull 7y ago
        Otherwise encrypted illegal content is stored on everyone's computers, and technically makes them potential felons. And thanks to the magic of the blockchain, it's indelibly recorded forever.
        camjohnson26 7y ago
        What’s to stop a hacker from planting such content on your computer?
        arcticbull 7y ago
        Infosec usually, the difference here is you're taking on the risk and data knowingly and willfully. "Officer, it's part of a blockchain thing called Sia" is unlikely to satiate them.
  - antpls 7y ago
    The next step is to fight the idea of "illegal content". Bits are just bits, some just upset the elites, just like nipples.
- galaxyLogic 7y ago
  I find the concept of "Illegal Content" conceptually questionable. Authorities do not care much, I assume, if you possess illegal content. What they care about is if you share illegal content. Say you have a pro-democracy pamphlet on your hard-drive in China. If they find it they would imprison you no doubt but they don't care so much to look for it, and it is hard to find if you never share it with anybody. SIA provides private storage so if your only crime is possessing illegal content but never sharing it with anybody no big deal (I would assume). But if you share it then SIA can not protect you because the exchange must happen somewhere outside of SIA.
  - dymk 7y ago
    > Authorities do not care much, I assume, if you possess illegal content. What they care about is if you share illegal content. You'd be wrong - people get nabbed for having or creating child porn, not just distributing it.
    - galaxyLogic 7y ago
      I am sure. It would be an interesting case for the Supreme Court. Did you do something illegal by allowing SIA to store encrypted data on your hard-disk not knowing what it was?
    - camjohnson26 7y ago
      Which is stupid. If I were a member of a foreign government I’d use sophisticated hackers to plant that kind of content on anyone I didn’t like. Automatic jail for life.
- camjohnson26 7y ago
  How does http handle illegal content? Is there any reason that content should be blocked at a protocol level instead of punishing the person who created it through existing regulations? If Sia has a way to remove such content it loses its decentralized nature, which is the whole reason it exists in the first place. Should it exist? Well sometimes important content is illegal, such as political content in China. It’s good to have a way to publish important illegal content, even if bad stuff slips through. What if someone posts illegal violent or sexual content? That is unfortunate, but it means that a real person took the time to post such content on the network. Keeping them from speaking will not change their warped beliefs, it will just hide them from the world and keep people from remembering that evil people exist. The person who published the content should be punished through existing regulations and the content should stay on the network to remind us that some people are horrible. Hopefully no one accesses it.
  - Fornax96 7y ago
    Perfectly said. Removing "illegal" content does not solve the fact that the content exists. The problem should be tackled at the source.
- anonymous5133 7y ago
  All data is encrypted automatically before it is uploaded to the network. It is impossible for individual hosting nodes to see what is being stored.
theonlyklas 7y ago
Although the cost of cloud backups are egregious compared to the cost of raw storage, I'll pay a premium for the peace of mind that comes from letting someone else be held liable for storing my data and credentials to access it. I can easily upload/download files to google, iCloud, or Dropbox from almost any device knowing only my email and password, which I find preferable to having to remember an arbitrary 29 word seed. With the amount of exit scams in cryptocurrency, too, I just don't trust any project to continue to provide the same amount of utility that they do now, if they provide any at all. I suppose you could use a custodian site to link an email and password to the seed, but then you enter a centralized third party to the mix. I see value in Sia but it's just not for the average person in its current state.
- Ajedi32 7y ago
  You could store your Sia seed for free on Google Drive. That way you get the benefit of Google being "liable" for your data and credentials, but with the much lower storage costs of Sia. Yes, it's a centralized third party, but you already implied you have no problem with that.
- Scaevolus 7y ago
  Sia's seeds are ridiculous-- the 29 words provide 300 bits of entropy. 100 bits would be a sufficient security margin against brute forcing, assuming a modern memory-hard KDF like Argon2. With a 100 bit password, assuming every flop of the 1.8 exaflops of the Top500 supercomputers tested a new password, it would still take 25,000 years to crack. Key stretching should add at least 30 bits of security by taking a billion operations-- $ perf stat argon2 asdfasdf -id -m 16 -t 16 <<< asdf Type: Argon2id Iterations: 16 Memory: 65536 KiB 0.781 seconds 6,331,020,712 cycles # 4.021 GHz 13,467,211,117 instructions # 2.13 insn per cycle Here's what 100 bits of security margin looks like with a more sophisticated scheme (abbrase): "Hope raised between unpleasant bellows. Devil rode sullenly, refugees waiting." => (first three letters) hopraibetunpbeldevrodsulrefwai.
deleted 7y ago
[deleted]
perlgeek 7y ago
> and their prices are suspiciously similar at about $20 per TB per month. Today it is possible to purchase a refurbished 1 TB hard drive for about $20 — so buyers are essentially paying Amazon each month the entire cost of the hard drives used to store their data! Really? That's the argument? That you could store your files cheaper on a single, refurbished disk without considering the cost of housing, electricity, network, physical security, redundancy, management etc.?
- Ajedi32 7y ago
  The implication is that with Sia you can get those same benefits without having to essentially pay the entire cost of a new hard drive every month. (Sia's rates are much cheaper.)
- lunchables 7y ago
  Aren't all those costs priced in? I assumed the data storage with Sia was redundant and encrypted, which should cover basically all of those things, right? Looks like it from looking at the site: >File segments are created using a technology called Reed-Solomon erasure coding, commonly used in CDs and DVDs. Erasure coding allows Sia to divide files in a redundant manner, where any 10 of 30 segments can fully recover a user's files. [...] >Before leaving a renter's computer, each file segment is encrypted. This ensures that hosts only store encrypted segments of user data. I'd love for someone to do an analysis on the performance and availability of the data. I wonder if you could pay more for faster speeds or higher levels of redundancy? That would be neat features.
- rossjudson 7y ago
  I think the pricing information should probably be updated. The current public page for Google One says $99/year for 2 TB.
b_tterc_p 7y ago
> How is this possible? By removing intermediaries > [Sia] makes a small transaction fee from each file contract. Am I wrong in observing that the number of intermediaries between hosting data on AWS == the number of intermediaries oh Sia == 1? What exactly is this doing? Blockchain contracts to rent out your hard drives?
- Ajedi32 7y ago
  They explain what the mean by that two sentences later: > Hosts on Sia do not have to worry about building and maintaining enormous datacenters, employing thousands of employees, and marketing their services. They only need to worry about providing reliable storage capacity to renters on Sia. And yes, it's essentially a decentralized storage market built on smart contracts. You can either pay to store your files (the client software handles redundancy, contracts, etc automatically) or _get_ paid to rent out space on your hard drive.
  - b_tterc_p 7y ago
    How do they verify that a host can actually store all of the data?
    - Ajedi32 7y ago
      If I recall correctly, the client pre-computes hashes of random fragments of each file then "quizzes" the hosts on that information periodically. If they're not actually storing a copy of the data, they won't be able to compute the resulting hash. (I may be misremembering some of the details, but I believe that's how it works in principle.)
      - b_tterc_p 7y ago
        What if the client lies about the pre computed hash quizzes?
    - Taek 7y ago
      Ajedi32 is not correct. Data is verified probabilistically on the Sia network. The blockchain has access to the Merkle root of the data that the host is supposed to be storing. The blockchain will request that the host provide a 64 byte segment of the data (chosen randomly) along with a Merkle proof that the data is part of the Merkle root. If the host can provide the data and the proof, the host is rewarded as though they've demonstrated that they have all of the data. If the host cannot provide those 64 bytes along with a proof, the host is punished as though they are not storing any of the data.
deleted 7y ago
[deleted]
dnprock 7y ago
This is a flashback for me to 2006. I was working on storage technology and reviewing a few products in p2p storage. There's too much burden to apply p2p to get space. Storage is cheap enough for this idea to work.
SimianLogic2 7y ago
Can someone ELI5 why you need a blockchain for this? Another startup could make a non-blockchain version of this that uses similar syncing technology to GDrive / Dropbox file sync that just syncs other people's stuff instead of yours and pays you monthly.
- hanniabu 7y ago
  Non-reliant on central services or entities.
  - snarf21 7y ago
    This also cuts both ways. Your business needs a file but your customer can't connect to that node because of some political blocking of internet connections. Who can you rely on? What central service can you call to get this fixed asap?
    - buildzr 7y ago
      The idea with Sia is you upload to not one, but several people, you pick the level of redundancy you want and if you can't get the file you want from any given one of them, they pay the price and you get it from someone else.
    - ukd1 7y ago
      It's kinda designed to expect that; you end up paying for more redundancy over more hosts, which is a win. Yes some will / may be down, but a) they're penalized financially, b) you can (by default do) have redundancy
  - idlewords 7y ago
    Except for the software developers.
    - toomuchtodo 7y ago
      So exhausting watching people argue that trust is no longer necessary when they're simply pushing the trust around. You have to trust someone at the end of the day, and it's more practical to use the law to trust them (and enforce that trust) than cryptography. If I want to decentralize my storage, I would replicate data between multiple cloud object stores (Backblaze, S3, Wasabi, etc, a library to abstract PUTs and GETs) and pay them for their service. I trust them, but can verify each is holding up their end of their commercial agreement (similar to RAID block scans, but with object hash verifications). They aren't randomly going to yank their storage node of the network like a home NAS device or other consumer level storage device, because it's their business to be available. A superior solution has yet to present itself. Sidenote: The blog post is holding cloud provider data egress fees up as an example of exorbitant bandwidth pricing; this is true, but only from cloud providers. You can get bandwidth extremely cheap from non-cloud providers or through other means[1]. [1] https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/ (random example picked from Google search)
johnorourke 7y ago
Pied Piper lives!
rsync 7y ago
"The cloud storage industry is very similar to the financial industry in that trust is a necessity ..." https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/ "I Found the Holy Grail of Backups"
nullandvoid 7y ago
This takes me back I still have a few of there coins sitting around from years ago that have done nothing but tanked ( this was somewhat expected but I wanted to diversify in alts and this looked like one of the cooler options )
buildbuildbuild 7y ago
Abuse concerns and “does it need a blockchain” cynicism aside I find all of these “crypto” cloud computing projects fascinating. I highly recommend playing around on their testnets. Storj and Filecoin are also competing in the storage space, and BitTorrent is vying to be a CDN alternative. SONM aims to be an EC2 competitor, and interestingly requires ID verification to participate in their testnet due to the clear abuse potential of unattributable internet-connected VMs. (which exist in plenty of other places anyways since at present date, hosting providers have no KYC requirements) Will the sharing economy extend to computing resources? I’ll stay tuned.
- Taek 7y ago
  Decentralized computing is a challenge because unlike storage, it's very expensive to encrypt computation. You see something like 10,000x slowdowns. With storage, we can encrypt and checksum everything client-side in a cryptographically secure manner, and we can have full faith that even though we are giving our data to someone else, they have no way to view the data or tamper with the data. If techniques like homomorphic encryption gain substantial speedups, I'm sure that we'll start to see decentralized/trustless computing, but for now you really need to be doing your outsourced computing with someone you can trust and someone who is regulated, because it's possible to act maliciously.
  - wmf 7y ago
    Unless SGX works.
    - rjeli 7y ago
      SGX has always seemed like the Netflix DRM problem to me.. eventually, pixels have to show up on a screen somewhere, right? Am I mistaken in this line of thinking? Homomorphic encryption solves this, but it’s throwing an awful lot of performance out the window.
      - Taek 7y ago
        I think the idea behind SGX based cloud computing would be that you send encrypted data and code to the enclave, which then decrypts the data, does the computation, encrypts the result, and then returns the encrypted result. The enclave gets to see everything that's going on, but in theory nothing can view or corrupt what's going on inside of the enclave. I'm personally pretty skeptical that SGX is a secure platform, however if you do decide that SGX works and is trustworthy, it would be possible to outsource your computation in a mostly cost effective way to anyone who has an SGX enclave.
jonas21 7y ago
> Hosts on Sia do not have to worry about building and maintaining enormous datacenters, employing thousands of employees, and marketing their services. They only need to worry about providing reliable storage capacity to renters on Sia. I would expect that building large datacenters and employing people to manage them is exactly how you go about providing reliable storage capacity. And while it's true that providers don't have to worry about marketing or branding, well... the flip side is that they can't do any marketing or branding and need to compete in a market where the product is completely commoditized.
- Nursie 7y ago
  > I would expect that building large datacenters and employing people to manage them is exactly how you go about providing reliable storage capacity. Further, I imagine that if there is money to be made providing reliable storage to Sia users, then those with experience of reliably providing low-cost, high-volume storage will corner the market pretty fast!
- Taek 7y ago
  Sia is built on a completely different assumption than most large datacenters - uptime requirements for hosts are around 95%, which equates to about 36 hours of downtime per month. This means that you can run a much leaner database and require substantially less expertise to keep things going. If a rack goes down, you don't need someone on-site to bring it up ASAP, you can fix it the next workday without upsetting your customers. 95% uptime for hosts translates to 99.99+% uptime for Sia users, because users store data across hosts in a 10-of-30 scheme. As long as 10 out of 30 of the hosts are online, you will be able to access your data. The probability of losing 20 hosts when each of your 30 hosts have 95% uptime is exceedingly small; your practical uptime depends more on the reliability of the software than it does the reliability of the hosts you use. As for commoditization: that's the goal! If we can completely commoditize data storage, prices should come down dramatically, and the market should be a lot more efficient. You won't need to be an Amazon or Google to have a stable and competitive data offering.
  - q3k 7y ago
abalone 7y ago
Don't give up, I won't give up Don't give up, no no no Don't give up, I won't give up Don't give up, no no no - Blockchain guys (I know I know.. I couldn’t resist.)
buildzr 7y ago
I'm trying to understand the reliability story better. I'm currently looking at this page: http://siastats.info/contracts_status http://siastats.info/contracts_status Does that mean for every individual hoster-renter contract, of ones that were established and used, some ~9% of them failed? That would essentially mean that to get equivalent numbers to those advertised by B2, which claims 11 9s I'd need to buy over 10 contracts. Is this actually how bad it is at the moment?
- Taek 7y ago
  You are correct that the failure rate for contracts is somewhere around 9%, and you are also correct that you need over 10 contracts to get 11 9's of redundancy, however this doesn't translate to needing 10x redundancy on the Sia network to get high reliability. Data is uploaded to Sia (per-default, it's configurable through the API) in a 10-of-30 scheme using Reed-Solomon coding, which means that each piece of data is held by 30 hosts, and out of those 30 hosts any 10 of them are sufficient to recover the original data. This has a total overhead of 3x, and the algorithms behind it are in my opinion super fascinating. If you assume each host independently has 91% uptime, you get this amount of downtime as a result: https://www.wolframalpha.com/input/?i=sum+of+(30+choose+x)+*+0.91%5Ex+*+(1-0.91)%5E(30-x),+from+1+to+10 https://www.wolframalpha.com/input/?i=sum+of+(30+choose+x)+*... In reality, the software reliability is a bigger factor in downtime than the host's reliability.
  - buildzr 7y ago
    > Data is uploaded to Sia (per-default, it's configurable through the API) in a 10-of-30 scheme using Reed-Solomon coding, which means that each piece of data is held by 30 hosts, and out of those 30 hosts any 10 of them are sufficient to recover the original data. This sounds good - I'm just trying to understand how this is counted - are you establishing 30 separate contracts to achieve this?
    - Taek 7y ago
      Sia maintains 50 contracts with hosts at all times, and uses 30 of them for each file segment that gets uploaded. We use state channels, so we can use the same contracts each time you upload a new file, minimizing the total amount of on-chain activity.
pmlnr 7y ago
These ideas are all the wrong direction to decentralize. We don't need more layers and more complexity and more blockchains, we need simple, reliable, accessible household appliances. For storage, for sharing, for posting. Please stop adding layers and layers of calculations, it's a waste of energy.
- gnode 7y ago
  > Please stop adding layers and layers of calculations, it's a waste of energy. The trend in electronics is that this has become less true over time. As transistors have shrunk, the relative cost of calculation has diminished compared to the cost of transmission over distances. A cryptographic operation can be cheap compared to accessing main memory because the distances signals travel within the crypto logic block are tiny. Complexity (as in more to understand) does not equate sensibly to higher power consumption.
- anonymous5133 7y ago
  From what I understand, right now these projects are working on the backbone of the systems. In the future, other developers will be able to create services on top of these platforms to resell the storage space. So maybe someone will have an online cloud hosting service with a simple drag and drop online interface. They will charge a fee to the customer and the website operator will simply buy storage contracts on SIA or similar providers to provide the actual hosting and distribution of the files.
  - filebase 7y ago
    You would be correct! Filebase[1] provides S3-compatible object storage at a fraction of the price. We happily use Sia on our backend. :) [1] https://filebase.com https://filebase.com
ukd1 7y ago
Long time user and follower of Sia. It's always surprising to me that it's overlooked compared to filecoin (which is similar, but seems like $250m+ vaporware to date). I use it to have tertiary backups. Weird caveat I found: the smallest chargeable size is actually pretty large.
- xur17 7y ago
  > Weird caveat I found: the smallest chargeable size is actually pretty large. This and the cost to form contracts is is large enough that it doesn't make economical sense until you are storing > $1gb. That said, they seem to slowly be working through a number of these concerns. They very recently released seed based backups, one of the biggest issues I had last time I looked.
  - ukd1 7y ago
    Ya; that's another one - the same wallet on multiple machines = different set of contracts. Didn't notice that was fixed though...
    - zherbert 7y ago
      We do not yet have the ability to share the same set of contracts across multiple machines, but it is on the roadmap. This would include using the same set of contracts on desktop and mobile devices, which will open up some intriguing use-cases.
neiman 7y ago
I tried Sia about 2 years ago. It worked "ok" for half a GB files, However, for 4-5 GB my upload got stuck while the help forum couldn't provide answers besides "wait for the next release". There were also unclear issues back then. Are the files encrypted on my computer before being uploaded? are they sharded between many hosts? or are they stored in one host? How long will they be stored. Most of these issues seem like they could be improved. Not an inherent faults of the technology or something. I like Sia's idea and tech in general; I'll give it a try again now.
- Taek 7y ago
  Sia has come a long way in the past two years. Uploads, downloads, and repairs are both much more stable and also a lot faster. Sia can effortlessly sustain 100mbps upload and download if your connection is that fast. Sia is happy with files that are hundreds of GBs in size today, and with a filesystem that that gets up to about 20 TB. Beyond that it starts to struggle, but we're continuously working to expand its capabilities. Data has always been encrypted client-side, even in very early releases. Data today is sharded between hosts in a 10-of-30 Reed-Solomon scheme. I think you will find that the experience is much smoother and more complete compared to the experience from 2 years ago. And, a critical feature is now available as of the latest release: seed based file recovery. After you upload your files, it's possible to create a snapshot of your data that you can recover later on a different machine using nothing more than your wallet seed. This makes Sia a practical and cost effective solution for secondary or tertiary backup.
nisten 7y ago
There is one missing piece to all this, I/O. Given they publish their apps on all platforms they could probably add this in the future as an ssd or PCIe storage tier by doing regular disk benchmarks. However it seems like a major limiting factor, given that random access speeds on spinning drives or even cheap ssds can drop drastically by an order or two of magnitude (i.e. from 130MB/s to 2MB/s). Given how they're doing error correction in a distributed manner, having a fast storage tier would be a major advantage for them which would eliminate any problems from flash storage wearing out and fully gain the benefit of throughput, random access times and cheap new prices of nvme drives. At the moment I probably can't use this for streaming my own 4k videos to myself because an up to 300mbps network is not good enough for that, the slowest storage in it would be a bottleneck for a significant amounts of the experience of watching the streamed file. And I definitely can't use it for application storage given how important response time is to the user experience and user growth. That all aside, those 2 are relatively extreme examples and there's a lot of usage scenarios in between them. I do admire how far they've gotten this given all the blockchain hyped products that have failed. They seem to have a solid system here that will only improve on performance and put pressure on bringing online storage prices down.
- Taek 7y ago
  Most 4K video is closer to 50mbps. If your home connection is that fast, Sia should be more than capable of delivering smooth 4K streaming to your home. A single host with slow storage is not a blocking factor because Sia downloads in parallel from as many as 30 hosts at once when composing a stream of data. Even if each has an I/O bottleneck of 1 MB/s (8mbps), you'll be able to get your video out at 240mbps. In practice today, Sia hosts are almost all network bottlenecked, most use HDDs which do not have trouble keeping up with network latencies (HDD seek times are 11ms, network latencies usually 50ms+) or throughputs (HDDs can usually get close to 100 MB/s, far more than the network throughput) Can I ask, have you tried streaming 4K video from the latest version of Sia? Seek times might be 5-10 seconds, but I believe that once your stream has loaded you will see completely smooth playback. Our next release has a focus on reducing those seek times, but the code is not ready yet.
  - 7y ago
jchw 7y ago
> Today it is possible to purchase a refurbished 1 TB hard drive for about $20 — so buyers are essentially paying Amazon each month the entire cost of the hard drives used to store their data! What a refurbished hard disk does not give you: - Redundant storage - Redundant power/network/etc. - Geographic distribution - A network API - Any networking, or uplinks at all - Access control - A team ensuring the reliability and security of your data - Global network of datacenters and pops - Integration with other cloud services All that and Amazon et al. do not store your 1TB of data in a single refurbished hard disk. This is beyond non-sense. Sure, some of these costs are amortized, but I still think this is a lame comparison. If all you wanted was at-cost block storage with crappy reliability and speed, you could probably accomplish that in easier ways.
- Taek 7y ago
  One of the main innovations of Sia is providing a low cost means to turn that 1 TB refurbished hard drive into a useful cloud product. Sia is able to provide, at 1/10th the cost of Amazon, high throughput, high reliability, high redundancy, geographically distributed cloud storage, and it can do that without high reliability drives or datacenters.
- leetbulb 7y ago
  Off topic, but: I purchased three refurb 6TB HGST He HDDs in Jan. from a reputable shop. All three failed last month. I was so happy that the shop completely refunded all of them because the warranty is generally very limited on refurb HDDs. Normal RMA is 5yr on this specific model. There's no way refurb drives are cost effective for these environments.
  - AdamJacobMuller 7y ago
    On balance, I purchased 45 refurb 4TB SAS drives around the same time. I had two DOA with 0% failure rate since then. Also, very anecdotal, I know.
    - leetbulb 7y ago
      I'm wondering if the fact mine were He filled refurbs could've been a factor.
siavosh 7y ago
Can someone explain the difference between Sia vs Filecoin/IPFS. The latter seems to be still working on solving some fundamental crypto primitives before they can offer what seems like what Sia already claims to have solved?
- wmf 7y ago
  Yes, Filecoin is trying to come up with a "more efficient" and possibly "elegant" implementation based on algorithms that are still being invented. Of course, with 100x the budget it's possible that they could still overtake Sia.
tempsolution 7y ago
This reads like a typical blockchain non-sense hype article. Before you even go down the road of comparing sia to S3, what about some unbiased benchmarks? Show us the 99.999999999% durability, the pretty much unlimited bandwidth, the single digit milli-second latencies, the practically unlimited capacity. If you want to compare Sia to anything, compare it to Dropbox or OneDrive. S3 was made for an entirely different purpose and Sia can't even remotely compete with it. As a personal data store? Maybe. But wait until this thing becomes more popular and hackers start making a mess. Will be interesting to see how stable this "fully decentralized" network is, once it gets the full attention from bad actors as all the cloud providers.
- rolltiide 7y ago
  coming from someone who is not a fan of Siacoin from when I last used it in 2017, I think you miss the point what I like and want is encrypted distributed data that I can store and retrieve quickly and cheaply I also like an economic model that aligns the hosts to maintain the data, and how the proliferation of that economic model lets me invest in its growth from merely being a passive non-productive speculator, and finally I would like a clear easy to understand ROI from being a productive host: can I subsidize the cost of my hardware that I'm already interested in having. any cryptocurrency can do that, but how the software uses the cryptocurrency is a key factor, and unfortunately we can like an idea and team but get forced to accept an economic model that has flaws so for me, it is easy for me to understand why teams want to promise the sky against S3, and it really isn't that relevant to using this securely. like, you might be able to subsidize all of your hardware and provide a service without worrying about if the SIA comparison is accurate because its really not relevant. but finally, it turns out that unmonetized and non-cryptocurrency based decentralized cloud storage networks are good enough! so I stick with IPFS. hope that was useful to someone.
  - carlosdp 7y ago
    IPFS isn't decentralized cloud storage though. Unless you host your file yourself on your IPFS node, your file is not guaranteed to be available. Hence why Filecoin is a thing they are working on.
zherbert 7y ago
Hi all, author here, just wanted to say I am thrilled to see the HN community discuss decentralized cloud storage in more detail. There are many drawbacks today and narrow use-cases, but our goal is to continue to improve Sia – and over the coming years we will prove that decentralized storage can compete directly with Amazon, Google, and Microsoft. We feel confident that the marketplace dynamics, in particular, will foster competition between hosts on the network and maintain prices that are an order of magnitude cheaper for storage and two orders of magnitude cheaper for bandwidth.
- zherbert 7y ago
  I would also add – give Sia a try, especially if you have tried it before our v1.4.0/v1.4.1 releases! https://sia.tech/get-started https://sia.tech/get-started
- fitzroy 7y ago
  Gotta say, the fact that the installation instructions consist primarily of getting around the "unknown / unidentified developer" restriction doesn't inspire confidence.
- solidasparagus 7y ago
  Pretty arrogant to say you're going to make cloud storage providers obsolete when you haven't even built a competing solution. You can't even share data yet... And what about all the other features that cloud providers offer? You say "[the] Sia software is exponentially improving, and performance and featureset is quickly approaching Amazon S3", but I see absolutely no evidence of even the most basic features of cloud storage providers. Authorization? Regulatory compliance? High-performance bandwidth? High-reliability public API/URLs? Customer support? Private cloud peering? Zero effort integration with many (most?) major data-related open source projects? Versioning? One-click, no-knowledge reliability? How much do you need to grow to hit cloud scale? 1,000,000x? More? And you're already claiming that you're going to make S3 obsolete? Your blog would hold more weight if you weren't so over-the-top, bullshit-level optimistic.
  - peterkelly 7y ago
Proven 7y ago
If only there was a use case for it
alexnewman 7y ago
I was greatly skeptical of these types of systems but ipfs proved me wrong. It's fine, when combined with a CDN. I'm interested in what new use cases Sia handles when CDN + IPFS works so well. For instance my blog https://www.wuli.nu https://www.wuli.nu (source code https://github.com/posix4e/blog https://github.com/posix4e/blog)
- psKama 7y ago
  Can you please clarify which CDN features are you referring to on your blog (can't find any) or anywhere else? Also can you elaborate what do you even mean by "so well"? Sia was started to be built before IPFS was even announced and has hosted PBs of data already as a working product. On the other hand IPFS hasn't even released it's public testnet so it would be great to understand your point. Edit: btw by IPFS I assume you refer to their Sia equivalent product, Filecoin. At least I do, as IPFS cannot be compared to Sia alone.
  - alexnewman 7y ago
    I am in no way referring to filecoin. I am talking about IPFS and cloudflare integration. https://blog.cloudflare.com/distributed-web-gateway/ https://blog.cloudflare.com/distributed-web-gateway/
  - alexnewman 7y ago
    I actually think filecoin is worthless and IPFS is the only valuable thing that came out of that group
truth_seeker 7y ago
Can somebody from Sia team compare it to Radix DLT and it's mechanism to shard and distribute data ? https://www.radixdlt.com/ https://www.radixdlt.com/
x3haloed 7y ago
Is forums.sia.tech down temporarily or permanently?
2youmich 7y ago
Sia is a future
gridlockd 7y ago
I have a feeling this is an apples to oranges comparison. Can you serve clients directly over HTTP using Sia in a permissioned manner? If not, then it's not usable like S3. If it's not usable like S3, then S3 pricing comparison is meaningless. Also, how correlated is Sia pricing to the overall cryptocurrency market? Will my costs go through the roof when there's another crypto bubble? Would it actually scale? If Sia became popular, would prices go up like Bitcoin transaction prices? Isn't the extremely cheap bandwidth due to spare capacity of average broadband plans? If so, wouldn't providers eventually cut that off? Broadband pricing relies on the fact that most users don't use all of the bandwidth most of the time.
hereme888 7y ago
Just switched to backing my data on Sia. Was previously using Nextcloud, which is great, but more expensive. Took a little time to set up, but renting 200 GB for 10 cents/mo, in a decentralized system that encrypts my data, has great uptime, and doesn't mine my personal or "aggregated" data, is unbeatable. I heard Sia is not great at handling large number of files, so I'm using a backup software to create automated, compressed backups first, and Sia handles those larger files. I'll see how it pans out over time.

276 points by Geee 7y ago

yasp 7y ago

How do services like Sia handle illegal content?

jameslevy 7y ago
I believe that censorship resistance is a key value proposition for decentralized file storage. So its agnostic handling of illegal content is a feature, not a bug.
- yazan94 7y ago
  In the spirit of blockchain and decentralization, I think you're right. But practically, explaining that to the authorities will probably not be a pleasant experience if they manage to find cp or other illegal data on your hard drive.
  - Ajedi32 7y ago
    The data is encrypted by the client, so that's theoretically impossible. Neither you nor the authorities have the ability to determine what file fragments (whole flies aren't stored; just fragments) are on your PC.
    - modeless 7y ago
      It's not theoretically impossible for law enforcement to discover the encryption keys to some illegal files on someone's computer. Then they could go around arresting people who host the chunks. Of course those people wouldn't have known about those files in particular, but they were knowingly running software that enables hosting illegal files. A judge would have to decide if that's OK.
      - Ajedi32 7y ago
        Doesn't AWS also have the same problem? Someone could upload encrypted illegal content, then later law enforcement could discover the key. Why hasn't AWS gotten in trouble for that yet?
        modeless 7y ago
        No large cloud provider allows anonymous usage. They take down content on request and assist law enforcement in catching the non-anonymous people doing illegal things on their platforms.
        Ajedi32 7y ago
        Okay, bad example. AWS is a paid service so obviously they have more information about their users than most. What about Dropbox or OneDrive? Both of those services allow files to be stored and shared with no requirement to identify yourself when you're creating an account, beyond providing an email address.
        txcwpalpha 7y ago
        Both Dropbox and OneDrive scan your content to determine if it matches known illegal content (be it illegal pornography, DRM content, etc) and will remove it and remove you from their service. See https://www.microsoft.com/en-us/photodna https://www.microsoft.com/en-us/photodna If someone pre-encrypts data and then uploads that to Dropbox, it gets more complicated and AFAIK involves more of monitoring the IP addresses that things are coming from and/or being shared to.
        lmpostor 7y ago
        Same argument for Bitcoin, cash etc. You have a thousand degrees of slippery slope in what constitutes "enable". Does paying your workers in cash for tips enable tax evasion? Also, I'm not sure what scenario in which you would be able to find the encryption key, yet not have access to the wallet controlling the contracts? Even if you did, I'm not sure why hosts could not assist in cancelling contracts in a similar fashion to cloud providers taking down content on request.
    - wmf 7y ago
      Someone could upload data to you then later reveal the keys. If the authorities have the metadata and keys they can determine who is storing the data.
    - empath75 7y ago
      So basically Jeffrey Epstein gives you a CD full of encrypted files and asks you to hold on to them for him? Shouldn't be any problem with that.
- pradn 7y ago
  I wonder about whether it's right to be renting your storage space away without knowing what it's being used for. I don't want to directly contribute toward bad things happening.
  - arbol 7y ago
    We need to move away from centralised internet. You don't have any real proof that Company XYZ aren't using your money to do bad things. You would stop participating if it turned out the Sia network was mostly used for bad things. In the same vein, you may choose not to use Amazon because it is unfairly treating warehouse workers.
  - brokenmachine 7y ago
    It's fine then because you'd only be indirectly contributing. Just like paying your taxes or buying fast food and clothes.
Ajedi32 7y ago
All data is fragmented and encrypted, so it's impossible to know whether a particular host is storing a fragment of illegal content or not. I suppose if you somehow _did_ find out you were hosting a fragment of illegal content you could breach the storage contract (and pay the resulting penalty) but that's very unlikely because, again, all content is encrypted.
- alexkcd 7y ago
  Seems like this could be a common use case for pirates. Similar to how Megaupload or Fileshare worked a few years ago. Sia's encryption doesn't protect the file hosts, if the uploader posts a link + decryption keys for pirated content to a public file sharing forum. Does the take down notice go to everyone that hosts a fragment of the file? Who is liable if the uploader can't be traced?
  - Ajedi32 7y ago
    Good question. I suppose that would only work if whatever keys the pirate needed to publish to allow the data to be downloaded wouldn't _also_ give the keyholder the ability to delete that data. Otherwise the copyright holder could just issue the command to delete the file themselves. I'm not sure if Sia works that way or not; would be interesting to see.
    - alexkcd 7y ago
      Since they say their intent is to compete on price with S3 / CDNs, it seems possible to be able to download a file without having permissions to delete that file. If that were not the case, then Sia would be limited to personal backup only.
      - ragle 7y ago
        It's confusing, because they refer to themselves as a potential competitor to S3 several times in the linked article, but I thought I read somewhere that conceptually what they're building is actually just the data persistence layer of a service like S3? A complete S3-like service would require a third-party tool on top of Sia. Goobox[1], for example, uses sia as a storage backend and provides an S3-compatible API[2]. In other words, right now - I think if you are interacting with Sia directly you can do whatever you want with the files you have access to. Not 100% sure about that. [1] - https://goobox.io/ https://goobox.io/ [2] - https://doc.goobox.io/#section/Why-you-should-choose-Goobox-S3-API-over-the-other-REST-APIs https://doc.goobox.io/#section/Why-you-should-choose-Goobox-...
uwuhn 7y ago
It doesn't. The last real application of Sia that I heard of during the previous crypto boom basically sounded like renting your HD space out to people who would potentially use it to store illegal content.
- arcticbull 7y ago
  That's the biggest risk of permissionless / decentralized -- you learn quickly why permissioning and centralization are important.
  - hanniabu 7y ago
    Can you provide some of the reasons?
    - arcticbull 7y ago
      Otherwise encrypted illegal content is stored on everyone's computers, and technically makes them potential felons. And thanks to the magic of the blockchain, it's indelibly recorded forever.
      - camjohnson26 7y ago
        What’s to stop a hacker from planting such content on your computer?
        arcticbull 7y ago
        Infosec usually, the difference here is you're taking on the risk and data knowingly and willfully. "Officer, it's part of a blockchain thing called Sia" is unlikely to satiate them.
- antpls 7y ago
  The next step is to fight the idea of "illegal content". Bits are just bits, some just upset the elites, just like nipples.
galaxyLogic 7y ago
I find the concept of "Illegal Content" conceptually questionable. Authorities do not care much, I assume, if you possess illegal content. What they care about is if you share illegal content. Say you have a pro-democracy pamphlet on your hard-drive in China. If they find it they would imprison you no doubt but they don't care so much to look for it, and it is hard to find if you never share it with anybody. SIA provides private storage so if your only crime is possessing illegal content but never sharing it with anybody no big deal (I would assume). But if you share it then SIA can not protect you because the exchange must happen somewhere outside of SIA.
- dymk 7y ago
  > Authorities do not care much, I assume, if you possess illegal content. What they care about is if you share illegal content. You'd be wrong - people get nabbed for having or creating child porn, not just distributing it.
  - galaxyLogic 7y ago
    I am sure. It would be an interesting case for the Supreme Court. Did you do something illegal by allowing SIA to store encrypted data on your hard-disk not knowing what it was?
  - camjohnson26 7y ago
    Which is stupid. If I were a member of a foreign government I’d use sophisticated hackers to plant that kind of content on anyone I didn’t like. Automatic jail for life.
camjohnson26 7y ago
How does http handle illegal content? Is there any reason that content should be blocked at a protocol level instead of punishing the person who created it through existing regulations? If Sia has a way to remove such content it loses its decentralized nature, which is the whole reason it exists in the first place. Should it exist? Well sometimes important content is illegal, such as political content in China. It’s good to have a way to publish important illegal content, even if bad stuff slips through. What if someone posts illegal violent or sexual content? That is unfortunate, but it means that a real person took the time to post such content on the network. Keeping them from speaking will not change their warped beliefs, it will just hide them from the world and keep people from remembering that evil people exist. The person who published the content should be punished through existing regulations and the content should stay on the network to remind us that some people are horrible. Hopefully no one accesses it.
- Fornax96 7y ago
  Perfectly said. Removing "illegal" content does not solve the fact that the content exists. The problem should be tackled at the source.
anonymous5133 7y ago
All data is encrypted automatically before it is uploaded to the network. It is impossible for individual hosting nodes to see what is being stored.

theonlyklas 7y ago

Although the cost of cloud backups are egregious compared to the cost of raw storage, I'll pay a premium for the peace of mind that comes from letting someone else be held liable for storing my data and credentials to access it. I can easily upload/download files to google, iCloud, or Dropbox from almost any device knowing only my email and password, which I find preferable to having to remember an arbitrary 29 word seed. With the amount of exit scams in cryptocurrency, too, I just don't trust any project to continue to provide the same amount of utility that they do now, if they provide any at all. I suppose you could use a custodian site to link an email and password to the seed, but then you enter a centralized third party to the mix. I see value in Sia but it's just not for the average person in its current state.

Ajedi32 7y ago
You could store your Sia seed for free on Google Drive. That way you get the benefit of Google being "liable" for your data and credentials, but with the much lower storage costs of Sia. Yes, it's a centralized third party, but you already implied you have no problem with that.
Scaevolus 7y ago
Sia's seeds are ridiculous-- the 29 words provide 300 bits of entropy. 100 bits would be a sufficient security margin against brute forcing, assuming a modern memory-hard KDF like Argon2. With a 100 bit password, assuming every flop of the 1.8 exaflops of the Top500 supercomputers tested a new password, it would still take 25,000 years to crack. Key stretching should add at least 30 bits of security by taking a billion operations-- $ perf stat argon2 asdfasdf -id -m 16 -t 16 <<< asdf Type: Argon2id Iterations: 16 Memory: 65536 KiB 0.781 seconds 6,331,020,712 cycles # 4.021 GHz 13,467,211,117 instructions # 2.13 insn per cycle Here's what 100 bits of security margin looks like with a more sophisticated scheme (abbrase): "Hope raised between unpleasant bellows. Devil rode sullenly, refugees waiting." => (first three letters) hopraibetunpbeldevrodsulrefwai.

deleted 7y ago

[deleted]

perlgeek 7y ago

> and their prices are suspiciously similar at about $20 per TB per month. Today it is possible to purchase a refurbished 1 TB hard drive for about $20 — so buyers are essentially paying Amazon each month the entire cost of the hard drives used to store their data! Really? That's the argument? That you could store your files cheaper on a single, refurbished disk without considering the cost of housing, electricity, network, physical security, redundancy, management etc.?

Ajedi32 7y ago
The implication is that with Sia you can get those same benefits without having to essentially pay the entire cost of a new hard drive every month. (Sia's rates are much cheaper.)
lunchables 7y ago
Aren't all those costs priced in? I assumed the data storage with Sia was redundant and encrypted, which should cover basically all of those things, right? Looks like it from looking at the site: >File segments are created using a technology called Reed-Solomon erasure coding, commonly used in CDs and DVDs. Erasure coding allows Sia to divide files in a redundant manner, where any 10 of 30 segments can fully recover a user's files. [...] >Before leaving a renter's computer, each file segment is encrypted. This ensures that hosts only store encrypted segments of user data. I'd love for someone to do an analysis on the performance and availability of the data. I wonder if you could pay more for faster speeds or higher levels of redundancy? That would be neat features.
rossjudson 7y ago
I think the pricing information should probably be updated. The current public page for Google One says $99/year for 2 TB.

b_tterc_p 7y ago

> How is this possible? By removing intermediaries > [Sia] makes a small transaction fee from each file contract. Am I wrong in observing that the number of intermediaries between hosting data on AWS == the number of intermediaries oh Sia == 1? What exactly is this doing? Blockchain contracts to rent out your hard drives?

Ajedi32 7y ago
They explain what the mean by that two sentences later: > Hosts on Sia do not have to worry about building and maintaining enormous datacenters, employing thousands of employees, and marketing their services. They only need to worry about providing reliable storage capacity to renters on Sia. And yes, it's essentially a decentralized storage market built on smart contracts. You can either pay to store your files (the client software handles redundancy, contracts, etc automatically) or _get_ paid to rent out space on your hard drive.
- b_tterc_p 7y ago
  How do they verify that a host can actually store all of the data?
  - Ajedi32 7y ago
    If I recall correctly, the client pre-computes hashes of random fragments of each file then "quizzes" the hosts on that information periodically. If they're not actually storing a copy of the data, they won't be able to compute the resulting hash. (I may be misremembering some of the details, but I believe that's how it works in principle.)
    - b_tterc_p 7y ago
      What if the client lies about the pre computed hash quizzes?
  - Taek 7y ago
    Ajedi32 is not correct. Data is verified probabilistically on the Sia network. The blockchain has access to the Merkle root of the data that the host is supposed to be storing. The blockchain will request that the host provide a 64 byte segment of the data (chosen randomly) along with a Merkle proof that the data is part of the Merkle root. If the host can provide the data and the proof, the host is rewarded as though they've demonstrated that they have all of the data. If the host cannot provide those 64 bytes along with a proof, the host is punished as though they are not storing any of the data.

deleted 7y ago

[deleted]

dnprock 7y ago

This is a flashback for me to 2006. I was working on storage technology and reviewing a few products in p2p storage. There's too much burden to apply p2p to get space. Storage is cheap enough for this idea to work.

SimianLogic2 7y ago

Can someone ELI5 why you need a blockchain for this? Another startup could make a non-blockchain version of this that uses similar syncing technology to GDrive / Dropbox file sync that just syncs other people's stuff instead of yours and pays you monthly.

hanniabu 7y ago
Non-reliant on central services or entities.
- snarf21 7y ago
  This also cuts both ways. Your business needs a file but your customer can't connect to that node because of some political blocking of internet connections. Who can you rely on? What central service can you call to get this fixed asap?
  - buildzr 7y ago
    The idea with Sia is you upload to not one, but several people, you pick the level of redundancy you want and if you can't get the file you want from any given one of them, they pay the price and you get it from someone else.
  - ukd1 7y ago
    It's kinda designed to expect that; you end up paying for more redundancy over more hosts, which is a win. Yes some will / may be down, but a) they're penalized financially, b) you can (by default do) have redundancy
- idlewords 7y ago
  Except for the software developers.
  - toomuchtodo 7y ago
    So exhausting watching people argue that trust is no longer necessary when they're simply pushing the trust around. You have to trust someone at the end of the day, and it's more practical to use the law to trust them (and enforce that trust) than cryptography. If I want to decentralize my storage, I would replicate data between multiple cloud object stores (Backblaze, S3, Wasabi, etc, a library to abstract PUTs and GETs) and pay them for their service. I trust them, but can verify each is holding up their end of their commercial agreement (similar to RAID block scans, but with object hash verifications). They aren't randomly going to yank their storage node of the network like a home NAS device or other consumer level storage device, because it's their business to be available. A superior solution has yet to present itself. Sidenote: The blog post is holding cloud provider data egress fees up as an example of exorbitant bandwidth pricing; this is true, but only from cloud providers. You can get bandwidth extremely cheap from non-cloud providers or through other means[1]. [1] https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/ (random example picked from Google search)

johnorourke 7y ago

Pied Piper lives!

rsync 7y ago

"The cloud storage industry is very similar to the financial industry in that trust is a necessity ..." https://www.stavros.io/posts/holy-grail-backups/ https://www.stavros.io/posts/holy-grail-backups/ "I Found the Holy Grail of Backups"

nullandvoid 7y ago

This takes me back I still have a few of there coins sitting around from years ago that have done nothing but tanked ( this was somewhat expected but I wanted to diversify in alts and this looked like one of the cooler options )

buildbuildbuild 7y ago

Abuse concerns and “does it need a blockchain” cynicism aside I find all of these “crypto” cloud computing projects fascinating. I highly recommend playing around on their testnets. Storj and Filecoin are also competing in the storage space, and BitTorrent is vying to be a CDN alternative. SONM aims to be an EC2 competitor, and interestingly requires ID verification to participate in their testnet due to the clear abuse potential of unattributable internet-connected VMs. (which exist in plenty of other places anyways since at present date, hosting providers have no KYC requirements) Will the sharing economy extend to computing resources? I’ll stay tuned.

Taek 7y ago
Decentralized computing is a challenge because unlike storage, it's very expensive to encrypt computation. You see something like 10,000x slowdowns. With storage, we can encrypt and checksum everything client-side in a cryptographically secure manner, and we can have full faith that even though we are giving our data to someone else, they have no way to view the data or tamper with the data. If techniques like homomorphic encryption gain substantial speedups, I'm sure that we'll start to see decentralized/trustless computing, but for now you really need to be doing your outsourced computing with someone you can trust and someone who is regulated, because it's possible to act maliciously.
- wmf 7y ago
  Unless SGX works.
  - rjeli 7y ago
    SGX has always seemed like the Netflix DRM problem to me.. eventually, pixels have to show up on a screen somewhere, right? Am I mistaken in this line of thinking? Homomorphic encryption solves this, but it’s throwing an awful lot of performance out the window.
    - Taek 7y ago
      I think the idea behind SGX based cloud computing would be that you send encrypted data and code to the enclave, which then decrypts the data, does the computation, encrypts the result, and then returns the encrypted result. The enclave gets to see everything that's going on, but in theory nothing can view or corrupt what's going on inside of the enclave. I'm personally pretty skeptical that SGX is a secure platform, however if you do decide that SGX works and is trustworthy, it would be possible to outsource your computation in a mostly cost effective way to anyone who has an SGX enclave.

jonas21 7y ago

> Hosts on Sia do not have to worry about building and maintaining enormous datacenters, employing thousands of employees, and marketing their services. They only need to worry about providing reliable storage capacity to renters on Sia. I would expect that building large datacenters and employing people to manage them is exactly how you go about providing reliable storage capacity. And while it's true that providers don't have to worry about marketing or branding, well... the flip side is that they can't do any marketing or branding and need to compete in a market where the product is completely commoditized.

Nursie 7y ago
> I would expect that building large datacenters and employing people to manage them is exactly how you go about providing reliable storage capacity. Further, I imagine that if there is money to be made providing reliable storage to Sia users, then those with experience of reliably providing low-cost, high-volume storage will corner the market pretty fast!
Taek 7y ago
Sia is built on a completely different assumption than most large datacenters - uptime requirements for hosts are around 95%, which equates to about 36 hours of downtime per month. This means that you can run a much leaner database and require substantially less expertise to keep things going. If a rack goes down, you don't need someone on-site to bring it up ASAP, you can fix it the next workday without upsetting your customers. 95% uptime for hosts translates to 99.99+% uptime for Sia users, because users store data across hosts in a 10-of-30 scheme. As long as 10 out of 30 of the hosts are online, you will be able to access your data. The probability of losing 20 hosts when each of your 30 hosts have 95% uptime is exceedingly small; your practical uptime depends more on the reliability of the software than it does the reliability of the hosts you use. As for commoditization: that's the goal! If we can completely commoditize data storage, prices should come down dramatically, and the market should be a lot more efficient. You won't need to be an Amazon or Google to have a stable and competitive data offering.
- q3k 7y ago

abalone 7y ago

Don't give up, I won't give up Don't give up, no no no Don't give up, I won't give up Don't give up, no no no - Blockchain guys (I know I know.. I couldn’t resist.)

buildzr 7y ago

I'm trying to understand the reliability story better. I'm currently looking at this page: http://siastats.info/contracts_status http://siastats.info/contracts_status Does that mean for every individual hoster-renter contract, of ones that were established and used, some ~9% of them failed? That would essentially mean that to get equivalent numbers to those advertised by B2, which claims 11 9s I'd need to buy over 10 contracts. Is this actually how bad it is at the moment?

Taek 7y ago
You are correct that the failure rate for contracts is somewhere around 9%, and you are also correct that you need over 10 contracts to get 11 9's of redundancy, however this doesn't translate to needing 10x redundancy on the Sia network to get high reliability. Data is uploaded to Sia (per-default, it's configurable through the API) in a 10-of-30 scheme using Reed-Solomon coding, which means that each piece of data is held by 30 hosts, and out of those 30 hosts any 10 of them are sufficient to recover the original data. This has a total overhead of 3x, and the algorithms behind it are in my opinion super fascinating. If you assume each host independently has 91% uptime, you get this amount of downtime as a result: https://www.wolframalpha.com/input/?i=sum+of+(30+choose+x)+*+0.91%5Ex+*+(1-0.91)%5E(30-x),+from+1+to+10 https://www.wolframalpha.com/input/?i=sum+of+(30+choose+x)+*... In reality, the software reliability is a bigger factor in downtime than the host's reliability.
- buildzr 7y ago
  > Data is uploaded to Sia (per-default, it's configurable through the API) in a 10-of-30 scheme using Reed-Solomon coding, which means that each piece of data is held by 30 hosts, and out of those 30 hosts any 10 of them are sufficient to recover the original data. This sounds good - I'm just trying to understand how this is counted - are you establishing 30 separate contracts to achieve this?
  - Taek 7y ago
    Sia maintains 50 contracts with hosts at all times, and uses 30 of them for each file segment that gets uploaded. We use state channels, so we can use the same contracts each time you upload a new file, minimizing the total amount of on-chain activity.

pmlnr 7y ago

These ideas are all the wrong direction to decentralize. We don't need more layers and more complexity and more blockchains, we need simple, reliable, accessible household appliances. For storage, for sharing, for posting. Please stop adding layers and layers of calculations, it's a waste of energy.

gnode 7y ago
> Please stop adding layers and layers of calculations, it's a waste of energy. The trend in electronics is that this has become less true over time. As transistors have shrunk, the relative cost of calculation has diminished compared to the cost of transmission over distances. A cryptographic operation can be cheap compared to accessing main memory because the distances signals travel within the crypto logic block are tiny. Complexity (as in more to understand) does not equate sensibly to higher power consumption.
anonymous5133 7y ago
From what I understand, right now these projects are working on the backbone of the systems. In the future, other developers will be able to create services on top of these platforms to resell the storage space. So maybe someone will have an online cloud hosting service with a simple drag and drop online interface. They will charge a fee to the customer and the website operator will simply buy storage contracts on SIA or similar providers to provide the actual hosting and distribution of the files.
- filebase 7y ago
  You would be correct! Filebase[1] provides S3-compatible object storage at a fraction of the price. We happily use Sia on our backend. :) [1] https://filebase.com https://filebase.com

ukd1 7y ago

Long time user and follower of Sia. It's always surprising to me that it's overlooked compared to filecoin (which is similar, but seems like $250m+ vaporware to date). I use it to have tertiary backups. Weird caveat I found: the smallest chargeable size is actually pretty large.

xur17 7y ago
> Weird caveat I found: the smallest chargeable size is actually pretty large. This and the cost to form contracts is is large enough that it doesn't make economical sense until you are storing > $1gb. That said, they seem to slowly be working through a number of these concerns. They very recently released seed based backups, one of the biggest issues I had last time I looked.
- ukd1 7y ago
  Ya; that's another one - the same wallet on multiple machines = different set of contracts. Didn't notice that was fixed though...
  - zherbert 7y ago
    We do not yet have the ability to share the same set of contracts across multiple machines, but it is on the roadmap. This would include using the same set of contracts on desktop and mobile devices, which will open up some intriguing use-cases.

neiman 7y ago

I tried Sia about 2 years ago. It worked "ok" for half a GB files, However, for 4-5 GB my upload got stuck while the help forum couldn't provide answers besides "wait for the next release". There were also unclear issues back then. Are the files encrypted on my computer before being uploaded? are they sharded between many hosts? or are they stored in one host? How long will they be stored. Most of these issues seem like they could be improved. Not an inherent faults of the technology or something. I like Sia's idea and tech in general; I'll give it a try again now.

Taek 7y ago
Sia has come a long way in the past two years. Uploads, downloads, and repairs are both much more stable and also a lot faster. Sia can effortlessly sustain 100mbps upload and download if your connection is that fast. Sia is happy with files that are hundreds of GBs in size today, and with a filesystem that that gets up to about 20 TB. Beyond that it starts to struggle, but we're continuously working to expand its capabilities. Data has always been encrypted client-side, even in very early releases. Data today is sharded between hosts in a 10-of-30 Reed-Solomon scheme. I think you will find that the experience is much smoother and more complete compared to the experience from 2 years ago. And, a critical feature is now available as of the latest release: seed based file recovery. After you upload your files, it's possible to create a snapshot of your data that you can recover later on a different machine using nothing more than your wallet seed. This makes Sia a practical and cost effective solution for secondary or tertiary backup.

nisten 7y ago

There is one missing piece to all this, I/O. Given they publish their apps on all platforms they could probably add this in the future as an ssd or PCIe storage tier by doing regular disk benchmarks. However it seems like a major limiting factor, given that random access speeds on spinning drives or even cheap ssds can drop drastically by an order or two of magnitude (i.e. from 130MB/s to 2MB/s). Given how they're doing error correction in a distributed manner, having a fast storage tier would be a major advantage for them which would eliminate any problems from flash storage wearing out and fully gain the benefit of throughput, random access times and cheap new prices of nvme drives. At the moment I probably can't use this for streaming my own 4k videos to myself because an up to 300mbps network is not good enough for that, the slowest storage in it would be a bottleneck for a significant amounts of the experience of watching the streamed file. And I definitely can't use it for application storage given how important response time is to the user experience and user growth. That all aside, those 2 are relatively extreme examples and there's a lot of usage scenarios in between them. I do admire how far they've gotten this given all the blockchain hyped products that have failed. They seem to have a solid system here that will only improve on performance and put pressure on bringing online storage prices down.

Taek 7y ago
Most 4K video is closer to 50mbps. If your home connection is that fast, Sia should be more than capable of delivering smooth 4K streaming to your home. A single host with slow storage is not a blocking factor because Sia downloads in parallel from as many as 30 hosts at once when composing a stream of data. Even if each has an I/O bottleneck of 1 MB/s (8mbps), you'll be able to get your video out at 240mbps. In practice today, Sia hosts are almost all network bottlenecked, most use HDDs which do not have trouble keeping up with network latencies (HDD seek times are 11ms, network latencies usually 50ms+) or throughputs (HDDs can usually get close to 100 MB/s, far more than the network throughput) Can I ask, have you tried streaming 4K video from the latest version of Sia? Seek times might be 5-10 seconds, but I believe that once your stream has loaded you will see completely smooth playback. Our next release has a focus on reducing those seek times, but the code is not ready yet.
- 7y ago

jchw 7y ago

> Today it is possible to purchase a refurbished 1 TB hard drive for about $20 — so buyers are essentially paying Amazon each month the entire cost of the hard drives used to store their data! What a refurbished hard disk does not give you: - Redundant storage - Redundant power/network/etc. - Geographic distribution - A network API - Any networking, or uplinks at all - Access control - A team ensuring the reliability and security of your data - Global network of datacenters and pops - Integration with other cloud services All that and Amazon et al. do not store your 1TB of data in a single refurbished hard disk. This is beyond non-sense. Sure, some of these costs are amortized, but I still think this is a lame comparison. If all you wanted was at-cost block storage with crappy reliability and speed, you could probably accomplish that in easier ways.

Taek 7y ago
One of the main innovations of Sia is providing a low cost means to turn that 1 TB refurbished hard drive into a useful cloud product. Sia is able to provide, at 1/10th the cost of Amazon, high throughput, high reliability, high redundancy, geographically distributed cloud storage, and it can do that without high reliability drives or datacenters.
leetbulb 7y ago
Off topic, but: I purchased three refurb 6TB HGST He HDDs in Jan. from a reputable shop. All three failed last month. I was so happy that the shop completely refunded all of them because the warranty is generally very limited on refurb HDDs. Normal RMA is 5yr on this specific model. There's no way refurb drives are cost effective for these environments.
- AdamJacobMuller 7y ago
  On balance, I purchased 45 refurb 4TB SAS drives around the same time. I had two DOA with 0% failure rate since then. Also, very anecdotal, I know.
  - leetbulb 7y ago
    I'm wondering if the fact mine were He filled refurbs could've been a factor.

siavosh 7y ago

Can someone explain the difference between Sia vs Filecoin/IPFS. The latter seems to be still working on solving some fundamental crypto primitives before they can offer what seems like what Sia already claims to have solved?

wmf 7y ago
Yes, Filecoin is trying to come up with a "more efficient" and possibly "elegant" implementation based on algorithms that are still being invented. Of course, with 100x the budget it's possible that they could still overtake Sia.

tempsolution 7y ago

This reads like a typical blockchain non-sense hype article. Before you even go down the road of comparing sia to S3, what about some unbiased benchmarks? Show us the 99.999999999% durability, the pretty much unlimited bandwidth, the single digit milli-second latencies, the practically unlimited capacity. If you want to compare Sia to anything, compare it to Dropbox or OneDrive. S3 was made for an entirely different purpose and Sia can't even remotely compete with it. As a personal data store? Maybe. But wait until this thing becomes more popular and hackers start making a mess. Will be interesting to see how stable this "fully decentralized" network is, once it gets the full attention from bad actors as all the cloud providers.

rolltiide 7y ago
coming from someone who is not a fan of Siacoin from when I last used it in 2017, I think you miss the point what I like and want is encrypted distributed data that I can store and retrieve quickly and cheaply I also like an economic model that aligns the hosts to maintain the data, and how the proliferation of that economic model lets me invest in its growth from merely being a passive non-productive speculator, and finally I would like a clear easy to understand ROI from being a productive host: can I subsidize the cost of my hardware that I'm already interested in having. any cryptocurrency can do that, but how the software uses the cryptocurrency is a key factor, and unfortunately we can like an idea and team but get forced to accept an economic model that has flaws so for me, it is easy for me to understand why teams want to promise the sky against S3, and it really isn't that relevant to using this securely. like, you might be able to subsidize all of your hardware and provide a service without worrying about if the SIA comparison is accurate because its really not relevant. but finally, it turns out that unmonetized and non-cryptocurrency based decentralized cloud storage networks are good enough! so I stick with IPFS. hope that was useful to someone.
- carlosdp 7y ago
  IPFS isn't decentralized cloud storage though. Unless you host your file yourself on your IPFS node, your file is not guaranteed to be available. Hence why Filecoin is a thing they are working on.

zherbert 7y ago

Hi all, author here, just wanted to say I am thrilled to see the HN community discuss decentralized cloud storage in more detail. There are many drawbacks today and narrow use-cases, but our goal is to continue to improve Sia – and over the coming years we will prove that decentralized storage can compete directly with Amazon, Google, and Microsoft. We feel confident that the marketplace dynamics, in particular, will foster competition between hosts on the network and maintain prices that are an order of magnitude cheaper for storage and two orders of magnitude cheaper for bandwidth.

zherbert 7y ago
I would also add – give Sia a try, especially if you have tried it before our v1.4.0/v1.4.1 releases! https://sia.tech/get-started https://sia.tech/get-started
fitzroy 7y ago
Gotta say, the fact that the installation instructions consist primarily of getting around the "unknown / unidentified developer" restriction doesn't inspire confidence.
solidasparagus 7y ago
Pretty arrogant to say you're going to make cloud storage providers obsolete when you haven't even built a competing solution. You can't even share data yet... And what about all the other features that cloud providers offer? You say "[the] Sia software is exponentially improving, and performance and featureset is quickly approaching Amazon S3", but I see absolutely no evidence of even the most basic features of cloud storage providers. Authorization? Regulatory compliance? High-performance bandwidth? High-reliability public API/URLs? Customer support? Private cloud peering? Zero effort integration with many (most?) major data-related open source projects? Versioning? One-click, no-knowledge reliability? How much do you need to grow to hit cloud scale? 1,000,000x? More? And you're already claiming that you're going to make S3 obsolete? Your blog would hold more weight if you weren't so over-the-top, bullshit-level optimistic.
- peterkelly 7y ago

Proven 7y ago

If only there was a use case for it

alexnewman 7y ago

I was greatly skeptical of these types of systems but ipfs proved me wrong. It's fine, when combined with a CDN. I'm interested in what new use cases Sia handles when CDN + IPFS works so well. For instance my blog https://www.wuli.nu https://www.wuli.nu (source code https://github.com/posix4e/blog https://github.com/posix4e/blog)

psKama 7y ago
Can you please clarify which CDN features are you referring to on your blog (can't find any) or anywhere else? Also can you elaborate what do you even mean by "so well"? Sia was started to be built before IPFS was even announced and has hosted PBs of data already as a working product. On the other hand IPFS hasn't even released it's public testnet so it would be great to understand your point. Edit: btw by IPFS I assume you refer to their Sia equivalent product, Filecoin. At least I do, as IPFS cannot be compared to Sia alone.
- alexnewman 7y ago
  I am in no way referring to filecoin. I am talking about IPFS and cloudflare integration. https://blog.cloudflare.com/distributed-web-gateway/ https://blog.cloudflare.com/distributed-web-gateway/
- alexnewman 7y ago
  I actually think filecoin is worthless and IPFS is the only valuable thing that came out of that group

truth_seeker 7y ago

Can somebody from Sia team compare it to Radix DLT and it's mechanism to shard and distribute data ? https://www.radixdlt.com/ https://www.radixdlt.com/

x3haloed 7y ago

Is forums.sia.tech down temporarily or permanently?

2youmich 7y ago

Sia is a future

gridlockd 7y ago

I have a feeling this is an apples to oranges comparison. Can you serve clients directly over HTTP using Sia in a permissioned manner? If not, then it's not usable like S3. If it's not usable like S3, then S3 pricing comparison is meaningless. Also, how correlated is Sia pricing to the overall cryptocurrency market? Will my costs go through the roof when there's another crypto bubble? Would it actually scale? If Sia became popular, would prices go up like Bitcoin transaction prices? Isn't the extremely cheap bandwidth due to spare capacity of average broadband plans? If so, wouldn't providers eventually cut that off? Broadband pricing relies on the fact that most users don't use all of the bandwidth most of the time.

hereme888 7y ago

Just switched to backing my data on Sia. Was previously using Nextcloud, which is great, but more expensive. Took a little time to set up, but renting 200 GB for 10 cents/mo, in a decentralized system that encrypts my data, has great uptime, and doesn't mine my personal or "aggregated" data, is unbeatable. I heard Sia is not great at handling large number of files, so I'm using a backup software to create automated, compressed backups first, and Sia handles those larger files. I'll see how it pans out over time.