6 ms·

Yes, being opens source is motivation, because you can create your own cloud with your own hardware when you need it. And it's additional assurance that you'll

Yes, being opens source is motivation, because you can create your own cloud with your own hardware when you need it. And it's additional assurance that you'll be able to deploy your system even if they go out of business.

edanm 15y ago
Is there anyone, anywhere, who considers that as a plus? Who would actually consider rolling out their own cloud infrastructure?
- notmyname 15y ago
  Actually, many people consider it. Some are simply cautious about hosting their data with a third party. Some are prevented from using a third party for compliance or regulatory reasons. Also, it's generally more cost-effective for extremely large datasets to be self-hosted rather than hosted by a third party.
  - edanm 15y ago
    Aren't the bulk of customers who turn to the cloud rather small operations, who are trying to "outsource" as much of their infrastructure issues as possible? And aren't these customers much more concerned about pricing, rather than possible future growth? Note: I don't mean to ask this sarcastically. I'm actually asking.
    - notmyname 15y ago
      From my experience working with Rackspace Cloud Files, customer sizes are all over the map. Some customers are very small. Some are very large. I know that S3 has a similar variance in customer size. From my experience talking to users (and potential users) of Openstack (http://openstack.org http://openstack.org), there again is variance. Most people are relatively small (a few hundred GB to a few hundred TB). Some are much bigger (several PB). The most exciting thing I heard was that CERN is evaluating Openstack swift (http://swift.openstack.org http://swift.openstack.org) for their storage needs. A researcher from CERN gave a keynote at the last Openstack design summit. CERN generates 25 PB / year and has a 20 year retention policy. They have vast storage needs. The storage needs vary greatly. I've seen that outsourcing infrastructure is great to a point, but the largest users can generally get substantial cost savings by bringing their infrastructure back in house.
- mseebach 15y ago
  The cloud is great for scaling, but once you have a large dataset and more-or-less predictable growth, it could easily become more economic to handle it your self. Using something like Nimbus would make such a migration easier. On the other hand, it's not like the S3 interface is rocket-science. Re-writing your apps file-storage interaction is the least of the effort in a multi-terabyte-migration.
- asharp 15y ago
  A cloud hosting provider.
- spatten 15y ago
  A lot of Canadian companies are unable to use S3 (or EC2) due to the Patriot Act. Sure, they can get around this by using European buckets, but that kind of sucks for latency. I can easily imagine setting up a company using this software with servers in Canada using this software. So yes, it's a plus. (It's not just Canada of course, and it's not just the Patriot act. Gambling companies, for example, can't host in the US)
blackiron 15y ago
The license for server-side code - the one you would use to create your own cloud - is AGPL. Isn't this license too restrictive for business?.
- hugoroy 15y ago
  Why would it be? It's the same as the GPL, only difference is: modified version sources must be available to remote-network-interaction users. I don't see what's restrictive for business.
  - mseebach 15y ago
    If you go and sell somebody a hosting solution based on Nimbus, you'd need to share your source code. What I'm not sure of, is if you build, say, a photo sharing webapp using Nimbus as the storage back-end, does your webapp become AGPL by linking? I'm fairly certain GPL would require this, but, as per the rationale for AGPL, you don't care about that when you run a webapp. Curiously, if Nimbus adopted the exact S3 API instead of "similar to", it would not constitute linking, as it's using a standard interface.
    - justincormack 15y ago
      AGPL uses the same definition of linking as GPL so communication over a network API to a storage backend is not linking. http://www.quora.com/Does-the-AGPL-extend-the-idea-of-linking-to-include-network-access http://www.quora.com/Does-the-AGPL-extend-the-idea-of-linkin...
      - mseebach 15y ago
        Ah, thanks for that. I was under the impression that the bar for linking across the network was rather higher.
JoeAltmaier 15y ago
Its a leap that folks desiring cloud storage will want to host their own some day. I think its a little like "I want an open-source car, so when they switch back to horses, I'm ready!"
- bmelton 15y ago
  It's less like that than it is "When we're operating at a scale that makes it more affordable to own hardware than to lease, then it'd be nice if we could." The other unmentioned benefit is in scratching your own itch. If you want feature X, and Amazon won't give it to you, you can develop it in-house and host it yourself.
- jasongullickson 15y ago
  I think it's more like saying "I want a drivers license, so when I can afford a car I don't have to keep riding the bus."
rarrrrrr 15y ago
(SpiderOak / Nimbus.io cofounder here) In addition to supporting the founders personal ethics about software freedom, we feel an open source backend is important for just the sake of confidence. Some people will want to purchase the minimum of 10 machines and host a Nimbus.io storage cluster themselves (and we are also making our hardware specs open source.) Other cloud storage providers may even do this. We hope a few people will consider the hosted option, paying Nimbus.io $0.06 per GB. In any case, all of these are a win for us. We're already spending money every day to maintain a reliable storage backend for our encrypted Backup & Sync business at SpiderOak.com. Nimbus.io is an evolution from that. Community involvement here is most welcome. :) Aside from that, it's just a design we are excited to share. Every other distributed storage system I could find uses replication instead of parity. A system based on parity sacrifices latency but can deliver higher throughput on individual requests (at about 1/3 the cost.) There are use cases even outside of archival storage where this is attractive.
- hemancuso 15y ago
  I don't see how a parity based implementation can work in a meaningful way across multiple datacenters. You certainly couldn't rebuild if you lost an entire datacenter due to disaster. Replication is the only way here. So any comparison to S3 in that regard is meaningless - Nimbus can't achieve that level of durability, correct? Additionally, if you're just doing parity across multiple chassis in a single datacenter and lost a couple racks do to a power outage it would seem the network would likely shit the bed trying to rebuild, potentially bringing the whole system down. Have you guys worked through nastier failure cases that architectures like S3 can avoid?
  - rarrrrrr 15y ago
    Excellent points. Geographic redundancy with parity compliments the network topology we find in many cities: a metro area fiber ring connecting many data centers with low cost site-to-site (not internet) bandwidth. It's even lower cost to just buy excess capacity with lower QOS. Every archival storage provider I've talked to has a write-heavy workload. Write traffic maybe more than 3x read traffic. So for example in this situation replicating between two sites requires a site-to-site connection equal to the size of the incoming data. Since site-to-site connections are full-duplex, in the parity system the bandwidth for reads and writes is provided at a similar price to what would be spent on replication bandwidth for writes. That said, the first iterations of Nimbus.io won't provide geo redundancy beyond the geo-redundancy that creating an offsite backup inherently provides. We expect to add on geo redundancy storage as an upgrade option at a slightly higher price (still way under S3.) Replying to your second point: If transient conditions like only a couple racks lost power, the system wouldn't trigger an automatic rebuild right away. It would continue to service requests with parity and hinted-handoff until the machines come back online. In any case, when the system decides a full rebuild is needed, the rebuild rate is balanced with servicing new requests (similar to how a RAID controller can give tunable priority to rebuild vs. traffic.)
methodin 15y ago
Isn't that the whole point of S3 - you don't need to make your own cloud with your own hardware?

edanm 15y ago

Is there anyone, anywhere, who considers that as a plus? Who would actually consider rolling out their own cloud infrastructure?

notmyname 15y ago
Actually, many people consider it. Some are simply cautious about hosting their data with a third party. Some are prevented from using a third party for compliance or regulatory reasons. Also, it's generally more cost-effective for extremely large datasets to be self-hosted rather than hosted by a third party.
- edanm 15y ago
  Aren't the bulk of customers who turn to the cloud rather small operations, who are trying to "outsource" as much of their infrastructure issues as possible? And aren't these customers much more concerned about pricing, rather than possible future growth? Note: I don't mean to ask this sarcastically. I'm actually asking.
  - notmyname 15y ago
    From my experience working with Rackspace Cloud Files, customer sizes are all over the map. Some customers are very small. Some are very large. I know that S3 has a similar variance in customer size. From my experience talking to users (and potential users) of Openstack (http://openstack.org http://openstack.org), there again is variance. Most people are relatively small (a few hundred GB to a few hundred TB). Some are much bigger (several PB). The most exciting thing I heard was that CERN is evaluating Openstack swift (http://swift.openstack.org http://swift.openstack.org) for their storage needs. A researcher from CERN gave a keynote at the last Openstack design summit. CERN generates 25 PB / year and has a 20 year retention policy. They have vast storage needs. The storage needs vary greatly. I've seen that outsourcing infrastructure is great to a point, but the largest users can generally get substantial cost savings by bringing their infrastructure back in house.
mseebach 15y ago
The cloud is great for scaling, but once you have a large dataset and more-or-less predictable growth, it could easily become more economic to handle it your self. Using something like Nimbus would make such a migration easier. On the other hand, it's not like the S3 interface is rocket-science. Re-writing your apps file-storage interaction is the least of the effort in a multi-terabyte-migration.
asharp 15y ago
A cloud hosting provider.
spatten 15y ago
A lot of Canadian companies are unable to use S3 (or EC2) due to the Patriot Act. Sure, they can get around this by using European buckets, but that kind of sucks for latency. I can easily imagine setting up a company using this software with servers in Canada using this software. So yes, it's a plus. (It's not just Canada of course, and it's not just the Patriot act. Gambling companies, for example, can't host in the US)

blackiron 15y ago

The license for server-side code - the one you would use to create your own cloud - is AGPL. Isn't this license too restrictive for business?.

hugoroy 15y ago
Why would it be? It's the same as the GPL, only difference is: modified version sources must be available to remote-network-interaction users. I don't see what's restrictive for business.
- mseebach 15y ago
  If you go and sell somebody a hosting solution based on Nimbus, you'd need to share your source code. What I'm not sure of, is if you build, say, a photo sharing webapp using Nimbus as the storage back-end, does your webapp become AGPL by linking? I'm fairly certain GPL would require this, but, as per the rationale for AGPL, you don't care about that when you run a webapp. Curiously, if Nimbus adopted the exact S3 API instead of "similar to", it would not constitute linking, as it's using a standard interface.
  - justincormack 15y ago
    AGPL uses the same definition of linking as GPL so communication over a network API to a storage backend is not linking. http://www.quora.com/Does-the-AGPL-extend-the-idea-of-linking-to-include-network-access http://www.quora.com/Does-the-AGPL-extend-the-idea-of-linkin...
    - mseebach 15y ago
      Ah, thanks for that. I was under the impression that the bar for linking across the network was rather higher.

JoeAltmaier 15y ago

Its a leap that folks desiring cloud storage will want to host their own some day. I think its a little like "I want an open-source car, so when they switch back to horses, I'm ready!"

bmelton 15y ago
It's less like that than it is "When we're operating at a scale that makes it more affordable to own hardware than to lease, then it'd be nice if we could." The other unmentioned benefit is in scratching your own itch. If you want feature X, and Amazon won't give it to you, you can develop it in-house and host it yourself.
jasongullickson 15y ago
I think it's more like saying "I want a drivers license, so when I can afford a car I don't have to keep riding the bus."

rarrrrrr 15y ago

(SpiderOak / Nimbus.io cofounder here) In addition to supporting the founders personal ethics about software freedom, we feel an open source backend is important for just the sake of confidence. Some people will want to purchase the minimum of 10 machines and host a Nimbus.io storage cluster themselves (and we are also making our hardware specs open source.) Other cloud storage providers may even do this. We hope a few people will consider the hosted option, paying Nimbus.io $0.06 per GB. In any case, all of these are a win for us. We're already spending money every day to maintain a reliable storage backend for our encrypted Backup & Sync business at SpiderOak.com. Nimbus.io is an evolution from that. Community involvement here is most welcome. :) Aside from that, it's just a design we are excited to share. Every other distributed storage system I could find uses replication instead of parity. A system based on parity sacrifices latency but can deliver higher throughput on individual requests (at about 1/3 the cost.) There are use cases even outside of archival storage where this is attractive.

hemancuso 15y ago
I don't see how a parity based implementation can work in a meaningful way across multiple datacenters. You certainly couldn't rebuild if you lost an entire datacenter due to disaster. Replication is the only way here. So any comparison to S3 in that regard is meaningless - Nimbus can't achieve that level of durability, correct? Additionally, if you're just doing parity across multiple chassis in a single datacenter and lost a couple racks do to a power outage it would seem the network would likely shit the bed trying to rebuild, potentially bringing the whole system down. Have you guys worked through nastier failure cases that architectures like S3 can avoid?
- rarrrrrr 15y ago
  Excellent points. Geographic redundancy with parity compliments the network topology we find in many cities: a metro area fiber ring connecting many data centers with low cost site-to-site (not internet) bandwidth. It's even lower cost to just buy excess capacity with lower QOS. Every archival storage provider I've talked to has a write-heavy workload. Write traffic maybe more than 3x read traffic. So for example in this situation replicating between two sites requires a site-to-site connection equal to the size of the incoming data. Since site-to-site connections are full-duplex, in the parity system the bandwidth for reads and writes is provided at a similar price to what would be spent on replication bandwidth for writes. That said, the first iterations of Nimbus.io won't provide geo redundancy beyond the geo-redundancy that creating an offsite backup inherently provides. We expect to add on geo redundancy storage as an upgrade option at a slightly higher price (still way under S3.) Replying to your second point: If transient conditions like only a couple racks lost power, the system wouldn't trigger an automatic rebuild right away. It would continue to service requests with parity and hinted-handoff until the machines come back online. In any case, when the system decides a full rebuild is needed, the rebuild rate is balanced with servicing new requests (similar to how a RAID controller can give tunable priority to rebuild vs. traffic.)

methodin 15y ago

Isn't that the whole point of S3 - you don't need to make your own cloud with your own hardware?