3 ms·

> I don't quite understand that statement. I didn't make a single statement, but I also made what is more of an accusation (i.e. stating the opinion of others

by mmt 8y ago

> I don't quite understand that statement. I didn't make a single statement, but I also made what is more of an accusation (i.e. stating the opinion of others I disagree with) than my own assertion. > Personally I like spending them on the bits of the software that give you the USP, Not something thats already solved. I'm suggesting that most people (or at least in my own "bubble", as it were) don't differentiate. Re-inventing the wheel is just fine to them, so long as it's fun. That said, I'm not convinced large (enough) "global" filesystems are a solved problem. Of course, GitLab doesn't necessarily need such a thing, but implementing that kind of storage "sharding" (for lack of a better word) may well be spending an innovation token, too. > or just host it in the cloud. Except that doesn't magically solve any of the architectural/operational problems. For all we know they're still doing Ceph-with-NFS, just paying much more to do so, without the ability to optimize the hardware for cost and/or performance.

KaiserPro 8y ago
Gotcha. I do agree that, a lot of my time can be taken up with explaining why I don't want to replace a file server with a Cassandra cluster(yes really). I was at one point all for single name space global filesystems. GPFS makes data affinity, HSM and provide rich event hooks. However, its global, which means global outages. It has its user cases though. Depending on your use case sharding a file system can be pretty simple. The root folder can be a directory of dynamic symlinks controlled by config management. However its never as simple as that, as you know.
- mmt 8y ago
  > I do agree that, a lot of my time can be taken up with explaining why I don't want to replace a file server with a Cassandra cluster(yes really). Adding the complexity of a DBMS (and a non-standard one at that) on top of a distributed system seems excessive, but I'd believe it. > GPFS I've never been a strong advocate for proprietary/enterprise solutions, as that's an easy way to spend more than a cloud provider would cost. Since storage (hardware) is the worst offender for markups, I'm extra-sensitive in that area. > However its never as simple as that, as you know. Indeed, which is why I suggested it would require innovation. For any non-trivial case, the app would need to be aware, and it may as well do all that work.

by mmt 8y ago

KaiserPro 8y ago

Gotcha. I do agree that, a lot of my time can be taken up with explaining why I don't want to replace a file server with a Cassandra cluster(yes really). I was at one point all for single name space global filesystems. GPFS makes data affinity, HSM and provide rich event hooks. However, its global, which means global outages. It has its user cases though. Depending on your use case sharding a file system can be pretty simple. The root folder can be a directory of dynamic symlinks controlled by config management. However its never as simple as that, as you know.

mmt 8y ago
> I do agree that, a lot of my time can be taken up with explaining why I don't want to replace a file server with a Cassandra cluster(yes really). Adding the complexity of a DBMS (and a non-standard one at that) on top of a distributed system seems excessive, but I'd believe it. > GPFS I've never been a strong advocate for proprietary/enterprise solutions, as that's an easy way to spend more than a cloud provider would cost. Since storage (hardware) is the worst offender for markups, I'm extra-sensitive in that area. > However its never as simple as that, as you know. Indeed, which is why I suggested it would require innovation. For any non-trivial case, the app would need to be aware, and it may as well do all that work.