11 ms·

What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distribut

What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use.

koolba 10y ago
> What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use. Have you tried EFS? https://aws.amazon.com/efs/ https://aws.amazon.com/efs/
- mdekkers 10y ago
  No, since it is stupidly expensive in itself. Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks.
  - koolba 10y ago
    > No, since it is stupidly expensive in itself. EFS is ridiculous cheap if you use it the way it's intended. It's billed per-byte so you only store things that you actually need shared on there. If you're looking for an OCFS2 replacement for storing large volumes of data or to run something like Oracle RAC atop then you're probably better off either doing it yourself. > Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks. Besides the pricing argument, I'd disagree with all of those points. It's a joy to set up once you know what you're doing and performance is more than adequate for all the services I've used (with plenty of knobs to fiddle). Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise.
    - mdekkers 10y ago
      Hey koolba, I apologise in advance for nitpicking, but I'd like to make some counterpoints: It's a joy to set up once you know what you're doing Which is (in my own long and active AWS experience) another way of saying "You need to make a significant investment in time, effort, and tooling to get to a point where it is fast and simple to bring up a reasonably complex environment. You are re-learning domain knowledge you likely already have, just so that you can use Amazon's product. This in-itself isn't a bad thing, depending on what you get back for it, but the return value is "I now know what I am doing with AWS". performance is more than adequate I can get more than adequate pretty much everywhere. I am paying a serious premium, so I really want to be in the amazingly fast bracket, not the adequate. Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise. Yeah, they are, both directly as well as in terms of value for money. There are many ways to get increased performance at lower costs with reduced complexity (complexity also cotst real money). I change hosting provider roughly every 12 months to take advantage of investments in new generation hardware and good deals, and everytime I do so, I check out new features, updated pricing, and do the sums to work out value for money (I offer high performance, managed, HA hosting to selected clients)
      - jronald 10y ago
        You're charging people to manage their environments (providing expertise) and complaining about needing to develop additional skills to do so efficiently. If you have the data with price/performance comparisons, I'd think this is something the rest of us would be interested in.
        mdekkers 10y ago
        You're charging people to manage their environments (providing expertise) and complaining about needing to develop additional skills to do so efficiently. That is a straw man argument. AWS isn't the only way to efficiently manage infrastructure, and it can be argued it isn't even efficient, full stop.
        10y ago
  - gigatexal 10y ago
    Cloud is generally more expensiveness if you're still thinking in a non-cloud mindset. One has to rethink architectures to effectively use cloud providers.
    - mdekkers 10y ago
      I'd love to hear some more specifics, because this sounds like deliciously vague marketing speak. Cost is cost. 100$ isn't all of the sudden 50$ because I have "rethought my architectures". Personally, my mindset isn't cloud or non-cloud. It is about how how to solve an engineering problem in a reliable, maintainable, repeatable, and cost-effective manner.
      - gigatexal 10y ago
        Can you scale up and scale down seamlessly? Can you turn off unused instances? Can you automatically and seamlessly move to the latest hardware running on-prem? Do you want to focus some significant percentage of your time on managing infrastructure or do you want to focus on making what you sell to customers better? I know it sounds like marketing and I know there's a point at which the cost of cloud doesn't make sense, but I do believe that that inflection point is pretty low if you're in a field that is moving quickly or are say a new startup and need to get to market with a minimum-viable-product to get an idea off the ground -- what a shame it would be to fumble that opportunity re-inventing the wheel with infrastructure.
        mdekkers 10y ago
        Can you scale up and scale down seamlessly? Yes Can you turn off unused instances? Yes Can you automatically and seamlessly move to the latest hardware running on-prem? Yes Do you want to focus some significant percentage of your time on managing infrastructure... We spend about 10 minutes per day looking after the stack, with about 4 to 6 hours every 6 weeks looking at if and where we can improve things. We then spend about a week or so in depth once every year roughly evaluating different providers, and we typically end up moving to a different provider, which takes a few days. ...or do you want to focus on making what you sell to customers better? This (infrastructure) is a big part of what we sell to our customers. What we do (and have been doing for about 15 years) is what you kids appear to be calling "devops" now. I was an early adopter (and discarder) of CFEngine...
dcgoss 10y ago
Google Cloud can do this, however the volumes must be mounted in read-only mode.
- mdekkers 10y ago
  While that is an interesting proposition, it isn't what I am looking for.
  - Nacraile 10y ago
    Unfortunately, what you're looking for probably isn't possible. You can't just attach multiple writers to a block device, slap a standard filesystem on top, and expect it to work. Standard filesystem implementations are built on the assumption that they have exclusive access to the underlying block device. The block device interface provides no primitives suitable for concurrency control (no atomic compare-and-set, much less anything more sophisticated). Even if they did, you'd have to pretty much re-write the filesystem to make proper use of it. Even read-only multiple mounts are dodgy: the RO mounts aren't going to do cache coherence, and could trip up hard over intermediate states from the writer, but at least they won't corrupt the filesystem. Realistically, if you want multiple writers on a network block device, your application is going to have to go to the raw block device, and you get to figure out intra-client co-ordination on your own. This isn't actually terribly useful. As an ex-insider, the nonexistence of this feature is not due to technical difficulty (it should be pretty easy for EBS to let you have multiple writable attachments), it's because 0.1% of customers would find the feature actually useful, and the other 99.9% would use it to shoot themselves in the foot. If you want multiple machines accessing the same filesystem, you want a network attached filesystem. You can easily put samba / nfs-kernel-server / windows server in front of EBS. There are a few distributed filesystems, FOSS and propriety, if you need more scale. If you don't like any of the above, you're gonna have to build what you like yourself, or wait for somebody else to decide to build it for you.
    - mdekkers 10y ago
      Unfortunately, what you're looking for probably isn't possible. You can't just attach multiple writers to a block device, slap a standard filesystem on top, and expect it to work. Standard filesystem implementations are built on the assumption that they have exclusive access to the underlying block device. Yes, it is possible, and even in some cases (mine being one of them) desirable. I mentioned OCFS2, which is designed specifically for doing exactly that. When you do this right, it is an extremely high performance, simple way of sharing storage in a highly available manner.
nnx 10y ago
GCE supports mounting one volume as read-only from multiple instances (only one instance can write to the volume).
khc 10y ago
What's your workload like?
- mdekkers 10y ago
  workload in which terms? :)

koolba 10y ago

> What I would really like to see (on any cloud, really) is volumes that can be bound to multiple nodes/images/vm's/whatever. I have yet to find a solid distributed FS alternative to OCFS2 that actually performs in the same league and ease of use. Have you tried EFS? https://aws.amazon.com/efs/ https://aws.amazon.com/efs/

mdekkers 10y ago
No, since it is stupidly expensive in itself. Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks.
- koolba 10y ago
  > No, since it is stupidly expensive in itself. EFS is ridiculous cheap if you use it the way it's intended. It's billed per-byte so you only store things that you actually need shared on there. If you're looking for an OCFS2 replacement for storing large volumes of data or to run something like Oracle RAC atop then you're probably better off either doing it yourself. > Moreover, AWS services in general are ridiculously expensive for what has always turned out to be a sub-par experience. Overly complicated to set up and manage, lacklustre performance, and premium pricing. Thanks, but no thanks. Besides the pricing argument, I'd disagree with all of those points. It's a joy to set up once you know what you're doing and performance is more than adequate for all the services I've used (with plenty of knobs to fiddle). Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise.
  - mdekkers 10y ago
    Hey koolba, I apologise in advance for nitpicking, but I'd like to make some counterpoints: It's a joy to set up once you know what you're doing Which is (in my own long and active AWS experience) another way of saying "You need to make a significant investment in time, effort, and tooling to get to a point where it is fast and simple to bring up a reasonably complex environment. You are re-learning domain knowledge you likely already have, just so that you can use Amazon's product. This in-itself isn't a bad thing, depending on what you get back for it, but the return value is "I now know what I am doing with AWS". performance is more than adequate I can get more than adequate pretty much everywhere. I am paying a serious premium, so I really want to be in the amazingly fast bracket, not the adequate. Outside of external bandwidth pricing I don't even consider it that far out of line pricing wise. Yeah, they are, both directly as well as in terms of value for money. There are many ways to get increased performance at lower costs with reduced complexity (complexity also cotst real money). I change hosting provider roughly every 12 months to take advantage of investments in new generation hardware and good deals, and everytime I do so, I check out new features, updated pricing, and do the sums to work out value for money (I offer high performance, managed, HA hosting to selected clients)
    - jronald 10y ago
      You're charging people to manage their environments (providing expertise) and complaining about needing to develop additional skills to do so efficiently. If you have the data with price/performance comparisons, I'd think this is something the rest of us would be interested in.
      - mdekkers 10y ago
        You're charging people to manage their environments (providing expertise) and complaining about needing to develop additional skills to do so efficiently. That is a straw man argument. AWS isn't the only way to efficiently manage infrastructure, and it can be argued it isn't even efficient, full stop.
        10y ago
- gigatexal 10y ago
  Cloud is generally more expensiveness if you're still thinking in a non-cloud mindset. One has to rethink architectures to effectively use cloud providers.
  - mdekkers 10y ago
    I'd love to hear some more specifics, because this sounds like deliciously vague marketing speak. Cost is cost. 100$ isn't all of the sudden 50$ because I have "rethought my architectures". Personally, my mindset isn't cloud or non-cloud. It is about how how to solve an engineering problem in a reliable, maintainable, repeatable, and cost-effective manner.
    - gigatexal 10y ago
      Can you scale up and scale down seamlessly? Can you turn off unused instances? Can you automatically and seamlessly move to the latest hardware running on-prem? Do you want to focus some significant percentage of your time on managing infrastructure or do you want to focus on making what you sell to customers better? I know it sounds like marketing and I know there's a point at which the cost of cloud doesn't make sense, but I do believe that that inflection point is pretty low if you're in a field that is moving quickly or are say a new startup and need to get to market with a minimum-viable-product to get an idea off the ground -- what a shame it would be to fumble that opportunity re-inventing the wheel with infrastructure.
      - mdekkers 10y ago
        Can you scale up and scale down seamlessly? Yes Can you turn off unused instances? Yes Can you automatically and seamlessly move to the latest hardware running on-prem? Yes Do you want to focus some significant percentage of your time on managing infrastructure... We spend about 10 minutes per day looking after the stack, with about 4 to 6 hours every 6 weeks looking at if and where we can improve things. We then spend about a week or so in depth once every year roughly evaluating different providers, and we typically end up moving to a different provider, which takes a few days. ...or do you want to focus on making what you sell to customers better? This (infrastructure) is a big part of what we sell to our customers. What we do (and have been doing for about 15 years) is what you kids appear to be calling "devops" now. I was an early adopter (and discarder) of CFEngine...

dcgoss 10y ago

Google Cloud can do this, however the volumes must be mounted in read-only mode.

mdekkers 10y ago
While that is an interesting proposition, it isn't what I am looking for.
- Nacraile 10y ago
  Unfortunately, what you're looking for probably isn't possible. You can't just attach multiple writers to a block device, slap a standard filesystem on top, and expect it to work. Standard filesystem implementations are built on the assumption that they have exclusive access to the underlying block device. The block device interface provides no primitives suitable for concurrency control (no atomic compare-and-set, much less anything more sophisticated). Even if they did, you'd have to pretty much re-write the filesystem to make proper use of it. Even read-only multiple mounts are dodgy: the RO mounts aren't going to do cache coherence, and could trip up hard over intermediate states from the writer, but at least they won't corrupt the filesystem. Realistically, if you want multiple writers on a network block device, your application is going to have to go to the raw block device, and you get to figure out intra-client co-ordination on your own. This isn't actually terribly useful. As an ex-insider, the nonexistence of this feature is not due to technical difficulty (it should be pretty easy for EBS to let you have multiple writable attachments), it's because 0.1% of customers would find the feature actually useful, and the other 99.9% would use it to shoot themselves in the foot. If you want multiple machines accessing the same filesystem, you want a network attached filesystem. You can easily put samba / nfs-kernel-server / windows server in front of EBS. There are a few distributed filesystems, FOSS and propriety, if you need more scale. If you don't like any of the above, you're gonna have to build what you like yourself, or wait for somebody else to decide to build it for you.
  - mdekkers 10y ago
    Unfortunately, what you're looking for probably isn't possible. You can't just attach multiple writers to a block device, slap a standard filesystem on top, and expect it to work. Standard filesystem implementations are built on the assumption that they have exclusive access to the underlying block device. Yes, it is possible, and even in some cases (mine being one of them) desirable. I mentioned OCFS2, which is designed specifically for doing exactly that. When you do this right, it is an extremely high performance, simple way of sharing storage in a highly available manner.

nnx 10y ago

GCE supports mounting one volume as read-only from multiple instances (only one instance can write to the volume).

khc 10y ago

What's your workload like?

mdekkers 10y ago
workload in which terms? :)