4 ms·

The standing joke amongst my co-workers is that ”enterprise” in this context is the tier you reach when you have gotten ripped of enough by third party vendors.

The standing joke amongst my co-workers is that ”enterprise” in this context is the tier you reach when you have gotten ripped of enough by third party vendors. You usual run of the mill enterprise use most of this horsepower to heat air and have large clusters of oracle db’s power even more expensive SAP modules. There’s a reason no serious iaas provider build infra this way. I have one example, actually an old customer to the vendor that used to employ me, that had probably well over 1k of tennants all with VM’s and DB’s backed by humongous SAN (replicated ofc!). Not nice when corrupt data was mirrored due to a software bug in said SAN system. This went to a national level, really, due to the nature of many of the customers. Sure you can build a reasonable architecture using the products and vendors mentioned, but, it doesn’t scale and it really locks you in. I’ve seen and worked this stuff at numerous enterprises. This only makes it my experience and one datapoint, so please bear that in mind! =D

a11595 8y ago
they don't build it this way because cloud is cheap and slow. I have a UCS farm with 2000VMs, connected to 3PB of usable all-flash across 2 arrays, replicated active/active to another array, second-hopped to cloud. That's one site. About 100k users across 2 domains - 1k? I said enterprise. A SAN is not scalable but internal disk is? You need to look up what a SAN is. The reason there is a standing joke with your coworkers is because you don't work on important things that run the world. When 1 minute of downtime costs you over $1mil, your "ripped off" Oracle costs, your SAN costs, etc, are lost in the rounding errors. I can have a SAN have hundreds of arrays, across multiple datacenters, and dynamically grow and shrink my storage needs. It is the definition of scalable. I can have one VM farm vmotion to another VM farm 30 miles away transparently, which will sit on a different array attached to the SAN. What happens when the storage needs of your server outgrow the drives you can shove in there? There are servers and databases a petabyte in size, pushing a million IOPS. They're in charge of money. If there is corruption on the SAN, well it infrequently happens, just like with servers and memory. Twice in my vast experience. You can roll back all writes on the replication software, and usually keep an undo journal a few days long, and something like hourly snapshots. This also protects against cryptoviruses and other types of corruption. This is why people like you work at small companies, fiddling with your cute little projects, while the world moves forward with you on the sidelines. I've been doing this for 20+ years, and have been at most fortune100 companies, in 80 countries. But yeah, your opinion, while being dismissive, is cute. The CIO is the guy in charge of getting this stuff at enterprises. It's clear you don't understand the impact of design. I am guessing you are not at an architect level. There's a reason for that, and a reason other people - the ones you put down in your post, are making the big decisions. People like you would cost the company millions of dollars in loss per year.
- jordanbeiber 8y ago
  I said tenants, not users. Tiny little detail that you overlooked there. =) Last company I worked for had a €150.000.000 tech budget and about ~90k employees. Again, if you read what I said you'll notice that I pointed out that you actually can build decent architectures using enterprisey stuff, it will however never scale horizontally in a resonable way. What happens when you have filled all your shelves in the array box? Ah, you'll need a new $250K box... Mentioning vmotion and all... yeah so... VSAN et al... brrr... I'll put my trust in the open source world rather than shoddy software from 3rd party vendors trying to appease to the latest fad. Hyperconverge my *ss! =) KISS is what rules but vendors are busy feeding channels and partners with impossible to penetrate acronyms leaving you in a mess sooner or later. Focus from these vendors is to keep integration at a minimum, which means API's are usually crappy and achieving a reasonable level of automation is often a chore. Now there is probably a little percentage of "enterprises" that uses tech in a sane way, but my bet is that the CIO you mention is a blockchain expert, as well as putting AI at the top of strategic actions to "implement" this year. He probably have commissioned a pre-study from Accenture that outlines these strategic imperatives. This is of course a little rant, but also true for the 90% of the 90% mentioned. Don't be defensive and close minded. That's what's leaving most enterprises in the dust. There's just a lot of inertia inherent within certain businesses that will let them continue to burn through cash on pointless tech for yet a while.
  - a11595 8y ago
    again, cute. $250K. very cute. Try $5mil for a box, at least, after a 50% vendor discount. VSAN has nothing to do with SAN, and no one on a SAN uses VSAN. You are again showing your lack of enterprise experience, yet you are strongly trashing what you don't understand. A server sees many arrays, over a SAN. That is called horizontal scaling. You can't do that with internal disk.
    - jordanbeiber 8y ago
      Oh, 250 was for the box, no disks. 75% discount. :) Keep it old-school! Vsan btw is vmwares attempt to actually achieve _horizontal_ scalability. I just don’t trust vmware with anything except for that bytecode vm. It used to rock, but time went ahead, and even as they IPO’d I thought they would end up dead in the water eventually. I had just been introduced to the wonderful zones in Solaris and it just made hypervisor vm’s seem silly. And here we are with cgroups and friends... Have a look at ceph if you’re interested: http://docs.ceph.com/docs/master/architecture/ http://docs.ceph.com/docs/master/architecture/ Anyway, I’m out. If your horizon extends to what vendors are prepared to sell you, take that 50 discount and run with it! One final thought: is that disk not ”internal” to the array? This is what makes blockstorage notoriously difficult to scale horizontally. You’ll need very clever software! GL&HF

a11595 8y ago

they don't build it this way because cloud is cheap and slow. I have a UCS farm with 2000VMs, connected to 3PB of usable all-flash across 2 arrays, replicated active/active to another array, second-hopped to cloud. That's one site. About 100k users across 2 domains - 1k? I said enterprise. A SAN is not scalable but internal disk is? You need to look up what a SAN is. The reason there is a standing joke with your coworkers is because you don't work on important things that run the world. When 1 minute of downtime costs you over $1mil, your "ripped off" Oracle costs, your SAN costs, etc, are lost in the rounding errors. I can have a SAN have hundreds of arrays, across multiple datacenters, and dynamically grow and shrink my storage needs. It is the definition of scalable. I can have one VM farm vmotion to another VM farm 30 miles away transparently, which will sit on a different array attached to the SAN. What happens when the storage needs of your server outgrow the drives you can shove in there? There are servers and databases a petabyte in size, pushing a million IOPS. They're in charge of money. If there is corruption on the SAN, well it infrequently happens, just like with servers and memory. Twice in my vast experience. You can roll back all writes on the replication software, and usually keep an undo journal a few days long, and something like hourly snapshots. This also protects against cryptoviruses and other types of corruption. This is why people like you work at small companies, fiddling with your cute little projects, while the world moves forward with you on the sidelines. I've been doing this for 20+ years, and have been at most fortune100 companies, in 80 countries. But yeah, your opinion, while being dismissive, is cute. The CIO is the guy in charge of getting this stuff at enterprises. It's clear you don't understand the impact of design. I am guessing you are not at an architect level. There's a reason for that, and a reason other people - the ones you put down in your post, are making the big decisions. People like you would cost the company millions of dollars in loss per year.

jordanbeiber 8y ago
I said tenants, not users. Tiny little detail that you overlooked there. =) Last company I worked for had a €150.000.000 tech budget and about ~90k employees. Again, if you read what I said you'll notice that I pointed out that you actually can build decent architectures using enterprisey stuff, it will however never scale horizontally in a resonable way. What happens when you have filled all your shelves in the array box? Ah, you'll need a new $250K box... Mentioning vmotion and all... yeah so... VSAN et al... brrr... I'll put my trust in the open source world rather than shoddy software from 3rd party vendors trying to appease to the latest fad. Hyperconverge my *ss! =) KISS is what rules but vendors are busy feeding channels and partners with impossible to penetrate acronyms leaving you in a mess sooner or later. Focus from these vendors is to keep integration at a minimum, which means API's are usually crappy and achieving a reasonable level of automation is often a chore. Now there is probably a little percentage of "enterprises" that uses tech in a sane way, but my bet is that the CIO you mention is a blockchain expert, as well as putting AI at the top of strategic actions to "implement" this year. He probably have commissioned a pre-study from Accenture that outlines these strategic imperatives. This is of course a little rant, but also true for the 90% of the 90% mentioned. Don't be defensive and close minded. That's what's leaving most enterprises in the dust. There's just a lot of inertia inherent within certain businesses that will let them continue to burn through cash on pointless tech for yet a while.
- a11595 8y ago
  again, cute. $250K. very cute. Try $5mil for a box, at least, after a 50% vendor discount. VSAN has nothing to do with SAN, and no one on a SAN uses VSAN. You are again showing your lack of enterprise experience, yet you are strongly trashing what you don't understand. A server sees many arrays, over a SAN. That is called horizontal scaling. You can't do that with internal disk.
  - jordanbeiber 8y ago
    Oh, 250 was for the box, no disks. 75% discount. :) Keep it old-school! Vsan btw is vmwares attempt to actually achieve _horizontal_ scalability. I just don’t trust vmware with anything except for that bytecode vm. It used to rock, but time went ahead, and even as they IPO’d I thought they would end up dead in the water eventually. I had just been introduced to the wonderful zones in Solaris and it just made hypervisor vm’s seem silly. And here we are with cgroups and friends... Have a look at ceph if you’re interested: http://docs.ceph.com/docs/master/architecture/ http://docs.ceph.com/docs/master/architecture/ Anyway, I’m out. If your horizon extends to what vendors are prepared to sell you, take that 50 discount and run with it! One final thought: is that disk not ”internal” to the array? This is what makes blockstorage notoriously difficult to scale horizontally. You’ll need very clever software! GL&HF