5 ms·

I'm having difficulty telling if you are being sarcastic here. Small to medium businesses are a huge market, Intuit, Dell and Microsoft have all done very well

by olefoo 18y ago

I'm having difficulty telling if you are being sarcastic here. Small to medium businesses are a huge market, Intuit, Dell and Microsoft have all done very well selling in to it.

SwellJoe 18y ago
It's the Internet. When in doubt, assume sarcasm. The point has already been made that the vast majority of "small to medium businesses" do not need scalability (not even a little bit--90% of those businesses could be hosted on a $10/month shared hosting account and never have a performance problem). So, we're talking about a tiny subset of an extremely cost-conscious market. Not only that, for the small businesses that do need scaling (the Twitters and the Justin.TVs), the vast majority of their scaling problems cannot be solved by adding a few applications like memcache and a pre-tuned MySQL to the distribution. Your application has to be built to scale, or you will not scale. The tuning of MySQL is trivial (and case-by-case specific, to boot...there is no way to say any particular MySQL configuration is good for all high performance use cases that balances all of the usual tradeoffs, and as much as there can be such a configuration the one provided by most distros is already pretty close to it). You could ask me how I know, with some confidence, that web scalabilty for small to medium businesses is a bad business idea, or I could just blurt out that my first business built web scalability products for small to medium businesses. It took me seven years to figure out that they don't have any money, and that they don't know enough to know to look at the niche solutions for their problems (they call Cisco, or F5 if they're really on the technical ball), so when they do spend money, they're spending it with companies that provide a huge range of products that are sort of ancillary to the task at hand...and also happen to have a few items in their product line that can sort of address the problem, if you squint right and send them enough money. Not that I'm bitter. (Seriously, I'm actually not bitter. Just amused when folks take the "pick a problem that you have, and chances are others will have it too, and there's your perfect business idea!" to an extreme that doesn't quite mesh with reality.) There may be the seed of a good idea in there somewhere. But building yet another Linux distribution is just dumb as rocks. And I mean that in the nicest possible way.
- olefoo 18y ago
  Yeah, I kind of figured, but I've spent the last three years growing a decidedly unsexy web-app from 1 webserver and two database machines to the small fleet of machines it is today; and if there were an extant distribution that took some of the pain away I would be on top of it. But as you say it's somewhat hard to get people to spend money on server operating systems. The real problem is that there is too much variation for widely usable solution to come to the fore. Basic webhosting has things like webmin or (yech!) Ensim, that package up the standard tasks in nifty web frontend. But managing a heterogeneous network of webservers and databases is still more difficult than it ought to be. Thanks for the considered answer. I don't think anyone was thinking that this was a viable business; more just people wishing there were a solution to the problems they are dealing with.
  - SwellJoe 18y ago
    and if there were an extant distribution that took some of the pain away I would be on top of it The problem isn't that there isn't someone out there that would be happy to sell you "scaling in a box". The problem is that scaling is not a generic problem. You can't sell scaling in a box. Your application has to be built to scale, or it will not scale. Tuning MySQL is trivial. Adding sharding to your application is the hard part, and no automated solution will know how to do that. The Django, RoR, and Catalyst/DBIx folks are all thinking about the problem, and they may be able to do something about it. Replication probably could be made easier. But it's a far less useful solution than it seems on the surface--if there were no latency, you could just use replication. But there is latency, and so you still need application level support to insure you don't ask for data that doesn't exist yet. Basic webhosting has things like webmin or (yech!) Ensim I'm really glad there was no "yech!" for Webmin, as I take that sort of thing personally. (Check my profile for why. And, we are thinking about these sorts of problems, and we're addressing the ones that can logically be addressed in a reasonably generic way.) But managing a heterogeneous network of webservers and databases is still more difficult than it ought to be. We're working on this, too (as are a couple dozen of our competitors). But scaling your application is still your problem to solve. While one could build (and Jamie and I are building) an application to manage multi-server deployments, no product can generically make an app scale. Thanks for the considered answer. I don't think anyone was thinking that this was a viable business; more just people wishing there were a solution to the problems they are dealing with. My feeling is that it's coming at it from way off in the wrong direction (another niche Linux distro...again, just couldn't be a dumber idea), not that there won't be money spent on scaling issues and multiple server related issues. I'll also point out that from where I sit (which I humbly submit is a position of reasonable knowledge of the industry) scaling is not the biggest problem facing hosting providers--not even close. They simply are not being asked to solve scaling problems--us nerds love to talk about scaling. It feels good to think about all those millions of requests that we'll be serving when we can scale up (ah, those magic words!), but in reality, the average website gets a handful of visitors a day and the average webserver could host 250 average websites without breaking a sweat. But, a side effect of all of the work that is going into more efficiently making use of all the machines in a data center is that a few scaling problems will go away or be mitigated. Ramping up new "servers" in a cloud environment becomes trivial enough to do on-demand--when you need ten servers, you ask for them, and when you don't need them anymore, you drop them. Databases are getting more "cloud" oriented. SimpleDB, Hadoop, BigTable, CouchDB, are all looking at the database problem from a functional zero-side effects perspective, and the end result is a system where related queries can be made in parallel and sane results can be expected. So, there are solutions to a lot of these kinds of problems coming...but they aren't what you expect (or even know you want yet) and they aren't drop-in "MySQL now goes ten times faster" solutions (nor are they, "every new MySQL server linearly increases my apps performance and I didn't have to do anything to my app"). Those kinds of silver bullets just do not exist, and won't exist.

by olefoo 18y ago

I'm having difficulty telling if you are being sarcastic here. Small to medium businesses are a huge market, Intuit, Dell and Microsoft have all done very well selling in to it.

SwellJoe 18y ago

It's the Internet. When in doubt, assume sarcasm. The point has already been made that the vast majority of "small to medium businesses" do not need scalability (not even a little bit--90% of those businesses could be hosted on a $10/month shared hosting account and never have a performance problem). So, we're talking about a tiny subset of an extremely cost-conscious market. Not only that, for the small businesses that do need scaling (the Twitters and the Justin.TVs), the vast majority of their scaling problems cannot be solved by adding a few applications like memcache and a pre-tuned MySQL to the distribution. Your application has to be built to scale, or you will not scale. The tuning of MySQL is trivial (and case-by-case specific, to boot...there is no way to say any particular MySQL configuration is good for all high performance use cases that balances all of the usual tradeoffs, and as much as there can be such a configuration the one provided by most distros is already pretty close to it). You could ask me how I know, with some confidence, that web scalabilty for small to medium businesses is a bad business idea, or I could just blurt out that my first business built web scalability products for small to medium businesses. It took me seven years to figure out that they don't have any money, and that they don't know enough to know to look at the niche solutions for their problems (they call Cisco, or F5 if they're really on the technical ball), so when they do spend money, they're spending it with companies that provide a huge range of products that are sort of ancillary to the task at hand...and also happen to have a few items in their product line that can sort of address the problem, if you squint right and send them enough money. Not that I'm bitter. (Seriously, I'm actually not bitter. Just amused when folks take the "pick a problem that you have, and chances are others will have it too, and there's your perfect business idea!" to an extreme that doesn't quite mesh with reality.) There may be the seed of a good idea in there somewhere. But building yet another Linux distribution is just dumb as rocks. And I mean that in the nicest possible way.

olefoo 18y ago
Yeah, I kind of figured, but I've spent the last three years growing a decidedly unsexy web-app from 1 webserver and two database machines to the small fleet of machines it is today; and if there were an extant distribution that took some of the pain away I would be on top of it. But as you say it's somewhat hard to get people to spend money on server operating systems. The real problem is that there is too much variation for widely usable solution to come to the fore. Basic webhosting has things like webmin or (yech!) Ensim, that package up the standard tasks in nifty web frontend. But managing a heterogeneous network of webservers and databases is still more difficult than it ought to be. Thanks for the considered answer. I don't think anyone was thinking that this was a viable business; more just people wishing there were a solution to the problems they are dealing with.
- SwellJoe 18y ago
  and if there were an extant distribution that took some of the pain away I would be on top of it The problem isn't that there isn't someone out there that would be happy to sell you "scaling in a box". The problem is that scaling is not a generic problem. You can't sell scaling in a box. Your application has to be built to scale, or it will not scale. Tuning MySQL is trivial. Adding sharding to your application is the hard part, and no automated solution will know how to do that. The Django, RoR, and Catalyst/DBIx folks are all thinking about the problem, and they may be able to do something about it. Replication probably could be made easier. But it's a far less useful solution than it seems on the surface--if there were no latency, you could just use replication. But there is latency, and so you still need application level support to insure you don't ask for data that doesn't exist yet. Basic webhosting has things like webmin or (yech!) Ensim I'm really glad there was no "yech!" for Webmin, as I take that sort of thing personally. (Check my profile for why. And, we are thinking about these sorts of problems, and we're addressing the ones that can logically be addressed in a reasonably generic way.) But managing a heterogeneous network of webservers and databases is still more difficult than it ought to be. We're working on this, too (as are a couple dozen of our competitors). But scaling your application is still your problem to solve. While one could build (and Jamie and I are building) an application to manage multi-server deployments, no product can generically make an app scale. Thanks for the considered answer. I don't think anyone was thinking that this was a viable business; more just people wishing there were a solution to the problems they are dealing with. My feeling is that it's coming at it from way off in the wrong direction (another niche Linux distro...again, just couldn't be a dumber idea), not that there won't be money spent on scaling issues and multiple server related issues. I'll also point out that from where I sit (which I humbly submit is a position of reasonable knowledge of the industry) scaling is not the biggest problem facing hosting providers--not even close. They simply are not being asked to solve scaling problems--us nerds love to talk about scaling. It feels good to think about all those millions of requests that we'll be serving when we can scale up (ah, those magic words!), but in reality, the average website gets a handful of visitors a day and the average webserver could host 250 average websites without breaking a sweat. But, a side effect of all of the work that is going into more efficiently making use of all the machines in a data center is that a few scaling problems will go away or be mitigated. Ramping up new "servers" in a cloud environment becomes trivial enough to do on-demand--when you need ten servers, you ask for them, and when you don't need them anymore, you drop them. Databases are getting more "cloud" oriented. SimpleDB, Hadoop, BigTable, CouchDB, are all looking at the database problem from a functional zero-side effects perspective, and the end result is a system where related queries can be made in parallel and sane results can be expected. So, there are solutions to a lot of these kinds of problems coming...but they aren't what you expect (or even know you want yet) and they aren't drop-in "MySQL now goes ten times faster" solutions (nor are they, "every new MySQL server linearly increases my apps performance and I didn't have to do anything to my app"). Those kinds of silver bullets just do not exist, and won't exist.