11 ms·
Because not all internet is equal.
Most ISP's give home users a significant discount on their service, in exchange for agreeing to certain assumptions about th
Because not all internet is equal.
Most ISP's give home users a significant discount on their service, in exchange for agreeing to certain assumptions about their use.
This is done because most home users could never afford internet, if they had to pay for full price.
For example, in this area, a dedicated 20x20mb connection to the internet costs about $500USD per month (Metro Ethernet). You can do almost anything you want with this connection, since your paying for the full cost. Run servers, re-sell it to others at markup, host hundreds of websites on it, anything that's legal to do in the US, you can do with this connection.
However, most people can't afford $500/month at their house, and most people would like to have faster download speeds than 20mb. These folks might instead buy a Comcast residential plan that offers 50mb down, 10mb up, for roughly $80 (which is what that plan costs here).
$80/month is a lot cheaper than $500/month. The reason it's cheaper is because you agree to use it very lightly (no servers, no commercial use, no re-selling, keep total bandwidth use under 250-500GB, ect).
What some people try to do, is 'cheat' that system. They try to buy a 'home' connection, and run their business-level services on it. ISP's crack down on that, because it's not fair to the others whose service that person is disrupting.
However, some home users have legitimate home uses that appear to be cheats in the system. Running a Minecraft server for your friends is a 'valid' home use, but it sometimes conflicts with an ISP's rule (which might make an assumption that all servers are 'business class only' uses).
- - -
Sometimes it gets trickier -- where there's competition, anyone can buy high speed internet. (Urbanized Cities are usually ok here, lots of people sell Fiber lines / Metro Ethernet / Microwave / ect).
But at your house, there's usually zero competition by law. A provider like Comcast might be the only option you have for high speed internet (perhaps they bought exclusive franchise rights to your city, or perhaps your state or local level government has created a law preventing new providers from competing in your area (this isn't just theoretical, both of those things commonly happen in the USA)
At this point, your usually stuck. The provider was given monopoly status, but isn't being controlled by the regulations needed to ensure it doesn't abuse it's monopoly status. What can you do, besides raise a lot of complaints about that monopoly provider's rules.
- - -
TL/DR : Home users never pay for most of the speeds they get. Therefore, those speeds have to be shared among lots of people. ISP's use rules to make sure everyone shares nicely. By default, those rules are overly harsh. Good ISP's properly loosen the restrictions to allow obviously-good home uses.
I'm not sure what would be preferable. They could sell you metered bandwidth; but everyone always complains about metered bandwidth, and no one wants to accidentally have their 14 year old nephew who comes to visit for a week blow out your caps and cost you tons of money because they left Bittorrent seeding a bunch of popular movies.
They could just sell you lower bitrates. But then everyone would suffer, as they wouldn't get the short bursts of quick bandwidth they need to browse the web or download files.
Or they could do what they're doing, which is sell under separate residential and business plans. The residential plans are sold under the assumption that you'll be using them for ordinary personal use by you and your family, and that you're not going to build a data center in your basement and start reselling bandwidth to people colocating servers in your basement.
"Business" and "residential" have fairly well established meanings. There are already plenty of zoning laws that make business vs. residential, or residential vs. commercial vs. industrial distinctions. Selling a residential plan that includes in its terms of service "you may not use this for commercial purposes" is not unreasonable.
Now, I do object to residential plans that unilaterally block ports without recourse, or disallow any running of "servers", as there are lots of valid reasons why someone would run a server for personal, as opposed to commercial, purposes. But I can't find a good way to allow everyone to share a larger pool of bandwidth at a lower price for home use, without allowing some people to take advantage of it for profit, without making the business/residential distinction.
And sure, there will always be grey areas. What if you work remotely? What about personal uses that are abusive? These are things that need to be dealt with on a case by case basis, or spelled out clearly in the ToS. You can't really include the entire complexity of the ToS in a single ad, so I'm not sure you can claim false advertising just because there are a few terms in the ToS which weren't unambiguously stated in any "advertising" about the service.
Really the only difference between "residential" and "commercial" plans should be service guarantees. I do not expect Comcast's residential cable service to have any particular guarantee of service; if I had a commercial contract, I would certainly demand some sort of guarantee about available bandwidth and uptime. That is what we have now, more or less.
To put it another way, suppose nobody gets any service guarantee with their residential connection. Why should it be acceptable to say, "How dare you maximize your use of this, when it reduces availability for everyone else?!?" Nobody gets a guarantee of service at all.
Now, if we do get guarantees about our minimum service level, then there is an easy solution, which is to just reduce throughput for "heavy users" to the point where everyone else is getting their minimum guaranteed service. That would not be so bad at all: minimum service guarantees rather than maximum service limits. Of course, once you start making guarantees like that, people start expecting you to follow through and your ability to grossly oversell your network's capacity is constrained (what a tragedy).
So, let's think back to the analog, circuit-switched telephone network. If you have a telephone line, you generally expect to be able to make a call at any time. But for anyone to be able to make a call at any time across the country, you would need to have one cross-country line for each person on the side with lower number of people (assuming, of course, that you divide the country in half and need to run cross country lines between the two halve; bear with me, this is a bit of a simplified example).
Of course, if you built one line for each person, then the vast majority of them would go unused. It would be completely ridiculous to overbuild like this; everyone would be paying for a ton of capacity that's unused. So how about if they build it to comfortably cover 10% more capacity than they've ever had used in the past. This allows for enough headroom for spikes, and when usage starts crossing that line they know that they need to start building out more capacity.
But of course, then some disaster strikes, and everyone tries to call their loved ones at once to check in on them. Usage surges to more than 10% higher than their previous high, and they run out of lines. Oops, now they've violated their "minimum service level". So, should they only have sold you long-distance service when they knew that there was less than one line per endpoint? Well, sure, it would have been ridiculously expensive and you wouldn't have bought it to have your own dedicated line the whole way across the country.
Now, the considerations in a packet switched network aren't all that much different. You can be more efficient about sharing the same resources because you can even share a single line, with several people sending packets at different times. But if you want to provide some kind of guaranteed minimum service, you need to take whatever you most limited path is, and divide up its cost and bandwidth between all customers who could possibly use it simultaneously. And I guarantee you, that will add up to paying a fortune for a pittance of bandwidth.
Why should it be acceptable to say, "How dare you
maximize your use of this, when it reduces availability
for everyone else?!?" Nobody gets a guarantee of
service at all.
Because if they do that, people will start leaving that service, saying "it's slow and glitchy, this other service gets much better ratings on SpeedTest.net and all of my friends on this other service say it's more reliable for them", when the reason for that is "this other service" actually has an enforces a ToS against abusive use of the network.
Does anybody get a "guarantee of service" to the public roads? Then why should anyone complain if someone else decides to run a private parade on those public roads, blocking everyone else? Or should we all get a "guarantee of service", and then deny anyone the right to have a parade ever?
Sometimes, you need to share resources without precise, hard and fast constraints, but instead based on social conventions, definitions of completely egregious behavior, and consequences for that completely egregious behavior.
Well metered connection - the problem is the pricing. 10USD per month for contentedness + $1 per terabyte seems like a good deal. And even if my nephew decided to download all the porn of the internet he will run out of space before I run out of money.
Your post is based on a lot of actual and epistemological assumptions that are either incorrect, or undefended. For example:
> This is done because most home users could never afford internet, if they had to pay for full price.
There is an epistemological problem here about what "full price" means, which is subjective. I would argue that "full price" is simply "whatever AT&T/Comcast charges you". You seem to want there to be some "other" type of price that is the "real price".
Perhaps you are saying that ISPs make more money from business customers than from residential customers. Perhaps this is true, but the fact that it is convenient for the business model of an ISP to charge some customers more does not mean that other customers are not paying "full price".
Perhaps you are saying that residential customers are being provided connections that are below cost. But in an efficient market, an ISP could of course decide not to serve residential customers. Seeing as how ISPs continue to serve residential customers, it must be because residential customers are in fact paying enough money to cover the costs of the services they receive (e.g. "full price").
> The reason it's cheaper is because you agree to use it very lightly
You seem to be suggesting that the primary factor in network costs is use, but all the sources I read (for example this one)[1] suggest that connecting the last mile is expensive, and routing an arbitrary volume of packets over that pre-installed copper is extremely cheap.
If this is true, then it is the residential market that is subsidizing the high-margin business connections, because the sprawling (profitable, predictable) residential networks are the ones that are shortening the last-mile distance to (sparser) business links.
> What some people try to do, is 'cheat' that system.
There is an epistemological assumption in here that "the system" is fair or impartial or acknowledged in general by society or based on objective costs, or something.
> Home users never pay for most of the speeds they get.
I've covered this earlier, but just as an additional argument, I did a cost comparison, and consumer vs business class in my area for my ISP is merely double. Those packages all include website hosting, Exchange, Sharepoint, static IPs, 24x7 support, etc. Just as a point of comparison, market rates for 2GB of Sharepoint hosting are $25/month, hosted Exchange is $20/month, 24x7 support is who knows, etc... So fair to say, very little of the additional revenue could possibly be going to network costs.
[1] http://www.muninetworks.org/content/solving-middle-mile-availability-does-not-solve-last-mile-problems http://www.muninetworks.org/content/solving-middle-mile-avai...
We seem to have gotten off on the wrong foot -- perhaps I've been unclear :
> There is an epistemological problem here about what "full price" means, which is subjective... You seem to want there to be some "other" type of price that is the "real price".
I think I made this pretty clear, but I'll re-state it to try to clarify what I mean : "Real price" is not an 'epistemological problem', it's simply the cost of dedicated internet service. That might come from AT&T or Comcast, but it might also come from US Signal, or Level 3, or Cogent, or Zayo, or any number of other providers.
When you pay for a typical home or 'small business' connection, you are not buying dedicated internet. You are buying shared internet. You've bought a small fraction of a connection. That's why, under times of heavy load, the connection slows down.
"Real price" is the cost of a connection that "never" slows down, because it is literally dedicated to you and only you.
For example, when you buy a connection from Comcast / Time Warner / Cox / Charter, they might advertise it as "up to 50mbps". Your not buying a solid 50mbps, your buying a tiny portion of that 50mbps line (a portion you share with your neighbors).
The "real price" of a 50mbps connection is much higher, because it's cost can't be spread out / shared among lots of people (since that connection is dedicated to you alone). I'm not inventing this price, it's a figure you get from an upstream provider in your area.
> Perhaps you are saying that ISPs make more money from business customers than from residential customers. Perhaps this is true, but the fact that it is convenient for the business model of an ISP to charge some customers more does not mean that other customers are not paying "full price".
Your twisting my words. I said business typically pay for dedicated internet ("full price"), where as residential users pay for shared internet ("partial price", or a small portion of the full price of that service.
Lots of people will complain they paid for "50x10". But they didn't, they paid for a small shared portion of that 50x10. They paid "partial price" for partial service, but feel entitled to full service. (They bought a shared connection, but feel entitled to a full, dedicated connection).
> You seem to be suggesting that the primary factor in network costs is use, but all the sources I read (for example this one)[1] suggest that connecting the last mile is expensive, and routing an arbitrary volume of packets over that pre-installed copper is extremely cheap.
Your have a few facts correct here, but have come to the wrong conclusion.
You are correct that 'routing arbitrary packets is cheap'. However, there are lots and lots of other costs your ignoring. For instance, installing copper is not cheap. Hooking up that copper to the internet at large, is not cheap. (You have to have internet access, to sell it.) Maintaining copper is not cheap. Simply letting copper sit in the ground, without touching it in any way, is not cheap. (One quick example: Copper is not a fixed expense in the US, you typically have to pay each month for the 'right of way' to even let copper sit in the ground or in the air, untouched. Every single foot is billed. In my city, that cost is 33cents per foot, so to run a cable from two buildings two miles apart, costs $290 per month, just for the right-of-way for that cable to exist. That doesn't include the cost to install or maintain it, or connect it to the internet, or pay anyone to install it, or pay anyone to maintain the network behind it, or any other expenses -- simply for permission for that cable to exist at all runs multiple-hundred-dollars each month)
You are correct that 'last mile is expensive'. But you have it backwards -- high-margin business connections subsidize cheap residential ones. Businesses pay for those two to three mile hops, and home users get cheaper access by virtue of being 'on-the-way' to a business. This is why many businesses have fiber to their location even while most homes don't.
> There is an epistemological assumption in here that "the system" is fair or impartial or acknowledged in general by society or based on objective costs, or something.
Nope. No assumptions. Every home user signs an agreement, that clearly states what is "fair". They include terms like "no hosting a data center" or "don't use more than x00GB per month bandwidth". I'm not inventing or assuming a definition of "fair", it's spelled out in detail when you subscribe for service.
If you don't like a given ISP's definition of "fair", you buy something else with less restrictions that you feel is "fair". That usually ends up being dedicated internet service, since you can typically do (almost) anything you want with it.
Some ISP's have ridiculious definitions of what's "fair" that break traditionally expected norms. ISP's with ridiculous restrictions should totally eliminate those (e.g. ISP's should allow friends to play Minecraft on their connection).
But the simply act of having some restrictions is not, in and of itself, ridiculous. Since the product is by-nature shared, you need some level of protection against bad neighbors, or you end up with a 'tragedy of the commons' problem.
> I did a cost comparison, and consumer vs business class in my area for my ISP is merely double.
Your comparison is not even remotely fair. You did a cost comparison for "consumer shared" and "business class shared". The marginal cost between those is very little (they're both the same shared connection, business class just gets higher priority).
A real comparison would be you comparing your home ISP, with a Business Metro Ethernet line, or a Business Dedicated Fiber line. It will probably take you a little while to do a proper comparison, as at these prices, you almost always have to talk to a sales person first (pricing isn't public). In my area, a dedicated 20x20 runs $400 to $800 a month, assuming your already on-net (most businesses aren't, they pay even more).
If I can add as an aside, both yours and Drew's arguments are interesting reads. Both of you are arguing well.
To add to the data, the dedicated fiber line at our business goes for ~$700 and is 500 up 500 down (in Canada, BC).
that scales slightly better than the connections we get residentially, take from that what you will.
In my opinion, if the reality is that me and my neighbours are all sharing a single 50x10, then having an third party non regulatory agency dictate the use of a utility to me without disclosing any of the factors they use to reach that decision, feels like an unfair situation. I'm also surprised to learn that so little bandwidth is present. I wonder about that.
> "Real price" is the cost of a connection that "never" slows down, because it is literally dedicated to you and only you.
So I have come to the conclusion that in part we are talking past each other. You are saying “Consider the set of people such that they need more bandwidth than is available in the pre-existing copper/fiber cabeling to the premises. In that case they need new cable/fiber. This is expensive. The fiber that they will receive is theirs alone (since, for starters, nobody else seems to need it).” Yes, that is all true.
However you are conflating this with the alternative “Anyone who does not do this shares a line with other persons and therefore can have no reasonable expectations about their level of service.” Which is not true. It is, in fact, the responsibility of the ISP to make a reasonable effort to control oversubscription. Some ISPs do not do this, of course, but the consumer is not wrong for having the expectation.
Here in Austin, we have 8x4 DOCSIS3 to the building. That’s about 340MBit/s down. If you divide that by the ten units in this building, that’s 34Mbit of (dedicated) bandwidth to each unit. Meanwhile my advertised speed is 35Mbit. I’m never going to notice if everybody saturates their connections and I lose 3% of my bandwidth. So.
There are higher packages, and so I dunno, maybe there is this class of people that is just super annoyed that they are suddenly surfing at “only” 34Mbit during peak hours. But as a huge computer nerd I am not in that set, and I suspect the number of people who are seriously annoyed by oversubscription in Austin is a rounding error.
> However, there are lots and lots of other costs your ignoring. For instance, installing copper is not cheap.
I think we are in agreement here. Essentially I am saying that the cost difference between routing 1 packet and routing 1,000,000 packets is negligible, because the cost to route 1 packet involves installing the copper/fiber (and upkeep apparently).
What you are saying is “Well, what if you need hundreds of megabits of speed at your apartment?” And yeah, that is a problem, because in that case we need to run new cable/fiber. But this is not really a “shared line” problem. There is no service here that advertises a 300mbit shared line. It doesn’t exist.
> But you have it backwards -- high-margin business connections subsidize cheap residential ones. Businesses pay for those two to three mile hops, and home users get cheaper access by virtue of being 'on-the-way' to a business.
This seems absurd to me, to the point of being unbelievable. It may be that business class fiber is installed first, because no ordinary resident will lease their own fiber. However “installed first” and “subsidizes the cost of other subscribers” are two totally orthogonal things.
If you consider a fully-connected residential network, it spans a larger area, and has a lot more cable installed (and probably is a lot more expensive to build), than a fully-connected business network. Thus, connecting an arbitrary business to a fully-connected residential network will always be cheaper than connecting an arbitrary residence to a fully-connected business network. This doesn’t have anything to do with “first” or “second”, it has to do with the size and linear feet of cable and square mileage of coverage of one type of network when measured against the other.
> Every home user signs an agreement, that clearly states what is "fair". I'm not inventing or assuming a definition of "fair", it's spelled out in detail when you subscribe for service.
This is literally the COMPLETE relevant part of my residential-class internet connection terms of service from a major ISP:
> The Services are for your reasonable, personal non-commercial use only. You may not provide the Services to any person who is not a member or guest in your household, or to persons outside your premises, whether for a fee or otherwise. You will take reasonable precautions to prevent others from gaining unauthorized access to the Services.
Can I run an HTTP server? I have no idea. Can I use 2TB a month? No clue. There is absolutely nothing "clear" about what I can or cannot do with my internet connection, and if you think being "clear" and "fair" is standard business practice for ISPs in the United States you need a real reality check.
It's actually pretty difficult, technically, to provide you with a minimum guaranteed amount of bandwidth, without simultaneously limiting everyone else.
The problem is that there are many layers in the networking stack which interact in complicated ways. TCP congestion control algorithms and TCP slow start, which happen at the endpoints. Buffering, which happens at pretty much every intermediate hop in the path. And traffic shaping, which can pretty much only drop packets to limit bandwidth used.
If I'm supposed to guarantee you 768kbps, that means that I need to drop any packets from everyone else that exceeds the capacity minus 768kbps. But if I do this for everybody, I'm back to the case where I'm giving everyone a dedicated 768kbps, and they aren't able to share excess bandwidth at all.
Now, you could imagine a scheme in which I don't limit other people's bandwidth until you actually start using some. But then if it's congested when you start your transfer, you won't actually be able to send data at your full 768kbps at first, so TCP congestion control will kick in and limit you to less than that. So when traffic shaping looks at your stream, it sees something less than being used. Should it then limit everyone else such that it guarantees you your full 768kbps? If it does that every time someone uses their connection, then services that trickle just a small amount of data will essentially guarantee that everyone is always being guaranteed their 768kbps by limiting everyone else, and you're back to the "OK, now everyone has a dedicated 768kbps connection that won't go faster".
Or should it limit everyone else based on the bandwidth that you are currently using? Say, when it notices that you are sending 10 kbps, it throttles everyone else down to allow you 15 kbps; then when you increase to sending 15 kbps, it throttles everyone else down to allow you to send 20 kbps, and so on, until you hit your guaranteed bandwidth. Based on the delays it takes for TCP congestion control to ramp up the speed it tries to send while traffic shaping ramps up the amount it's limiting everyone else to give you the bandwidth you need, and the latencies involved (generally tens to low hundreds of milliseconds round trips) it could be a few seconds before you are actually able to get up to your guaranteed bandwidth.
But those first few seconds are generally the most crucial to be delivered quickly. Most people are browsing the web, where there are lots of short page loads. You want those to be able to kick in quickly and then be done with.
Remember, this is an industry where we've only recently come to understand the effects of buffer bloat on performance. Effective quality of service guarantees, other than giving everyone X guaranteed bandwidth by limiting everyone else to the total minus X bandwidth (which, when the network approaches capacity, becomes not much different than giving everyone an X bandwidth dedicated line), is a hard problem.
My real point is that I do not see why people should be
punished for using their connection continuously. Comcast
did not offer me "burst service," they offered me "always
on" service with "speeds up to X." I feel little sympathy
for an ISP that massively oversold service and then claimed
that it is abusive for customers to actually use what they
are paying for.
Yes. It's always on service (as in, you can be connected and have the service available 100% of the time) and have speeds up to X available. That's what they promised, and that's what they deliver. They also sold this as a "Residential" service. They have other services for commercial and dedicated lines, that they will sell you for greater utilization.
You are not paying for continuous use of X, you are paying for an always available line that gives you speed up to X, for personal residential use.
Restrictions on not using the network abusively or not running commercial services on a home network are not some creepy abusive plot, they are really the best that the networks can do to provide you with affordable, high-speed broadband.
Do I wish that there were more competition? Sure. Do I wish that I got higher speeds for cheaper? Sure. Do I wish that they would be network neutral and not block ports running on my network? Absolutely. But there's absolutely nothing wrong with selling a residential plan that is for residential purposes, and enforcing that, separately from their commercial plan, and separately from their dedicated plan. It's not that difficult; if you want to use it for commercial purposes, use their commercial plan. If you want guaranteed bandwidth, buy a dedicated line.
And remember, the ISPs have not "massively oversold" their services. If you look at the FCC's report (measured in 2011): http://web.archive.org/web/20130511164730/http://transition.fcc.gov/cgb/measuringbroadbandreport/Measuring_U.S._-_Main_Report_Full.pdf http://web.archive.org/web/20130511164730/http://transition.... , you'll see several ISPs that actually exceed their advertised bandwidths even during peak hours. Now, some of them are clearly not doing so well by that metric; Century Link is not even hitting 80% off peak, and barely 50% on peak. I would say you have a good case for false advertising there. But if you notice, Comcast and Verizon FiOS manage to exceed their advertised bandwidth, on average, even on peak. This matches pretty well with my experience over the last 5 years or so. I've had both DSL and Cable. DSL routinely underperformed what was sold, which was already paltry. But cable has generally exceeded what was sold, when connecting to servers that could actually handle the load.
So, I'm not sure I'd call that "massively overselling", other than maybe Century Link (and the other DSL providers don't hold up too well either).
Think of it like a buffet. For efficiency and simplicity, it's an all-you-can eat plan; unlike food, people aren't very good at knowing how much they are going to use and so charging based on usage is much more unpopular, making buffet style plans much more populat. So they do "all you can eat". And the rules are, it's all you can eat, and in one sitting; you can't take a bunch with you and then sell it to someone else, nor can you horde food for takeout because "you're going to eat it later". Would you start complaining and railing that the all-you-can eat buffet massively overselling? No. That's all that they're doing with the internet plan; it's just that there are less social conventions and it's easier to accidentally (or deliberately) massively overuse your fair share of bandwidth than it is to massively abuse a buffet, since it all happens automatically.