3 ms·
One thing that always confuses me about the business factors in this kind of announcement, where they say that the vast majority of users won't be effected: >
by jrochkind1 6y ago
One thing that always confuses me about the business factors in this kind of announcement, where they say that the vast majority of users won't be effected:
> We evaluted CI/CD minute usage and found that 98.5% of free users use 400 CI/CD minutes or less per month.
Okay, so just that 1.5% of free users, each using at most 1600 minutes more than the 400 under the new limits... is enough cost to actually matter and make it worth making this change?
Or they anticipated that number going up if they didn't make the change?
It seems odd to me to say "This hardly effects anyone at all, almost everyone can keep doing exactly what they are doing for the same price (free in this case)", AND "this was necessary for the sustainability of our budget."
What am I missing?
- lacker 6y agoThis sort of thing happens a lot when you first set up a free plan. Some part of it turns out to cost more money than you expected, and it’s a small but growing segment. If you’re going to have to pull back in it eventually, why wait? Waiting might just annoy twice as many people when you inevitably have to introduce stricter limits later.
- TimWolla 6y ago> and it’s a small but growing segment They could have grandfathered the existing accounts into the 2000 minutes. Then the segment would no longer be growing and stay small, no?
- foota 6y agoExcept in this case individual users will also grow. You'd have to do something like grant them their current usage plus some buffer (without letting them know ahead of time maybe, since otherwise this is gameable).
- skinnymuch 6y agoWhy do you need a buffer? If you’re grandfathering in old and limiting new users more, what is gameable? Besides a small portion of people registering accounts in the short time between announcement and implementation?
- nialv7 6y agoThese are carefully constructed truthful statistic lies. Of all the free users, how many of them even have a repository? Out of all the users who have a repository, how many of them actually make any use of CI/CD? They are saying "1.5%" to make it sounds small, but those 1.5% could account for a significant portion of total minutes of CI/CD used.
- echelon 6y agoGreat point. And I guarantee you that they did the math to optimize for conversion and churn. Behind all product decisions are an army of analysts.
- sytse 6y agoWe certainly did the math. And a small percentage of the users generate a large percentage of the CI minutes. Hence the change.
- paxys 6y agoThis is exactly it. 1.5% of users does not equal to 1.5% of CI/CD minutes. And such messaging isn't always dishonest. If you have 99 users using 100 minutes each and 1 user using 10 million minutes, setting the max limit to say 1000 is a net benefit for the system and its users, at the expense of the one outlier.
- jrochkind1 6y agoYes, but in this case one user couldn't use 10 million minutes, they were already capped at 2000. Now they are capped at 400 instead. So that limits how extremely disproportionate the high percentile use could be. As I said, that 1.5% of users could have been using at most 1600 more minutes a month than the new max.
- hartator 6y agoThey can make infinite repos though!
- yebyen 6y agoThe math works out if you presume that 20% or more of that 1.5% is converted into a paying customer. If I was using GitLab that much and could pay $20 to support GitLab but wasn't already a paid subscriber, I probably would think about it. (I'm thinking about it now, and I don't even use GitLab.)
- bsder 6y ago> Or they anticipated that number going up if they didn't make the change? Do CI/CD minutes ever go down once you start? I suspect not. They probably plotted the trend line and realized that they had to do something. Or, it could be bifurcated. Most people use like 2 minutes but 1 in 100 use 2000 minutes. They may also be tracking and saw that people were using the free 2,000 minutes as "overflow" with multiple accounts.
- contravariant 6y agoIt wouldn't surprise me if 90% (or more) of their free users use exactly 0 minutes of CI/CD per month. So then it's more like 1 in 6 of the people that do use CI/CD need more than 400 minutes per month. Just to be clear, these numbers are completely made up. They seem reasonable though.
- darau1 6y agoDon't forget the fact that you have the option of running pipelines on your own machines. I do this for all my projects, and only really use shared runners by mistake when I forget to set it up. I am a little dismayed to find out that open source projects no longer get unlimited minutes, though.
- fortran2003 6y agoIt’s because cost isn’t the issue. The change is designed to convince more free users to switch to a paid account.
- jrochkind1 6y agoOK. But they're saying the number of free users that would have to switch to a paid account is 1.5%, right? Which doesn't necessarily seem like enough to justify a disruptive change that might scare customers... but I don't really know I'm hardly an expert or have experience in this kind of business. You think it is, is your hypothesis?
- swsieber 6y agoIf they have 100x the free users than paid users it would totally be worth it. It depends on the numbers.
- rco8786 6y agoA service like that will have an enornous amount of free users who signed up but don’t ever use the service and count as 0.
- dchuk 6y agoPretty incredulous position to take when a company offering free stuff makes adjustments to their free stuff
- jrochkind1 6y agoWhat position do you think I took? I just asked some questions out of curiosity, I did not mean to take any position. Do you have a blanket position against being curious about the business models or policies or statements of companies offering free stuff about their free stuff? Does that apply to facebook and google too?
- theptip 6y agoWorth thinking about this in the context of fat-head vs. long-tailed distributions, or less formally "whales". A very small number of users (the "head" or "whales") can account for most of the area under the curve, which in this case is the cost of delivering free build minutes. For a toy example, imagine one user uses 2k minutes, 10 use 100 minutes, and 1000 use 1 minute. In this case you have "Over 99% use 100 minutes or less", but 50% of your cost is going to the one 2k minute user. I have no idea if this is their exact curve, just showing how a fat-head distribution could explain what they are saying. I'd expect the "demand" (i.e. minutes used if they were not constrained / all paid for) to be a power law distribution. Also worth noting that there's a sort of bimodal selection effect going on here -- it's unlikely that you use exactly 2k minutes/month, since you'd be hitting your limit and that would be disruptive. So the closer you get to using 2k minutes, the more likely you are to pay for more than the free tier. So I'd expect this pricing change to also force some users that were previously on 400 minutes +- 100 to have to upgrade too; this will impact some of the 98.5% of free users that are using <= 400 _on average_ per month.