3 ms·

I've been working on high scale systems for over a decade. An early lesson was to put limits on everything. Because we started without them, customers leveraged

I've been working on high scale systems for over a decade. An early lesson was to put limits on everything. Because we started without them, customers leveraged the lack of limits, and cardinality became a huge problem. Real example: at SendGrid, you can attach "categories" to emails which can help organize your email analytics by enabling you to group emails by type. Just as you can view the statistics on all your email activity, you can go a step further and view the statistics by a particular category. Our ability to provide analytics blew up when several customers exploded that to literally millions of categories, most totally unique to a single message. We no longer release anything as unlimited. If a use case arises, we can always expand the limit, but establishing a limit after the fact is hard and customers don't like having to go back to change integrations.

paxys 4y ago
Putting a user-facing restriction in place is very different from having your underlying infrastructure itself make that assumption. If the product spec says a user will be allowed 5 messages per day, and that's all you design the system to support, you will inevitably run into challenges when the number has to be changed to 20 or 2000 or 20 billion down the line. At that point "but we only used an 8-bit integer because you said the max would be..." isn't a good enough answer, and you are going to be stuck in a very messy rewrite.
bee_rider 4y ago
Was it hurting anything to have millions of categories? I mean, I’d assume the metrics interface they’d gotten from you would be essentially useless, but if they had some workflow they worked, why not let them have it?