3 ms·
I've personally built all of the things you're talking about, on real products, with lots of traffic, and it just isn't that hard. Annoying, yes, but arguably a
by irq11 9y ago
I've personally built all of the things you're talking about, on real products, with lots of traffic, and it just isn't that hard. Annoying, yes, but arguably a core competency of any data team, and something you want to think very carefully about outsourcing. Also, most teams won't have to "scale" many of the things you're talking about...that's a problem that you only have to solve when you're the central point of failure for lots of companies.
As for the n^2 problem, yes, I can see that argument, but in my experience, n is always a small number (i.e. 2-10), and they rarely need to cross-communicate in a fully connected graph. In short, piping everything through an expensive, centralized point-of-failure run by strangers seems like such a risky proposition that I can't imagine many companies would have the integration complexity to justify the risks.
But again, people seem to be finding value in it, so maybe I'm missing something. I was honestly expecting you to tell me something about advanced data warehousing or integrated analytics tools or something like that. If you tell me that integrating with segment automatically starts correlating session data across providers, well...that starts to be compelling.
- pclark 9y agoGo tell a marketer that if they use Segment they can onboard new marketing tools and it'll 'just work' without any engineering time, and they'll shit a brick. (the data warehouse part is imho folly.)
- irq11 9y agoYes, I see this value. But again, that's something that could rather easily exist as an open-source, self-hosted tool, and I kind of wonder why the middleman hasn't been optimized away. We're in a world where people would rather self-host github to save a few bucks...this is an easier problem.
- pclark 9y agoopen source is basically dead courtesy of SaaS. people would much rather pay a few bucks than self-host. hell, there are entire successful companies that 'just' host open source software and wrap it up in a SaaS experience.
- hilbertseries 9y agoYou're just too technical and think everyone is on your level. I work for a mobile analytics provider and we once sent a json log lines file format to a customer, who replied that they didn't know how to parse it and asked us for CSV instead. So, if that's the caliber of developer you have, ask again how easy everything you just talked about is.
- irq11 9y agoI'm open to the argument that most development teams are utterly incompetent. I'm just somewhat skeptical of that argument.