4 ms·

From a BI consultancy perspective, not in the slightest. The amount of time we spend on ETL is not because of the difficulty of piping data around from place-to

by greggyb 11y ago

From a BI consultancy perspective, not in the slightest. The amount of time we spend on ETL is not because of the difficulty of piping data around from place-to-place. The difficulty is in modelling data appropriately to support ad-hoc analyses. Rather, the E and L portions don't provide much difficulty (hassle, frustration, sometimes time, sure, but they're not inherently difficult). The T, transformation, is huge in many ways. Think of it this way: the data model is the primary UI for an analyst or any power user. It also dictates query performance. Adding a visualization layer on top of Salesforce's schema, e.g. is not too helpful, regardless of where that data is living. You can answer trivial questions without too much difficulty, but the difficulty ramps up quickly. The data access patterns, types of logic necessary, and end-user demands are hugely different between an OLTP and OLAP workload. There's also potentially huge complexity in conforming dimensions across disparate source systems' data. Master data management is another huge component that hits a lot of the ETL pipeline. These concerns are all on top of hooking up the right ends of the hose to one another. I don't mean to disparage the product or company and hope I don't come across as if I am. I just want to point out that they address only a small component of a large process, which in turn is only a segment of the BI lifecycle.

dwmintz 11y ago
Agree that it's only one part of the data lifecycle, but if you don't have your own team of engineers to build all those connectors, you can't even get to the point of being able to transform your data. We at Looker (disclosure: work there), are totally focused on making the transformations fast, flexible and powerful. But without the data to transform, there's nothing we can do to help customers. So we're super psyched that Segment is stepping in to fill this void and get the E and L done, so we can T.
- greggyb 11y ago
  Absolutely. I was just trying to illustrate where a single tool doesn't pose a threat to BI companies, particularly from a consulting angle. Tools like Segment and yours are invaluable. I work for a Microsoft partner, so often am stuck in that ecosystem for better or worse, but the problems and solutions in the BI space are largely universal and transcend specific tools. Even when the tools make individual tasks trivial, the overall architecture and design of a data pipeline and visualization solution leave plenty of room for companies like the one I work for.

by greggyb 11y ago

dwmintz 11y ago

Agree that it's only one part of the data lifecycle, but if you don't have your own team of engineers to build all those connectors, you can't even get to the point of being able to transform your data. We at Looker (disclosure: work there), are totally focused on making the transformations fast, flexible and powerful. But without the data to transform, there's nothing we can do to help customers. So we're super psyched that Segment is stepping in to fill this void and get the E and L done, so we can T.

greggyb 11y ago
Absolutely. I was just trying to illustrate where a single tool doesn't pose a threat to BI companies, particularly from a consulting angle. Tools like Segment and yours are invaluable. I work for a Microsoft partner, so often am stuck in that ecosystem for better or worse, but the problems and solutions in the BI space are largely universal and transcend specific tools. Even when the tools make individual tasks trivial, the overall architecture and design of a data pipeline and visualization solution leave plenty of room for companies like the one I work for.