3 ms·

I remember learing decades ago that "In UNIX, everything is a file" I approach BI now as "In BI, everything is in your database". With everything in the databas

I remember learing decades ago that "In UNIX, everything is a file" I approach BI now as "In BI, everything is in your database". With everything in the database (SQL Server in my case) I can deliver BI to my users with quality tools like Tableau and Power BI - or just with generic web reports.

jon-wood 5y ago
It really is amazing how far you can get by just dumping everything into a single database and letting people do joins. We’re doing the same thing with Stitch replicating data from production databases and a few SaaS products into a big Postgres instance and it’s so low on maintenance hassle. The most advanced our “BI pipeline” gets is a few stored procedures run from a cron job to aggregate data in a particularly large table.
- intrasight 5y ago
  Exactly. I do BI at a Fortune 500. We have most everything anyone would care about in a database. Our job is to load and curate, create views and write queries.
  - CRConrad 5y ago
    Did you just invent Data Warehousing? If so, I have bad news for you...
    - intrasight 5y ago
      I do know that most project fail, and I suspect that a big part of that is having people focus on shiny new tools instead of just using a database and hiring some developers.
      - CRConrad 5y ago
        Yeah, I was mainly just being facetious about how what you described has "been a thing" for decades, and there's a whole industry around it (in which I've earned my living this century). So if you thought your insight was new, you'd be disappointed... :-) But, come to look at it in hindsight, my comment may have been misplaced -- maybe it should have gone as a reply to the same comment you replied to, with its "It really is amazing how far you can get..." At least if they meant "amazing" in the sense of "surprising". To me, what's amazing and surprising is only how many people don't already know this.

jon-wood 5y ago

It really is amazing how far you can get by just dumping everything into a single database and letting people do joins. We’re doing the same thing with Stitch replicating data from production databases and a few SaaS products into a big Postgres instance and it’s so low on maintenance hassle. The most advanced our “BI pipeline” gets is a few stored procedures run from a cron job to aggregate data in a particularly large table.

intrasight 5y ago
Exactly. I do BI at a Fortune 500. We have most everything anyone would care about in a database. Our job is to load and curate, create views and write queries.
- CRConrad 5y ago
  Did you just invent Data Warehousing? If so, I have bad news for you...
  - intrasight 5y ago
    I do know that most project fail, and I suspect that a big part of that is having people focus on shiny new tools instead of just using a database and hiring some developers.
    - CRConrad 5y ago
      Yeah, I was mainly just being facetious about how what you described has "been a thing" for decades, and there's a whole industry around it (in which I've earned my living this century). So if you thought your insight was new, you'd be disappointed... :-) But, come to look at it in hindsight, my comment may have been misplaced -- maybe it should have gone as a reply to the same comment you replied to, with its "It really is amazing how far you can get..." At least if they meant "amazing" in the sense of "surprising". To me, what's amazing and surprising is only how many people don't already know this.