5 ms·

RDBMS = data silo

stewbrew 7y ago
You make it sound as if this were a bad thing. RDBMS work well for many use cases. There are plenty of tools around to work with them. Good open source implementations exist.
- shmooth 7y ago
  I played with Denodo (data virtualization software) a couple years ago and thought it was pretty legit. In theory, it could be used to provide that industrial strength abstraction layer between your Tableau/Looker/etc. and your bajillion weird and not-so-weird (RDBMS) data sources. That would seem to make sense to me from the point of view of -- I would want my data visualization/analytics-type company to be able to concentrate on data visualization/analytics, not building some insane and never-ending data abstraction layer. The part that surprised me was that Denodo could allegedly do a lot of smart data caching, thus speeding things up (esp hadoop-oriented data sources) and keeping costs down. I'm guessing the other data virtualization providers can do similar.
  - kfk 7y ago
    I have had to work with the Denodo for the past 1+ year, a total nightmare. Data virtualization is a "good in theory" concept but "doesn't work in practice" reality. Going back to the original sources for each query doesn't work, it will always be slower than using a proper analytics data warehouse. Caching doesn't help because at that point you can just do ETL. Also Denodo itself is full of weird behaviors and bugs, my team collectively decided it's worth the most hate of all the "enterprise" tools we use. One thing Denodo is good for is as an "access layer", but then maybe PrestoDB would be worth a shot or maybe even just a sqlalchemy and python.
- dustingetz 7y ago
  Consider the case when you are a big company and have hundreds of them
- rqmedes 7y ago
  The problem space (Business) is not complex, its incredibly simple. Unfortunately the design patterns we use and the tooling is flawed. It has been this way for at least 25 years and the RDBMS, languages, design patterns, and architectures ( 2-tier/ 3-tier) are the cause. They make it simple to get started and even without knowing what you are doing you can easily churn out something that works if it is simple, doesn't change often, doesn't need to scale and deals with small amounts of data. But similar to http://blogs.tedneward.com/post/the-vietnam-of-computer-science/ http://blogs.tedneward.com/post/the-vietnam-of-computer-scie... "It represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy." RDBMS are the root cause.
  - jjeaff 7y ago
    I feel like you are coming from an alternate universe. Nosql is the quick and easy thing to start with, then your needs become more and more complex and nosql just won't cut it anymore. Sure, nosql can scale and perform, but only if your needs are very specific and simple. There are no major systems out there of even moderate complexity that aren't built on an rdbms.
    - dodobirdlord 7y ago
      > There are no major systems out there of even moderate complexity that aren't built on an rdbms. I don't think this claim is accurate. https://www.dynamodbguide.com/the-dynamo-paper/ https://www.dynamodbguide.com/the-dynamo-paper/
      - rqmedes 7y ago
        That's the problem, business systems are not complex, they are incredibly simple, they are made complex by the way we think about and structure our models and then interact with them. We have the wrong boundaries, and the wrong languages
        7y ago
aceregen 7y ago
I don't understand why this gets downvoted. While it may lack context (with the aim of being controversial), it sparked a healthy amount discussion here!

RDBMS = data silo

You make it sound as if this were a bad thing. RDBMS work well for many use cases. There are plenty of tools around to work with them. Good open source implementations exist.

shmooth 7y ago
I played with Denodo (data virtualization software) a couple years ago and thought it was pretty legit. In theory, it could be used to provide that industrial strength abstraction layer between your Tableau/Looker/etc. and your bajillion weird and not-so-weird (RDBMS) data sources. That would seem to make sense to me from the point of view of -- I would want my data visualization/analytics-type company to be able to concentrate on data visualization/analytics, not building some insane and never-ending data abstraction layer. The part that surprised me was that Denodo could allegedly do a lot of smart data caching, thus speeding things up (esp hadoop-oriented data sources) and keeping costs down. I'm guessing the other data virtualization providers can do similar.
- kfk 7y ago
  I have had to work with the Denodo for the past 1+ year, a total nightmare. Data virtualization is a "good in theory" concept but "doesn't work in practice" reality. Going back to the original sources for each query doesn't work, it will always be slower than using a proper analytics data warehouse. Caching doesn't help because at that point you can just do ETL. Also Denodo itself is full of weird behaviors and bugs, my team collectively decided it's worth the most hate of all the "enterprise" tools we use. One thing Denodo is good for is as an "access layer", but then maybe PrestoDB would be worth a shot or maybe even just a sqlalchemy and python.
dustingetz 7y ago
Consider the case when you are a big company and have hundreds of them
rqmedes 7y ago
The problem space (Business) is not complex, its incredibly simple. Unfortunately the design patterns we use and the tooling is flawed. It has been this way for at least 25 years and the RDBMS, languages, design patterns, and architectures ( 2-tier/ 3-tier) are the cause. They make it simple to get started and even without knowing what you are doing you can easily churn out something that works if it is simple, doesn't change often, doesn't need to scale and deals with small amounts of data. But similar to http://blogs.tedneward.com/post/the-vietnam-of-computer-science/ http://blogs.tedneward.com/post/the-vietnam-of-computer-scie... "It represents a quagmire which starts well, gets more complicated as time passes, and before long entraps its users in a commitment that has no clear demarcation point, no clear win conditions, and no clear exit strategy." RDBMS are the root cause.
- jjeaff 7y ago
  I feel like you are coming from an alternate universe. Nosql is the quick and easy thing to start with, then your needs become more and more complex and nosql just won't cut it anymore. Sure, nosql can scale and perform, but only if your needs are very specific and simple. There are no major systems out there of even moderate complexity that aren't built on an rdbms.
  - dodobirdlord 7y ago
    > There are no major systems out there of even moderate complexity that aren't built on an rdbms. I don't think this claim is accurate. https://www.dynamodbguide.com/the-dynamo-paper/ https://www.dynamodbguide.com/the-dynamo-paper/
    - rqmedes 7y ago
      That's the problem, business systems are not complex, they are incredibly simple, they are made complex by the way we think about and structure our models and then interact with them. We have the wrong boundaries, and the wrong languages
      - 7y ago

aceregen 7y ago

I don't understand why this gets downvoted. While it may lack context (with the aim of being controversial), it sparked a healthy amount discussion here!