4 ms·

Is there any medium size company that keeps all its data in 1 place?

fiddlerwoaroof 3y ago
It’s still fairly common for the majority of critical data to be in a single database. Even if microservice initiatives have moved some data out.
Spivak 3y ago
I've worked at two of them -- all data, sans pointers to content addressed immutable object storage, is stored in a single database. It worked well for us; it's not rainbows and unicorns wow all of our problems disappeared but we got what we needed with sufficient performance for an app that was not well optimized. There were challenges like the difficulty of working with a few huge tables that existed since the company's inception and were practically infeasible to do any schema changes on, but that's more a modeling issue. It was also a single point of failure service. If the db goes down so to does everything but constant sense of impending doom it was less of a problem than I thought with ndb.
- vmfunction 3y ago
  It is good to see people on HN are supporting this single db approach. And with development such OrioleDB, it would solve the horizontal scaling. And all the access to data are native.
- hasty_pudding 3y ago
  Y'all couldn't have some read-only clusters that could just take over as the master?
  - Spivak 3y ago
    Sorry, I mean a single point of failure service. I guess maybe the term would be large blast radius. The database implementation was HA to almost paranoid degrees but if we lost the database it was a total outage every service down for all customers. If the DB had issues there was really no sense of a partial outage.
swiftcoder 3y ago
Amazon's entire system ran on a single Oracle cluster up until about 2008 (they were still at the tail end of phasing that out when I was there in 2012).
- bob1029 3y ago
  In my time at Samsung, we were tracking most of the authoritative state for the factory in a single monster Oracle DB instance. This made troubleshooting systems integration problems fairly trivial once you had some business keys to look for. There are so many crazy tricks out there. You would be amazed at how much abuse an old school RDBMS engine can take on modern hardware. In-memory OLTP on servers that can host terabytes of RAM completely changes the game. Disaggregated compute & storage provides seamless OLAP capabilities on the same SQL connection instance for the most modern providers.
  - SPascareli13 3y ago
    Are you talking about GMES?
- randomdata 3y ago
  Oracle (the software) isn't a database, though. It is a database management system. It seems highly unlikely that it kept all the data in one place. Different workloads generally need different database structures. And Amazon circa 2008 was complex enough that it no doubt had a gamut of different workload needs, even if that was all managed under one management system. Really, as it pertains to this, it is a question of "where do I want to move the complexity to?", not "can I eliminate it?"
- alexchamberlain 3y ago
  There were no derivative stores (Redis caches? Solr views?) around it?

It’s still fairly common for the majority of critical data to be in a single database. Even if microservice initiatives have moved some data out.

Spivak 3y ago

I've worked at two of them -- all data, sans pointers to content addressed immutable object storage, is stored in a single database. It worked well for us; it's not rainbows and unicorns wow all of our problems disappeared but we got what we needed with sufficient performance for an app that was not well optimized. There were challenges like the difficulty of working with a few huge tables that existed since the company's inception and were practically infeasible to do any schema changes on, but that's more a modeling issue. It was also a single point of failure service. If the db goes down so to does everything but constant sense of impending doom it was less of a problem than I thought with ndb.

vmfunction 3y ago
It is good to see people on HN are supporting this single db approach. And with development such OrioleDB, it would solve the horizontal scaling. And all the access to data are native.
hasty_pudding 3y ago
Y'all couldn't have some read-only clusters that could just take over as the master?
- Spivak 3y ago
  Sorry, I mean a single point of failure service. I guess maybe the term would be large blast radius. The database implementation was HA to almost paranoid degrees but if we lost the database it was a total outage every service down for all customers. If the DB had issues there was really no sense of a partial outage.

swiftcoder 3y ago

Amazon's entire system ran on a single Oracle cluster up until about 2008 (they were still at the tail end of phasing that out when I was there in 2012).

bob1029 3y ago
In my time at Samsung, we were tracking most of the authoritative state for the factory in a single monster Oracle DB instance. This made troubleshooting systems integration problems fairly trivial once you had some business keys to look for. There are so many crazy tricks out there. You would be amazed at how much abuse an old school RDBMS engine can take on modern hardware. In-memory OLTP on servers that can host terabytes of RAM completely changes the game. Disaggregated compute & storage provides seamless OLAP capabilities on the same SQL connection instance for the most modern providers.
- SPascareli13 3y ago
  Are you talking about GMES?
randomdata 3y ago
Oracle (the software) isn't a database, though. It is a database management system. It seems highly unlikely that it kept all the data in one place. Different workloads generally need different database structures. And Amazon circa 2008 was complex enough that it no doubt had a gamut of different workload needs, even if that was all managed under one management system. Really, as it pertains to this, it is a question of "where do I want to move the complexity to?", not "can I eliminate it?"
alexchamberlain 3y ago
There were no derivative stores (Redis caches? Solr views?) around it?