12 ms·

>DBMSs will finally be completely autonomous and self-healing Hahahahahahahaha - such optimism. Autonomous, self-healing, horizontally-scalable stateless syst

by ukj 7y ago

>DBMSs will finally be completely autonomous and self-healing Hahahahahahahaha - such optimism. Autonomous, self-healing, horizontally-scalable stateless systems are hard-but-doable. Autonomous, self-healing, horizontally-scalable stateful systems are the stuff nightmares are made of, if your application layer doesn't relax its expectations re: ACID properties of the system.

bob1029 7y ago
I think compromises at the application layer are the trick here. Instead of trying to make every possible aspect of the business system ACID compliant, perhaps only small aspects of it actually need this level of transactional isolation. Alternatively, perhaps you have a business model which is amenable to document-based storage. If you can reduce your business modeling to: Serializable GUID-keyed objects - Any arbitrary business type instance that can be turned into JSON bytes. GUID-keyed transactions - Document the specific object GUID(s) to be locked for a transaction. You can very easily entertain a resilient, clustered solution. Going wide on GUID keys is trivial compared to alternative approaches. If you need to support any degree of SQL grammar or row-based storage, you are in for a completely different dimension of pain. I would never subject myself to this. Instead, I would reach for SQLite or Postgres.
carlineng 7y ago
This is the focus of Andy Pavlo's research: https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-driving-database-management-system.html https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-dr... Essentially, there is a long history of developing tools to make databases more "autonomous". Pavlo's research centers around accurately capturing the state of the database, and training a system to learn the impact of tuning one of the many available knobs.
- ukj 7y ago
  Tuning != Scaling Tuning == getting the most out of your hardware Scaling == my perfectly tuned node, with the lowest-latency/mostest IOPS IO subsystem, and highest number of CPUs, and the mostest amount of RAM can't handle my workloads. What now? Re-architect your monolith and re-consider your transactional boundaries - that's what.

by ukj 7y ago

bob1029 7y ago

I think compromises at the application layer are the trick here. Instead of trying to make every possible aspect of the business system ACID compliant, perhaps only small aspects of it actually need this level of transactional isolation. Alternatively, perhaps you have a business model which is amenable to document-based storage. If you can reduce your business modeling to: Serializable GUID-keyed objects - Any arbitrary business type instance that can be turned into JSON bytes. GUID-keyed transactions - Document the specific object GUID(s) to be locked for a transaction. You can very easily entertain a resilient, clustered solution. Going wide on GUID keys is trivial compared to alternative approaches. If you need to support any degree of SQL grammar or row-based storage, you are in for a completely different dimension of pain. I would never subject myself to this. Instead, I would reach for SQLite or Postgres.

carlineng 7y ago

This is the focus of Andy Pavlo's research: https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-driving-database-management-system.html https://www.cs.cmu.edu/~pavlo/blog/2018/04/what-is-a-self-dr... Essentially, there is a long history of developing tools to make databases more "autonomous". Pavlo's research centers around accurately capturing the state of the database, and training a system to learn the impact of tuning one of the many available knobs.

ukj 7y ago
Tuning != Scaling Tuning == getting the most out of your hardware Scaling == my perfectly tuned node, with the lowest-latency/mostest IOPS IO subsystem, and highest number of CPUs, and the mostest amount of RAM can't handle my workloads. What now? Re-architect your monolith and re-consider your transactional boundaries - that's what.