I'm believe there has to be a better way where planning can be more freeform. I've seen both sides of NoSQL versus a real DB, and I'm using a real DB for my cur

I'm believe there has to be a better way where planning can be more freeform. I've seen both sides of NoSQL versus a real DB, and I'm using a real DB for my current product which is where I package a small DB inside a Document within a NoSQL design and I have a real DB to index the documents. An interesting aspect is that the Database is a large part of the data pipe, but it often is insufficient for the product. The key issue being torn writes between the database and a service like twilio or elastic-search. A good way to broker this is via a queue, or to use a tailer via kafka for your data pipeline. My gut is that so many things are just held together by bail wire, and the complexity is depressing to me. So, here I am, inventing a new thing. I've invented a language called Adama ( https://www.adama-platform.com/ https://www.adama-platform.com/ ) which allow a document to have a schema. Accidentally, I added transformation logic to the schema and then hooked it up to a socket to discover that many things were possible. I had intentions of using this thing to help get my state under control with node.js for a complex board game, but the language took over such that the entire game was eventually built within the language. Kind of neat. My thesis is that the NoSQL pattern is fantastic for building products quickly if you don't mind the mess that schemaless evolves into, and a key weakness of what I have is that the document is limited in size in memory (which is then made tighter due to the reactive elements). However, I can leverage a logger much like a NoSQL solution to produce many of a stream of changes which I can shred into various solutions. For instance, I could put data changes directly into snowflake to provide massive scale queries. There are some neat possibilities with this approach, but limits as well. However, I'm not sure if planning ahead is the most effective thing for a startup. I believe a new discipline over the coming decades will emerge regarding how to pick the appropriate level atomic unit of data, and then the skills to break down that unit will be more common place.