You and I must be the only ones on HN using mongo at scale and enjoying it. We use it as an event database which collects over 100M+ semi-structured records da

You and I must be the only ones on HN using mongo at scale and enjoying it. We use it as an event database which collects over 100M+ semi-structured records daily with about 200 (and growing) different schemas... It keeps 1.5TB of records in the collection which is achieved using the invaluable capped collection function, and we can index the structured fields very easily. We also pipe the data into elastic for quick kibana querying but just that step requires a lot of index partitioning and mapping to work smoothly. I also feel that a significant amount of mongo's power lies in the aggregation pipeline which is often ignored and which can replace entire ETLs, sending the compute to the data can be much more efficient in a lot of use cases. All this runs for us on some relatively cheap (compared to 1.5TB of high i/o RDS) commodity hardware, running 3 replicated nodes. More complex ETL jobs or Hadoop users can pull data from mongo selectively and quickly - spark has a way to partition a query into N threads based on a provided key achieving line rate data pulls even from a single node. Throw in compression on disk and in-flight and you get something that is really compelling. We've all run out of disk on a database before, it's never fun. My one gripe is the learning curve for new developers. I do find that once leveraged, I see a lot more data being stored when it's as simple as nesting an object into your main object and saving it, which data teams like. Depends on your use cases, but really, it's been an invaluable tool so far.