11 ms·

Is there a word for a case of "I told you so" that went on for so long that it curdled from frustration, to despair, to cynicism, to a realignment of your under

by alrs 8y ago

Is there a word for a case of "I told you so" that went on for so long that it curdled from frustration, to despair, to cynicism, to a realignment of your understanding of the human project as something barely capable of tying its own shoes and making it out to the mailbox and back?

bubaflub 8y ago
Smug self-righteousness
- toomuchtodo 8y ago
  I’ll allow it. Years later, I’m still miffed at Graylog (centralized logging engine) for having required MongoDB for a small bit of auth and meta storage that could’ve easily been done in MySQL or PostgreSQL (RDS even), forcing the need for that much more ops work for a small Mongo cluster for HA. Everyone deprecating the use of Mongo is a welcoming turn of events. I shall recall these dark days to the next generation as “NoSQL Madness”, or more colloquially, “my schema is my app layer”.
  - Bucephalus355 8y ago
    Can you explain in more detail about “my schema is my app layer”? EDIT: fixing autocorrect
    - toomuchtodo 8y ago
      MongoDB by default is (was? It’s been a while since I’ve used it) schemaless, which means all of your data validation must take place in your app instead of the database. Your data integrity is then only as good as your weakest validation. Edit: scheme/schema autocorrect typos corrected. Thanks!
      - taspeotis 8y ago
        > my scheme is my app layer ... MongoDB by default is (was? It’s been a while since I’ve used it) schemeless https://www.google.com/search?q=define%3Aschema https://www.google.com/search?q=define%3Aschema
        deleted 8y ago
        [deleted]
      - chisleu 8y ago
        Right, but most people using it use model or data repository patterns to ensure correctness. It does offer nearly infinite flexibility provided you use it correctly. You can add fields without any sort of DB work, you just start adding fields to rows as needed and let it catch up organically. There are use cases where mongo makes a lot of sense. It's very popular in the node.js / RAD world for sure. I certainly have never been a huge fan by any means. Only relatively recently did they solve distributed writes.
        toomuchtodo 8y ago
        Unfortunately, I’d argue PostgreSQL gets you all the same benefits with JSON storage (fairly equivalent to Mongo docs), while also giving you all the goodness of a relational, transactional, schema enforcing RDBMS. PGSQL became Mongo faster than Mongo could become PGSQL.
        WrtCdEvrydy 8y ago
        This is the same thing that happened in Java. Other languages started prototyping features... that eventually just end up being implemented in Java.
        int_19h 8y ago
        Java did it way too slow, and that is a significant contributor to it being relegated to "legacy" in many areas. If it waited for the other languages to prototype stuff, it might have not been the case. The problem is that it waited for them to prototype it, refine it, release it, popularize it, and for their community to adopt it, before even starting to work on it in Java - which means that by the time they had it, most people who needed it were already elsewhere (not necessarily off JVM, just another language). Lambdas were a very good example - if you look at the closest competitor, C#, it got the first take on them back in 2005. Then a major refinement in 2008, adding type inference. By 2010, lambdas were idiomatic in C#. Java, in contrast, released the first version in 2014. And even then, they're still less powerful.
  - throwaway5752 8y ago
    "my scheme is my app layer" To be fair you can do use schema validators in Mongo. Not sure it's widespread in practice. And there are other distributed databases that aren't document stores that have schemas and various subsets of SQL implemented.
    - deleted 8y ago
      [deleted]
  - bane 8y ago
    I once worked at a place that, at some point in the past, had developed some kind of semantic graph database that never gained market traction. The author of it had ended up as CTO and kept seeking out uses for his work and ended up finding all kinds of odd places for it to live, including as the auth database in a large scale document analytics system and running part of the payroll. We were constantly running into all kinds of scalability issues where this 12 year old component was often the source of our pain, but he'd never even entertain a conversation to eliminate it and consolidate or replace it with something better.
    - seibelj 8y ago
      This is the definition of software engineering hubris distilled into a few paragraphs
    - duxup 8y ago
      I worked for a company that was old but had a fancy new product. It was just about to be released when.... A competitor bought us for stupid money. They fired most folks (not my small department) had their CTO decide between our fancy new product, and the product he created. There was no question, our product was amazing, his Frankenstein was two pieces of equipment cabled together in three items the footprint ... and still did less, and was quite a ways away from being "ready". If there was a silver lining, the Frankenstein product doomed that company and we got bought by the far more competent competitor they had.
      - qbaqbaqba 8y ago
        Happy ending.
        duxup 8y ago
        It's surprising how that works. I've been through numerous mergers, acquisitions, been in the company acquiring a company. Lots of hair pulling and I can say that it was really hard to predict the outcome for any individual positive or negative in every case until long afterward.
  - djrobstep 8y ago
    My pet theory is that NoSQL took off purely because people were sick of having to manage schema changes. Unfortunately, people reacted to being (justifiably) frustrated with schemas by throwing strict schemas out entirely, instead of making better schema management/migration tools.
    - staticassertion 8y ago
      Tell that to AWS. They've banned relational databases for specific workloads because Dynamo (nosql) provides more consistent performance, and is easier to operate. Tons of conflation of Mongo's problems with those of nosql in this thread.
      - djrobstep 8y ago
        Your average dev is not making decisions based on what might work best for one particular problem Amazon has.
        staticassertion 8y ago
        Do you think all of the companies that chose Cassandra and Dynamo were wrong to do so? There's no use case for NoSQL? There were no lessons learned, value adds from NoSQL? How do you explain the 'NewSQL' approach, which seems to be so clearly borne of what we've learned from NoSQL? It should be obvious that NoSQL has value, regardless of the issues with one of the earlier NoSQL DBs.
        pjmlp 8y ago
        I don't see a value other than fashion driven development, specially when comparing the bare bones browser GUI for Dynamo with something like SQL Server Management Studio or that whole story with primary and secondary indexes, with prices being set by index usage.
        menacingly 8y ago
  - baroffoos 8y ago
    Setting up graylog was one of the worst mistakes I made. It took forever to get all the required software installed and configured and then it was taking up all the ram on the server doing fuck all.
    - SteveNuts 8y ago
      That was just Java doing Java things. Business as usual.
      - RhodesianHunter 8y ago
        This is a nonsense comment doing language troll things.
    - deleted 8y ago
      [deleted]
    - atm0sphere 8y ago
      There were single script 1-click installs for it in bash all over for me..but yeah jvm is a hog.
      - yjftsjthsd-h 8y ago
        My experience with those one-liner installs is that they usually work... strictly speaking. They don't scale, they don't deal with edge cases, they know nothing of your environment. They install one piece of software (in an "interesting" way that won't upgrade), and that's it.
        sunnymanokl 8y ago
        My last pay check was $9500 working 12 hours a week online. My sisters friend has been averaging 15k for months now and she works about 20 hours a week. I can't believe how easy it was once I tried it out. This is what I do, HERE►► http://www.worktoday33.com http://www.worktoday33.com
        8y ago
  - Sandra_lisa4 8y ago
    XzXzXZXX
  - dominotw 8y ago
    It was just a discovery phase.
  - pwm 8y ago
    I agree with the general sentiment just wanted to point out that "my schema is my app layer" has some valid use-cases. At my current job we deal with highly complex schemas (modelling insurance contracts). Correctness is paramount but at the same time you need flexibility in terms of change over time. Defining these schemas at the DB level would be painful, a bit like writing web apps in assembly. Languages like Haskell (which we use) help here with their rich and expressive typing capabilities, so we can model these complex domains expressively and use the DB only for persistence. Admittedly it does have downsides, like having to write your own migration layer, but for this use-case the benefits outweigh the pains. PS.: We do use Postgres though ;) as it has 1st class json support and at the same time we have the luxury of using its relational capabilities where needed (think of a hybrid model).
  - speedplane 8y ago
    Maybe I’m a rare exception, but I chose NoSQL early on when it was still “hot” and have never looked back. We’ve grown from a couple megabytes of data do several dozen terabytes and have had countless issues, but scaling our database was never one of them.
  - twic 8y ago
    Well, happily, those days are gone for good. Who needs NoSQL when you have the blockchain!
    - ixtli 8y ago
      Glorious, cursed comment.
- deleted 8y ago
  [deleted]
- bostonpete 8y ago
  Wish we had HN gold for comments like this.
Bucephalus355 8y ago
Wow was MongoDB really that bad?
- emmanueloga_ 8y ago
  Possible alternative headline: "Company weights in trade-offs of having two different database systems, decide to consolidate into a single one." :-) They don't give too many reasons, but say that operationally it is gonna be easier for them to work with a single database than with two (no-brainer?)
  - dralley 8y ago
    As someone on the team, that didn't factor into the decision too much, although it's definitely a nice perk. Switching to Postgres made a lot of sense even when looking at Pulp as a standalone project rather than as a part of Satellite.
  - taude 8y ago
    Except, I'd argue that most large distributed systems these days are using much more than two types of data stores, especially when you start including name/value keystores (and in memory caching variants), big data pipelines, transactional data, "unstructured" data storage (like Mongo), time series, etc. I work in FinTech and you'll see several backing our apps, and this isn't uncommon.
  - zaphirplane 8y ago
    The licensing change surely has to have played a factor
- foobiekr 8y ago
  A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things like typos, etc. become impossible. Relational structures enforced in the database (and imho sadly lacking in the intervening layers which is something I’d like to get the time to deal with; though the FoubdationDB record layer certainly has an interesting if unexploited take on it) is a natural extension of this. But there’s a whole other world where people just don’t want to be tied to constraints. I suspect it’s a personality thing.
nodesocket 8y ago
This is a dead horse that's been beaten to death over and over again here on HN. MongoDB is garbage. MongoDB doesn't scale. MongoDB is $h!t says HN users. It just feels like an echo chamber. Are you running the latest version of MongoDB in a replica set with journaling enabled and write concern set to one? MongoDB has worked great for my uses, up to moderate write/read scale. Sure, if you are running "big data" or enterprise things, it might not be the best choice, but it's not the steaming pile of horse excrement that some HN users try to make it. Second, for those who still insist that MongoDB is crap, what is the best pure document store database then? I used to champion RethinkDB, but they failed and development has basically stopped. You wouldn't build a business on RethinkDB now a days unfortunately.
- AznHisoka 8y ago
  "Are you running the latest version of MongoDB?" I don't use MongoDB, but I imagine the answer to that question for most people is: No, because I already have an existing product in production with actual customers, and I can't afford to upgrade to the latest version.
  - nodesocket 8y ago
    With a replica set you should be able to do the update with zero or little downtime with some planning. Of course you'll have to look and watch for breaking changes if you running a really old version.
    - AznHisoka 8y ago
      It's not just the downtime. It's the actual work involved in running all your regressions to ensure backward compatability. ie. do all your existing queries still work? do all your current backup processes still work? did that arcane config variable you rely on change its stinking name?
      - enraged_camel 8y ago
        I was going to respond with “well, any database that can’t guarantee that kind of backwards compatibility shouldn’t be taken seriously,” but then I realized we are talking about MongoDB and I found myself giggling.
Thaxll 8y ago
Your comment makes no sense since RH is moving for reasons different that "MongoDB is bad".
- jayd16 8y ago
  Not being worth the effort isn't much better.
  - zaphirplane 8y ago
    You are assuming it’s the technology I’m assuming it’s the licensing change.
    - dralley 8y ago
      As someone on the team, it's not the licensing change. The post is correct when it says we've been slowly working on this since 2016. The license change certainly validates the decision, but it was made for purely technical reasons.
fred_is_fred 8y ago
"Web scale"
nf05papsjfVbc 8y ago
The intensity and frequency of frustration are quite likely to subside if you are able to accept these: - humans do not always act in their long term best interests - humans do not always act in a rational manner - often the former is indistinguishable from the cases where you are not aware of someone's incentives - true change requires that people really accept it internally (and not just mere commitment to act differently)

by alrs 8y ago

bubaflub 8y ago

Smug self-righteousness

toomuchtodo 8y ago
I’ll allow it. Years later, I’m still miffed at Graylog (centralized logging engine) for having required MongoDB for a small bit of auth and meta storage that could’ve easily been done in MySQL or PostgreSQL (RDS even), forcing the need for that much more ops work for a small Mongo cluster for HA. Everyone deprecating the use of Mongo is a welcoming turn of events. I shall recall these dark days to the next generation as “NoSQL Madness”, or more colloquially, “my schema is my app layer”.
- Bucephalus355 8y ago
  Can you explain in more detail about “my schema is my app layer”? EDIT: fixing autocorrect
  - toomuchtodo 8y ago
    MongoDB by default is (was? It’s been a while since I’ve used it) schemaless, which means all of your data validation must take place in your app instead of the database. Your data integrity is then only as good as your weakest validation. Edit: scheme/schema autocorrect typos corrected. Thanks!
    - taspeotis 8y ago
      > my scheme is my app layer ... MongoDB by default is (was? It’s been a while since I’ve used it) schemeless https://www.google.com/search?q=define%3Aschema https://www.google.com/search?q=define%3Aschema
      - deleted 8y ago
        [deleted]
    - chisleu 8y ago
      Right, but most people using it use model or data repository patterns to ensure correctness. It does offer nearly infinite flexibility provided you use it correctly. You can add fields without any sort of DB work, you just start adding fields to rows as needed and let it catch up organically. There are use cases where mongo makes a lot of sense. It's very popular in the node.js / RAD world for sure. I certainly have never been a huge fan by any means. Only relatively recently did they solve distributed writes.
      - toomuchtodo 8y ago
        Unfortunately, I’d argue PostgreSQL gets you all the same benefits with JSON storage (fairly equivalent to Mongo docs), while also giving you all the goodness of a relational, transactional, schema enforcing RDBMS. PGSQL became Mongo faster than Mongo could become PGSQL.
        WrtCdEvrydy 8y ago
        This is the same thing that happened in Java. Other languages started prototyping features... that eventually just end up being implemented in Java.
        int_19h 8y ago
        Java did it way too slow, and that is a significant contributor to it being relegated to "legacy" in many areas. If it waited for the other languages to prototype stuff, it might have not been the case. The problem is that it waited for them to prototype it, refine it, release it, popularize it, and for their community to adopt it, before even starting to work on it in Java - which means that by the time they had it, most people who needed it were already elsewhere (not necessarily off JVM, just another language). Lambdas were a very good example - if you look at the closest competitor, C#, it got the first take on them back in 2005. Then a major refinement in 2008, adding type inference. By 2010, lambdas were idiomatic in C#. Java, in contrast, released the first version in 2014. And even then, they're still less powerful.
- throwaway5752 8y ago
  "my scheme is my app layer" To be fair you can do use schema validators in Mongo. Not sure it's widespread in practice. And there are other distributed databases that aren't document stores that have schemas and various subsets of SQL implemented.
  - deleted 8y ago
    [deleted]
- bane 8y ago
  I once worked at a place that, at some point in the past, had developed some kind of semantic graph database that never gained market traction. The author of it had ended up as CTO and kept seeking out uses for his work and ended up finding all kinds of odd places for it to live, including as the auth database in a large scale document analytics system and running part of the payroll. We were constantly running into all kinds of scalability issues where this 12 year old component was often the source of our pain, but he'd never even entertain a conversation to eliminate it and consolidate or replace it with something better.
  - seibelj 8y ago
    This is the definition of software engineering hubris distilled into a few paragraphs
  - duxup 8y ago
    I worked for a company that was old but had a fancy new product. It was just about to be released when.... A competitor bought us for stupid money. They fired most folks (not my small department) had their CTO decide between our fancy new product, and the product he created. There was no question, our product was amazing, his Frankenstein was two pieces of equipment cabled together in three items the footprint ... and still did less, and was quite a ways away from being "ready". If there was a silver lining, the Frankenstein product doomed that company and we got bought by the far more competent competitor they had.
    - qbaqbaqba 8y ago
      Happy ending.
      - duxup 8y ago
        It's surprising how that works. I've been through numerous mergers, acquisitions, been in the company acquiring a company. Lots of hair pulling and I can say that it was really hard to predict the outcome for any individual positive or negative in every case until long afterward.
- djrobstep 8y ago
  My pet theory is that NoSQL took off purely because people were sick of having to manage schema changes. Unfortunately, people reacted to being (justifiably) frustrated with schemas by throwing strict schemas out entirely, instead of making better schema management/migration tools.
  - staticassertion 8y ago
    Tell that to AWS. They've banned relational databases for specific workloads because Dynamo (nosql) provides more consistent performance, and is easier to operate. Tons of conflation of Mongo's problems with those of nosql in this thread.
    - djrobstep 8y ago
      Your average dev is not making decisions based on what might work best for one particular problem Amazon has.
      - staticassertion 8y ago
        Do you think all of the companies that chose Cassandra and Dynamo were wrong to do so? There's no use case for NoSQL? There were no lessons learned, value adds from NoSQL? How do you explain the 'NewSQL' approach, which seems to be so clearly borne of what we've learned from NoSQL? It should be obvious that NoSQL has value, regardless of the issues with one of the earlier NoSQL DBs.
        pjmlp 8y ago
        I don't see a value other than fashion driven development, specially when comparing the bare bones browser GUI for Dynamo with something like SQL Server Management Studio or that whole story with primary and secondary indexes, with prices being set by index usage.
        menacingly 8y ago
- baroffoos 8y ago
  Setting up graylog was one of the worst mistakes I made. It took forever to get all the required software installed and configured and then it was taking up all the ram on the server doing fuck all.
  - SteveNuts 8y ago
    That was just Java doing Java things. Business as usual.
    - RhodesianHunter 8y ago
      This is a nonsense comment doing language troll things.
  - deleted 8y ago
    [deleted]
  - atm0sphere 8y ago
    There were single script 1-click installs for it in bash all over for me..but yeah jvm is a hog.
    - yjftsjthsd-h 8y ago
      My experience with those one-liner installs is that they usually work... strictly speaking. They don't scale, they don't deal with edge cases, they know nothing of your environment. They install one piece of software (in an "interesting" way that won't upgrade), and that's it.
      - sunnymanokl 8y ago
        My last pay check was $9500 working 12 hours a week online. My sisters friend has been averaging 15k for months now and she works about 20 hours a week. I can't believe how easy it was once I tried it out. This is what I do, HERE►► http://www.worktoday33.com http://www.worktoday33.com
      - 8y ago
- Sandra_lisa4 8y ago
  XzXzXZXX
- dominotw 8y ago
  It was just a discovery phase.
- pwm 8y ago
  I agree with the general sentiment just wanted to point out that "my schema is my app layer" has some valid use-cases. At my current job we deal with highly complex schemas (modelling insurance contracts). Correctness is paramount but at the same time you need flexibility in terms of change over time. Defining these schemas at the DB level would be painful, a bit like writing web apps in assembly. Languages like Haskell (which we use) help here with their rich and expressive typing capabilities, so we can model these complex domains expressively and use the DB only for persistence. Admittedly it does have downsides, like having to write your own migration layer, but for this use-case the benefits outweigh the pains. PS.: We do use Postgres though ;) as it has 1st class json support and at the same time we have the luxury of using its relational capabilities where needed (think of a hybrid model).
- speedplane 8y ago
  Maybe I’m a rare exception, but I chose NoSQL early on when it was still “hot” and have never looked back. We’ve grown from a couple megabytes of data do several dozen terabytes and have had countless issues, but scaling our database was never one of them.
- twic 8y ago
  Well, happily, those days are gone for good. Who needs NoSQL when you have the blockchain!
  - ixtli 8y ago
    Glorious, cursed comment.
deleted 8y ago
[deleted]
bostonpete 8y ago
Wish we had HN gold for comments like this.

Bucephalus355 8y ago

Wow was MongoDB really that bad?

emmanueloga_ 8y ago
Possible alternative headline: "Company weights in trade-offs of having two different database systems, decide to consolidate into a single one." :-) They don't give too many reasons, but say that operationally it is gonna be easier for them to work with a single database than with two (no-brainer?)
- dralley 8y ago
  As someone on the team, that didn't factor into the decision too much, although it's definitely a nice perk. Switching to Postgres made a lot of sense even when looking at Pulp as a standalone project rather than as a part of Satellite.
- taude 8y ago
  Except, I'd argue that most large distributed systems these days are using much more than two types of data stores, especially when you start including name/value keystores (and in memory caching variants), big data pipelines, transactional data, "unstructured" data storage (like Mongo), time series, etc. I work in FinTech and you'll see several backing our apps, and this isn't uncommon.
- zaphirplane 8y ago
  The licensing change surely has to have played a factor
foobiekr 8y ago
A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things like typos, etc. become impossible. Relational structures enforced in the database (and imho sadly lacking in the intervening layers which is something I’d like to get the time to deal with; though the FoubdationDB record layer certainly has an interesting if unexploited take on it) is a natural extension of this. But there’s a whole other world where people just don’t want to be tied to constraints. I suspect it’s a personality thing.

nodesocket 8y ago

This is a dead horse that's been beaten to death over and over again here on HN. MongoDB is garbage. MongoDB doesn't scale. MongoDB is $h!t says HN users. It just feels like an echo chamber. Are you running the latest version of MongoDB in a replica set with journaling enabled and write concern set to one? MongoDB has worked great for my uses, up to moderate write/read scale. Sure, if you are running "big data" or enterprise things, it might not be the best choice, but it's not the steaming pile of horse excrement that some HN users try to make it. Second, for those who still insist that MongoDB is crap, what is the best pure document store database then? I used to champion RethinkDB, but they failed and development has basically stopped. You wouldn't build a business on RethinkDB now a days unfortunately.

AznHisoka 8y ago
"Are you running the latest version of MongoDB?" I don't use MongoDB, but I imagine the answer to that question for most people is: No, because I already have an existing product in production with actual customers, and I can't afford to upgrade to the latest version.
- nodesocket 8y ago
  With a replica set you should be able to do the update with zero or little downtime with some planning. Of course you'll have to look and watch for breaking changes if you running a really old version.
  - AznHisoka 8y ago
    It's not just the downtime. It's the actual work involved in running all your regressions to ensure backward compatability. ie. do all your existing queries still work? do all your current backup processes still work? did that arcane config variable you rely on change its stinking name?
    - enraged_camel 8y ago
      I was going to respond with “well, any database that can’t guarantee that kind of backwards compatibility shouldn’t be taken seriously,” but then I realized we are talking about MongoDB and I found myself giggling.

Thaxll 8y ago

Your comment makes no sense since RH is moving for reasons different that "MongoDB is bad".

jayd16 8y ago
Not being worth the effort isn't much better.
- zaphirplane 8y ago
  You are assuming it’s the technology I’m assuming it’s the licensing change.
  - dralley 8y ago
    As someone on the team, it's not the licensing change. The post is correct when it says we've been slowly working on this since 2016. The license change certainly validates the decision, but it was made for purely technical reasons.

fred_is_fred 8y ago

"Web scale"

nf05papsjfVbc 8y ago

The intensity and frequency of frustration are quite likely to subside if you are able to accept these: - humans do not always act in their long term best interests - humans do not always act in a rational manner - often the former is indistinguishable from the cases where you are not aware of someone's incentives - true change requires that people really accept it internally (and not just mere commitment to act differently)