20 ms·

Red Hat Satellite to standardize on PostgreSQL backend

420 points by pplonski86 8y ago

alrs 8y ago
Is there a word for a case of "I told you so" that went on for so long that it curdled from frustration, to despair, to cynicism, to a realignment of your understanding of the human project as something barely capable of tying its own shoes and making it out to the mailbox and back?
- bubaflub 8y ago
  Smug self-righteousness
  - toomuchtodo 8y ago
    I’ll allow it. Years later, I’m still miffed at Graylog (centralized logging engine) for having required MongoDB for a small bit of auth and meta storage that could’ve easily been done in MySQL or PostgreSQL (RDS even), forcing the need for that much more ops work for a small Mongo cluster for HA. Everyone deprecating the use of Mongo is a welcoming turn of events. I shall recall these dark days to the next generation as “NoSQL Madness”, or more colloquially, “my schema is my app layer”.
    - Bucephalus355 8y ago
      Can you explain in more detail about “my schema is my app layer”? EDIT: fixing autocorrect
      - toomuchtodo 8y ago
        MongoDB by default is (was? It’s been a while since I’ve used it) schemaless, which means all of your data validation must take place in your app instead of the database. Your data integrity is then only as good as your weakest validation. Edit: scheme/schema autocorrect typos corrected. Thanks!
        taspeotis 8y ago
        > my scheme is my app layer ... MongoDB by default is (was? It’s been a while since I’ve used it) schemeless https://www.google.com/search?q=define%3Aschema https://www.google.com/search?q=define%3Aschema
        deleted 8y ago
        [deleted]
        chisleu 8y ago
        Right, but most people using it use model or data repository patterns to ensure correctness. It does offer nearly infinite flexibility provided you use it correctly. You can add fields without any sort of DB work, you just start adding fields to rows as needed and let it catch up organically. There are use cases where mongo makes a lot of sense. It's very popular in the node.js / RAD world for sure. I certainly have never been a huge fan by any means. Only relatively recently did they solve distributed writes.
        toomuchtodo 8y ago
        Unfortunately, I’d argue PostgreSQL gets you all the same benefits with JSON storage (fairly equivalent to Mongo docs), while also giving you all the goodness of a relational, transactional, schema enforcing RDBMS. PGSQL became Mongo faster than Mongo could become PGSQL.
        WrtCdEvrydy 8y ago
        This is the same thing that happened in Java. Other languages started prototyping features... that eventually just end up being implemented in Java.
        int_19h 8y ago
        Java did it way too slow, and that is a significant contributor to it being relegated to "legacy" in many areas. If it waited for the other languages to prototype stuff, it might have not been the case. The problem is that it waited for them to prototype it, refine it, release it, popularize it, and for their community to adopt it, before even starting to work on it in Java - which means that by the time they had it, most people who needed it were already elsewhere (not necessarily off JVM, just another language). Lambdas were a very good example - if you look at the closest competitor, C#, it got the first take on them back in 2005. Then a major refinement in 2008, adding type inference. By 2010, lambdas were idiomatic in C#. Java, in contrast, released the first version in 2014. And even then, they're still less powerful.
    - throwaway5752 8y ago
      "my scheme is my app layer" To be fair you can do use schema validators in Mongo. Not sure it's widespread in practice. And there are other distributed databases that aren't document stores that have schemas and various subsets of SQL implemented.
      - deleted 8y ago
        [deleted]
    - bane 8y ago
      I once worked at a place that, at some point in the past, had developed some kind of semantic graph database that never gained market traction. The author of it had ended up as CTO and kept seeking out uses for his work and ended up finding all kinds of odd places for it to live, including as the auth database in a large scale document analytics system and running part of the payroll. We were constantly running into all kinds of scalability issues where this 12 year old component was often the source of our pain, but he'd never even entertain a conversation to eliminate it and consolidate or replace it with something better.
      - seibelj 8y ago
        This is the definition of software engineering hubris distilled into a few paragraphs
      - duxup 8y ago
        I worked for a company that was old but had a fancy new product. It was just about to be released when.... A competitor bought us for stupid money. They fired most folks (not my small department) had their CTO decide between our fancy new product, and the product he created. There was no question, our product was amazing, his Frankenstein was two pieces of equipment cabled together in three items the footprint ... and still did less, and was quite a ways away from being "ready". If there was a silver lining, the Frankenstein product doomed that company and we got bought by the far more competent competitor they had.
        qbaqbaqba 8y ago
        Happy ending.
        duxup 8y ago
        It's surprising how that works. I've been through numerous mergers, acquisitions, been in the company acquiring a company. Lots of hair pulling and I can say that it was really hard to predict the outcome for any individual positive or negative in every case until long afterward.
    - djrobstep 8y ago
      My pet theory is that NoSQL took off purely because people were sick of having to manage schema changes. Unfortunately, people reacted to being (justifiably) frustrated with schemas by throwing strict schemas out entirely, instead of making better schema management/migration tools.
      - staticassertion 8y ago
        Tell that to AWS. They've banned relational databases for specific workloads because Dynamo (nosql) provides more consistent performance, and is easier to operate. Tons of conflation of Mongo's problems with those of nosql in this thread.
        djrobstep 8y ago
        Your average dev is not making decisions based on what might work best for one particular problem Amazon has.
        staticassertion 8y ago
        Do you think all of the companies that chose Cassandra and Dynamo were wrong to do so? There's no use case for NoSQL? There were no lessons learned, value adds from NoSQL? How do you explain the 'NewSQL' approach, which seems to be so clearly borne of what we've learned from NoSQL? It should be obvious that NoSQL has value, regardless of the issues with one of the earlier NoSQL DBs.
        pjmlp 8y ago
        I don't see a value other than fashion driven development, specially when comparing the bare bones browser GUI for Dynamo with something like SQL Server Management Studio or that whole story with primary and secondary indexes, with prices being set by index usage.
        menacingly 8y ago
    - baroffoos 8y ago
      Setting up graylog was one of the worst mistakes I made. It took forever to get all the required software installed and configured and then it was taking up all the ram on the server doing fuck all.
      - SteveNuts 8y ago
        That was just Java doing Java things. Business as usual.
        RhodesianHunter 8y ago
        This is a nonsense comment doing language troll things.
      - deleted 8y ago
        [deleted]
      - atm0sphere 8y ago
        There were single script 1-click installs for it in bash all over for me..but yeah jvm is a hog.
        yjftsjthsd-h 8y ago
        My experience with those one-liner installs is that they usually work... strictly speaking. They don't scale, they don't deal with edge cases, they know nothing of your environment. They install one piece of software (in an "interesting" way that won't upgrade), and that's it.
        sunnymanokl 8y ago
        My last pay check was $9500 working 12 hours a week online. My sisters friend has been averaging 15k for months now and she works about 20 hours a week. I can't believe how easy it was once I tried it out. This is what I do, HERE►► http://www.worktoday33.com http://www.worktoday33.com
        8y ago
    - Sandra_lisa4 8y ago
      XzXzXZXX
    - dominotw 8y ago
      It was just a discovery phase.
    - pwm 8y ago
      I agree with the general sentiment just wanted to point out that "my schema is my app layer" has some valid use-cases. At my current job we deal with highly complex schemas (modelling insurance contracts). Correctness is paramount but at the same time you need flexibility in terms of change over time. Defining these schemas at the DB level would be painful, a bit like writing web apps in assembly. Languages like Haskell (which we use) help here with their rich and expressive typing capabilities, so we can model these complex domains expressively and use the DB only for persistence. Admittedly it does have downsides, like having to write your own migration layer, but for this use-case the benefits outweigh the pains. PS.: We do use Postgres though ;) as it has 1st class json support and at the same time we have the luxury of using its relational capabilities where needed (think of a hybrid model).
    - speedplane 8y ago
      Maybe I’m a rare exception, but I chose NoSQL early on when it was still “hot” and have never looked back. We’ve grown from a couple megabytes of data do several dozen terabytes and have had countless issues, but scaling our database was never one of them.
    - twic 8y ago
      Well, happily, those days are gone for good. Who needs NoSQL when you have the blockchain!
      - ixtli 8y ago
        Glorious, cursed comment.
  - deleted 8y ago
    [deleted]
  - bostonpete 8y ago
    Wish we had HN gold for comments like this.
- Bucephalus355 8y ago
  Wow was MongoDB really that bad?
  - emmanueloga_ 8y ago
    Possible alternative headline: "Company weights in trade-offs of having two different database systems, decide to consolidate into a single one." :-) They don't give too many reasons, but say that operationally it is gonna be easier for them to work with a single database than with two (no-brainer?)
    - dralley 8y ago
      As someone on the team, that didn't factor into the decision too much, although it's definitely a nice perk. Switching to Postgres made a lot of sense even when looking at Pulp as a standalone project rather than as a part of Satellite.
    - taude 8y ago
      Except, I'd argue that most large distributed systems these days are using much more than two types of data stores, especially when you start including name/value keystores (and in memory caching variants), big data pipelines, transactional data, "unstructured" data storage (like Mongo), time series, etc. I work in FinTech and you'll see several backing our apps, and this isn't uncommon.
    - zaphirplane 8y ago
      The licensing change surely has to have played a factor
  - foobiekr 8y ago
    A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things like typos, etc. become impossible. Relational structures enforced in the database (and imho sadly lacking in the intervening layers which is something I’d like to get the time to deal with; though the FoubdationDB record layer certainly has an interesting if unexploited take on it) is a natural extension of this. But there’s a whole other world where people just don’t want to be tied to constraints. I suspect it’s a personality thing.
- nodesocket 8y ago
  This is a dead horse that's been beaten to death over and over again here on HN. MongoDB is garbage. MongoDB doesn't scale. MongoDB is $h!t says HN users. It just feels like an echo chamber. Are you running the latest version of MongoDB in a replica set with journaling enabled and write concern set to one? MongoDB has worked great for my uses, up to moderate write/read scale. Sure, if you are running "big data" or enterprise things, it might not be the best choice, but it's not the steaming pile of horse excrement that some HN users try to make it. Second, for those who still insist that MongoDB is crap, what is the best pure document store database then? I used to champion RethinkDB, but they failed and development has basically stopped. You wouldn't build a business on RethinkDB now a days unfortunately.
  - AznHisoka 8y ago
    "Are you running the latest version of MongoDB?" I don't use MongoDB, but I imagine the answer to that question for most people is: No, because I already have an existing product in production with actual customers, and I can't afford to upgrade to the latest version.
    - nodesocket 8y ago
      With a replica set you should be able to do the update with zero or little downtime with some planning. Of course you'll have to look and watch for breaking changes if you running a really old version.
      - AznHisoka 8y ago
        It's not just the downtime. It's the actual work involved in running all your regressions to ensure backward compatability. ie. do all your existing queries still work? do all your current backup processes still work? did that arcane config variable you rely on change its stinking name?
        enraged_camel 8y ago
        I was going to respond with “well, any database that can’t guarantee that kind of backwards compatibility shouldn’t be taken seriously,” but then I realized we are talking about MongoDB and I found myself giggling.
- Thaxll 8y ago
  Your comment makes no sense since RH is moving for reasons different that "MongoDB is bad".
  - jayd16 8y ago
    Not being worth the effort isn't much better.
    - zaphirplane 8y ago
      You are assuming it’s the technology I’m assuming it’s the licensing change.
      - dralley 8y ago
        As someone on the team, it's not the licensing change. The post is correct when it says we've been slowly working on this since 2016. The license change certainly validates the decision, but it was made for purely technical reasons.
- fred_is_fred 8y ago
  "Web scale"
- nf05papsjfVbc 8y ago
  The intensity and frequency of frustration are quite likely to subside if you are able to accept these: - humans do not always act in their long term best interests - humans do not always act in a rational manner - often the former is indistinguishable from the cases where you are not aware of someone's incentives - true change requires that people really accept it internally (and not just mere commitment to act differently)
lerax 8y ago
Indeed a sane choice
SlowRobotAhead 8y ago
Does someone have the time explain what NoSQL Mongo didn’t do that PostgreSQL does? I guess I don’t really understand the structured SQL vs NoSQL when you aren’t accessing directly but via a program. I also don’t understand why they would have supported a NoSQL and object-relational at the same time at any point.
- MBCook 8y ago
  From reading the post it sounds like they want transactions for some of the functionality they’re trying to build. I guess Mongo still doesn’t have those.
  - d4l3k 8y ago
    They recently added multi document transactions though. https://docs.mongodb.com/manual/core/transactions/ https://docs.mongodb.com/manual/core/transactions/
    - cutety 8y ago
      This may just be my bias against NoSQL (I’ll qualify this with I don’t believe it should never be used, but 90% of use cases psql/MySQL/etc is likely the better choice these days), but when it comes to choosing a DB engine when you want ACID transactions, I’d pick the one that was built to handle ACID transactions from the start, rather than one that just added it with this large caveat in the middle of the docs: > In most cases, multi-document transaction incurs a greater performance cost over single document writes, and the availability of multi-document transaction should not be a replacement for effective schema design. For many scenarios, the denormalized data model (embedded documents and arrays) will continue to be optimal for your data and use cases. That is, for many scenarios, modeling your data appropriately will minimize the need for multi-document transactions. So, to be able to have one of the core features of SQL, you lose the biggest feature of NoSQL which is super fast writes. Which, they try to remind you that if you model your data in correctly, by creating denormalized documents with embedded data, you probably won’t need them anyway. Which just makes me cringe at having to maintain these huge unstructured documents just so you get faster writes since you don’t have to worry about ACID. Or, you can use Postgres, which was built with ACID from the start, using technology and design patterns proven over decades, and has support for JSONB if you need unstructured/document storage that you can also (fairly) efficiently query. Maybe I’m just woefully uninformed, but I just can’t imagine a use case for Mongo unless your dealing with Google level data/traffic, and fortunately in their case, they have enough money to hire enough people that actually know how & when to use Mongo effectively to the point that it’s not just a fancy way to write to /dev/null. I’d actually be really interested in some non-Google scale use cases for Mongo, like where Postgres’ performance was actually an issue, and how much Mongo actually outperforms it, and the trade offs/issues switching. Most posts I’ve seen are of the opposite migration, but I want to see what all the hype for a DB engine (what I’ve always saw as one of the less _sexy_ areas of CS, at least in marketability, compared to ML/AI or Programing Language/Compilers)
bane 8y ago
Rule #1: Always start with PostgreSQL unless you have a very compelling reason not to.
- revskill 8y ago
  This reminds me of: Always bet on Javascript unless you have a very compelling reason not to.
- liquidise 8y ago
  "Very compelling reasons" means different things to different people. Which i suppose is a catch 22. The young but introspective dev may make the wrong call on a side project and learn to stick to psql in the future. The dev who is convinced "web scale" will arrive before their first AWS bill and picks Mongo as their production database will have a worse fate. As will their team and company by proxy. Rule #1: Always start with PostgreSQL. You don't have a compelling reason not to.
  - webninja 8y ago
    Is there a nice web interface that is made for PostgreSQL? I’m in love with the power that PHPMyAdmin brings to MySQL. Also there are DigitalOcean Ubuntu droplet images with it all pre-configured.
    - h1d 8y ago
      It always feels way snappier to use a local GUI connected through SSH than PhpMyAdmin and its interface also feels very cluttered to me.
- int_19h 8y ago
  Always start with in-memory data structure with straightforward persistence (i.e. load and save it all at once, using some popular format). If you need ACID, then start with SQLite. If you need scalability as well, then PostgreSQL. Most other features aren't worth the hassle of configuration compared to a single file on disk and a library to link to. (But I suspect that most NoSQL apps these days would do just fine with SQLite, perf-wise. As an industry, we don't have a good sense of scale.)
chatterbeak 8y ago
But MongoDb is web scale! How are they going to scale on PostgrSQL?
hartator 8y ago
Not sure I understand the MongoDB hate here. Migrations and denomalization is not somthing you have to deal with in MongoDB.
- Benjamin_Dobell 8y ago
  You almost certainly still need to worry about migrations. MongoDB can store data in a schemaless fashion, however, chances are your application has a data model (schema). Sure, you can avoid migrations and just tack on more and more backwards compatibility for old data layouts. But performance suffers, and chances are that code quality and correctness suffer too.
  - __david__ 8y ago
    From a certain perspective, adding backwards compatibility for old data layouts is a migration. A migration that gets run every the data is loaded instead of once during a classic db migration. This is inefficient in the long run.
  - hartator 8y ago
    You have to deal migrations maybe 1/20 the times than with SQL. Like no migrations for new tables, new db, and new columns. Column name changes can be handled on the application side or yes with a migration.
- world32 8y ago
  The migration comment has already been addressed. "Denormalization is not somthing you have to deal with in MongoDB." What do you mean exactly? If you have a piece of data, lets say an address, that exists in more than one place as is the mongodb way, you will absolutely have to deal with the fact that you need to now update that data multiple times. So denormalization is very mcuh some you have to deal with in Mongodb. Just like in a relational database, denormalization has serious consequences and should only be used when the need for performance outweighs the risks of having much more complex code and hence potentially many more hard-to-debug errors.
wcchandler 8y ago
Any guesses on a timeline? Satellite 6.6? Or will it be in the next major version? 7.0?
- dralley 8y ago
  Pulp 2 will continue using MongoDB until EOL. Pulp 3, which is still in development, is using Postgres -- but while the platform functionality is nearly done the plugins for e.g. RPM and Docker are not yet near feature parity.
Sandra_lisa4 8y ago
zxc
peteridah 8y ago
I am quite surprised that companies still use Red Hat Satellite Server. In 2010 while I was still at Red Hat, deploying and managing it was the mainstay of the consulting business, and it was based on Oracle RAC. It was to my mind already legacy software at that time.
- lunchables 8y ago
  What are the good alternatives for managing your Red Hat servers?
- ricklepick 8y ago
  Satellite of today is not Satellite of 2010 by any stretch. Satellite 6 is based on foreman + katello + pulp
  - dralley 8y ago
    And, for anyone who has experience with Satellite 6.0/6.1... it has gotten much, much, much better since then.
BossingAround 8y ago
A silly, genuine question: when I'm writing a NodeJS app, and I need to store a JSON in a DB that looks like this: { "a" : "b", "c" : { "ca":"cb", "cd":"ce" }, "d": { ... } } How do I store it? To me, NoSQL seemed like the choice in the past. This is not difficult in relational DBs, but it requires rather long SQL commands, and SQL table design. It seems to me that "MongoDB.insert(obj)" seems like the way to go, as it is much simpler. How do experienced Node devs solve this issue? Is it simply relational all the way? Is it Postgress's JSON field..?
- sandGorgon 8y ago
  postgres has a jsonb datatype that works pretty much like mongodb. https://blog.codeship.com/unleash-the-power-of-storing-json-in-postgres/ https://blog.codeship.com/unleash-the-power-of-storing-json-...
  - willvarfar 8y ago
    And other regular RDBMS like MariaDB and MySQL have much the same functionality. I store JSON in RDBMS all the time.
    - sandGorgon 8y ago
      interesting - i never knew that. I use postgresql, so dont have first hand knowledge, but i saw this https://tableplus.io/blog/2018/09/mariadb-vs-postgresql-a-quick-comparison.html https://tableplus.io/blog/2018/09/mariadb-vs-postgresql-a-qu... > JSON support: PostgreSQL supports JSON and JSONB while MariaDB doesn’t. It supports an alias for JSON instead, which is a LONGTEXT column. Even the proposed json column, is not indexed - postgresql jsonb is indexed and queries fast.
      - willvarfar 8y ago
        Yeah MariaDB got 'dynamic columns' first, which iirc supported key values but not arrays, and then MySQL added a more complete 'json' data type, so the exact functionality differs by flavour and version. The postgres support is simply nicer to use and more powerful, e.g. the indexing. But if your RDBMS is any fairly-recent version of MariaDB or MySQL, you can use JSON too. Of the top of my head, the thing I find most griping about the MySQL syntax is you have to do thing->>"$.key" instead of postgres-style thing->>'key' Whereas what I really want to be able to do is thing.key But most of the consumers of these modern 'dynamic columns' are apps and then it all really matters less if you don't spend all your days at the sql prompt writing stuff by hand.
fooblat 8y ago
I'm so out of date with Satellite server that my first reaction was: Wow it took this long to get oracle out!
dmoreno 8y ago
This sentence, when talking about support for old enterprise versions, is a bit suspicious: > Satellite will not use newer versions of MongoDB that are licensed under SSPL. So my read is that there is not only an intention of database consolidation, but also a desire of avoiding the SSPL.
- amyjess 8y ago
  Nothing "suspicious" about it. Red Hat also explicitly evicted Mongo from RHEL 8 because of the SSPL [0], and they have a longstanding policy of only shipping OSS. It's no secret, and they've been very upfront about not shipping anything SSPL. [0] Previous HN discussion: https://news.ycombinator.com/item?id=18919543 https://news.ycombinator.com/item?id=18919543
AzzieElbab 8y ago
I am no fun of MongoDB, but I must admit it has gotten a lot better in the past few years. So, why drop it now?
- nailer 8y ago
  New license that's not OSS (MongoDB Inc saying it is doesn't make it so, the OSD determines OSS, not some company).
  - ricklepick 8y ago
    The upstream project has been pretty vocal about their reasons to switch, and it's not because of the license. MongoDB just isn't the right tool for their use case, despite valiant attempts to utilize it.
    - VWWHFSfQ 8y ago
      This is very clearly because of licensing.
linkmotif 8y ago
I’ve never enjoyed Mongo for a second, not conceptually, not practically. But I don’t understand the people who act like Postgres is the end of everything. SQL never felt right for me either. It’s a fine language for business analysis, I guess, but is it actually pleasant for development? Do all of you who so enthusiastically post that it should be the first tool you reach for really never experience object-relational impedance? Do you really enjoy modeling like this?
- acdha 8y ago
  Yes: what you’re really doing is documenting your data structures and access patterns, which is way less work to do consciously rather than trying to bolt it on ad hoc. A really big thing is normalization which some people tend to downplay until they’ve had to write code to recurse complex structures enforcing consistency or making global changes. Similarly, atomicity and isolation are really useful characteristics to be able to take for granted without having to code around the problem everywhere. The usual arc I've seen for document stores, ISAMs, etc. is that people say this is great based on the first 20 minutes and then about a year later realize that they've spent thousands of lines of code implementing a really clunky subset of what they'd get out of the box with a SQL database and the believed ease of use or performance benefits were far less dramatic than promised or even negative.
  - linkmotif 8y ago
    Right, Postgres is better than document stores, but is it actually pleasant to store your data in square tables and then process it to runtime objects from that? Even with battle tested ORMs, it’s never felt right for me.
    - chousuke 8y ago
      I find it very pleasant, but I also tend to avoid ORMs. I can just write SQL queries to get me the data I need in any form I want, and it's not always useful to stuff that into objects when I can just process it as is.

420 points by pplonski86 8y ago

alrs 8y ago

Is there a word for a case of "I told you so" that went on for so long that it curdled from frustration, to despair, to cynicism, to a realignment of your understanding of the human project as something barely capable of tying its own shoes and making it out to the mailbox and back?

bubaflub 8y ago
Smug self-righteousness
- toomuchtodo 8y ago
  I’ll allow it. Years later, I’m still miffed at Graylog (centralized logging engine) for having required MongoDB for a small bit of auth and meta storage that could’ve easily been done in MySQL or PostgreSQL (RDS even), forcing the need for that much more ops work for a small Mongo cluster for HA. Everyone deprecating the use of Mongo is a welcoming turn of events. I shall recall these dark days to the next generation as “NoSQL Madness”, or more colloquially, “my schema is my app layer”.
  - Bucephalus355 8y ago
    Can you explain in more detail about “my schema is my app layer”? EDIT: fixing autocorrect
    - toomuchtodo 8y ago
      MongoDB by default is (was? It’s been a while since I’ve used it) schemaless, which means all of your data validation must take place in your app instead of the database. Your data integrity is then only as good as your weakest validation. Edit: scheme/schema autocorrect typos corrected. Thanks!
      - taspeotis 8y ago
        > my scheme is my app layer ... MongoDB by default is (was? It’s been a while since I’ve used it) schemeless https://www.google.com/search?q=define%3Aschema https://www.google.com/search?q=define%3Aschema
        deleted 8y ago
        [deleted]
      - chisleu 8y ago
        Right, but most people using it use model or data repository patterns to ensure correctness. It does offer nearly infinite flexibility provided you use it correctly. You can add fields without any sort of DB work, you just start adding fields to rows as needed and let it catch up organically. There are use cases where mongo makes a lot of sense. It's very popular in the node.js / RAD world for sure. I certainly have never been a huge fan by any means. Only relatively recently did they solve distributed writes.
        toomuchtodo 8y ago
        Unfortunately, I’d argue PostgreSQL gets you all the same benefits with JSON storage (fairly equivalent to Mongo docs), while also giving you all the goodness of a relational, transactional, schema enforcing RDBMS. PGSQL became Mongo faster than Mongo could become PGSQL.
        WrtCdEvrydy 8y ago
        This is the same thing that happened in Java. Other languages started prototyping features... that eventually just end up being implemented in Java.
        int_19h 8y ago
        Java did it way too slow, and that is a significant contributor to it being relegated to "legacy" in many areas. If it waited for the other languages to prototype stuff, it might have not been the case. The problem is that it waited for them to prototype it, refine it, release it, popularize it, and for their community to adopt it, before even starting to work on it in Java - which means that by the time they had it, most people who needed it were already elsewhere (not necessarily off JVM, just another language). Lambdas were a very good example - if you look at the closest competitor, C#, it got the first take on them back in 2005. Then a major refinement in 2008, adding type inference. By 2010, lambdas were idiomatic in C#. Java, in contrast, released the first version in 2014. And even then, they're still less powerful.
  - throwaway5752 8y ago
    "my scheme is my app layer" To be fair you can do use schema validators in Mongo. Not sure it's widespread in practice. And there are other distributed databases that aren't document stores that have schemas and various subsets of SQL implemented.
    - deleted 8y ago
      [deleted]
  - bane 8y ago
    I once worked at a place that, at some point in the past, had developed some kind of semantic graph database that never gained market traction. The author of it had ended up as CTO and kept seeking out uses for his work and ended up finding all kinds of odd places for it to live, including as the auth database in a large scale document analytics system and running part of the payroll. We were constantly running into all kinds of scalability issues where this 12 year old component was often the source of our pain, but he'd never even entertain a conversation to eliminate it and consolidate or replace it with something better.
    - seibelj 8y ago
      This is the definition of software engineering hubris distilled into a few paragraphs
    - duxup 8y ago
      I worked for a company that was old but had a fancy new product. It was just about to be released when.... A competitor bought us for stupid money. They fired most folks (not my small department) had their CTO decide between our fancy new product, and the product he created. There was no question, our product was amazing, his Frankenstein was two pieces of equipment cabled together in three items the footprint ... and still did less, and was quite a ways away from being "ready". If there was a silver lining, the Frankenstein product doomed that company and we got bought by the far more competent competitor they had.
      - qbaqbaqba 8y ago
        Happy ending.
        duxup 8y ago
        It's surprising how that works. I've been through numerous mergers, acquisitions, been in the company acquiring a company. Lots of hair pulling and I can say that it was really hard to predict the outcome for any individual positive or negative in every case until long afterward.
  - djrobstep 8y ago
    My pet theory is that NoSQL took off purely because people were sick of having to manage schema changes. Unfortunately, people reacted to being (justifiably) frustrated with schemas by throwing strict schemas out entirely, instead of making better schema management/migration tools.
    - staticassertion 8y ago
      Tell that to AWS. They've banned relational databases for specific workloads because Dynamo (nosql) provides more consistent performance, and is easier to operate. Tons of conflation of Mongo's problems with those of nosql in this thread.
      - djrobstep 8y ago
        Your average dev is not making decisions based on what might work best for one particular problem Amazon has.
        staticassertion 8y ago
        Do you think all of the companies that chose Cassandra and Dynamo were wrong to do so? There's no use case for NoSQL? There were no lessons learned, value adds from NoSQL? How do you explain the 'NewSQL' approach, which seems to be so clearly borne of what we've learned from NoSQL? It should be obvious that NoSQL has value, regardless of the issues with one of the earlier NoSQL DBs.
        pjmlp 8y ago
        I don't see a value other than fashion driven development, specially when comparing the bare bones browser GUI for Dynamo with something like SQL Server Management Studio or that whole story with primary and secondary indexes, with prices being set by index usage.
        menacingly 8y ago
  - baroffoos 8y ago
    Setting up graylog was one of the worst mistakes I made. It took forever to get all the required software installed and configured and then it was taking up all the ram on the server doing fuck all.
    - SteveNuts 8y ago
      That was just Java doing Java things. Business as usual.
      - RhodesianHunter 8y ago
        This is a nonsense comment doing language troll things.
    - deleted 8y ago
      [deleted]
    - atm0sphere 8y ago
      There were single script 1-click installs for it in bash all over for me..but yeah jvm is a hog.
      - yjftsjthsd-h 8y ago
        My experience with those one-liner installs is that they usually work... strictly speaking. They don't scale, they don't deal with edge cases, they know nothing of your environment. They install one piece of software (in an "interesting" way that won't upgrade), and that's it.
        sunnymanokl 8y ago
        My last pay check was $9500 working 12 hours a week online. My sisters friend has been averaging 15k for months now and she works about 20 hours a week. I can't believe how easy it was once I tried it out. This is what I do, HERE►► http://www.worktoday33.com http://www.worktoday33.com
        8y ago
  - Sandra_lisa4 8y ago
    XzXzXZXX
  - dominotw 8y ago
    It was just a discovery phase.
  - pwm 8y ago
    I agree with the general sentiment just wanted to point out that "my schema is my app layer" has some valid use-cases. At my current job we deal with highly complex schemas (modelling insurance contracts). Correctness is paramount but at the same time you need flexibility in terms of change over time. Defining these schemas at the DB level would be painful, a bit like writing web apps in assembly. Languages like Haskell (which we use) help here with their rich and expressive typing capabilities, so we can model these complex domains expressively and use the DB only for persistence. Admittedly it does have downsides, like having to write your own migration layer, but for this use-case the benefits outweigh the pains. PS.: We do use Postgres though ;) as it has 1st class json support and at the same time we have the luxury of using its relational capabilities where needed (think of a hybrid model).
  - speedplane 8y ago
    Maybe I’m a rare exception, but I chose NoSQL early on when it was still “hot” and have never looked back. We’ve grown from a couple megabytes of data do several dozen terabytes and have had countless issues, but scaling our database was never one of them.
  - twic 8y ago
    Well, happily, those days are gone for good. Who needs NoSQL when you have the blockchain!
    - ixtli 8y ago
      Glorious, cursed comment.
- deleted 8y ago
  [deleted]
- bostonpete 8y ago
  Wish we had HN gold for comments like this.
Bucephalus355 8y ago
Wow was MongoDB really that bad?
- emmanueloga_ 8y ago
  Possible alternative headline: "Company weights in trade-offs of having two different database systems, decide to consolidate into a single one." :-) They don't give too many reasons, but say that operationally it is gonna be easier for them to work with a single database than with two (no-brainer?)
  - dralley 8y ago
    As someone on the team, that didn't factor into the decision too much, although it's definitely a nice perk. Switching to Postgres made a lot of sense even when looking at Pulp as a standalone project rather than as a part of Satellite.
  - taude 8y ago
    Except, I'd argue that most large distributed systems these days are using much more than two types of data stores, especially when you start including name/value keystores (and in memory caching variants), big data pipelines, transactional data, "unstructured" data storage (like Mongo), time series, etc. I work in FinTech and you'll see several backing our apps, and this isn't uncommon.
  - zaphirplane 8y ago
    The licensing change surely has to have played a factor
- foobiekr 8y ago
  A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things like typos, etc. become impossible. Relational structures enforced in the database (and imho sadly lacking in the intervening layers which is something I’d like to get the time to deal with; though the FoubdationDB record layer certainly has an interesting if unexploited take on it) is a natural extension of this. But there’s a whole other world where people just don’t want to be tied to constraints. I suspect it’s a personality thing.
nodesocket 8y ago
This is a dead horse that's been beaten to death over and over again here on HN. MongoDB is garbage. MongoDB doesn't scale. MongoDB is $h!t says HN users. It just feels like an echo chamber. Are you running the latest version of MongoDB in a replica set with journaling enabled and write concern set to one? MongoDB has worked great for my uses, up to moderate write/read scale. Sure, if you are running "big data" or enterprise things, it might not be the best choice, but it's not the steaming pile of horse excrement that some HN users try to make it. Second, for those who still insist that MongoDB is crap, what is the best pure document store database then? I used to champion RethinkDB, but they failed and development has basically stopped. You wouldn't build a business on RethinkDB now a days unfortunately.
- AznHisoka 8y ago
  "Are you running the latest version of MongoDB?" I don't use MongoDB, but I imagine the answer to that question for most people is: No, because I already have an existing product in production with actual customers, and I can't afford to upgrade to the latest version.
  - nodesocket 8y ago
    With a replica set you should be able to do the update with zero or little downtime with some planning. Of course you'll have to look and watch for breaking changes if you running a really old version.
    - AznHisoka 8y ago
      It's not just the downtime. It's the actual work involved in running all your regressions to ensure backward compatability. ie. do all your existing queries still work? do all your current backup processes still work? did that arcane config variable you rely on change its stinking name?
      - enraged_camel 8y ago
        I was going to respond with “well, any database that can’t guarantee that kind of backwards compatibility shouldn’t be taken seriously,” but then I realized we are talking about MongoDB and I found myself giggling.
Thaxll 8y ago
Your comment makes no sense since RH is moving for reasons different that "MongoDB is bad".
- jayd16 8y ago
  Not being worth the effort isn't much better.
  - zaphirplane 8y ago
    You are assuming it’s the technology I’m assuming it’s the licensing change.
    - dralley 8y ago
      As someone on the team, it's not the licensing change. The post is correct when it says we've been slowly working on this since 2016. The license change certainly validates the decision, but it was made for purely technical reasons.
fred_is_fred 8y ago
"Web scale"
nf05papsjfVbc 8y ago
The intensity and frequency of frustration are quite likely to subside if you are able to accept these: - humans do not always act in their long term best interests - humans do not always act in a rational manner - often the former is indistinguishable from the cases where you are not aware of someone's incentives - true change requires that people really accept it internally (and not just mere commitment to act differently)

lerax 8y ago

Indeed a sane choice

SlowRobotAhead 8y ago

Does someone have the time explain what NoSQL Mongo didn’t do that PostgreSQL does? I guess I don’t really understand the structured SQL vs NoSQL when you aren’t accessing directly but via a program. I also don’t understand why they would have supported a NoSQL and object-relational at the same time at any point.

MBCook 8y ago
From reading the post it sounds like they want transactions for some of the functionality they’re trying to build. I guess Mongo still doesn’t have those.
- d4l3k 8y ago
  They recently added multi document transactions though. https://docs.mongodb.com/manual/core/transactions/ https://docs.mongodb.com/manual/core/transactions/
  - cutety 8y ago
    This may just be my bias against NoSQL (I’ll qualify this with I don’t believe it should never be used, but 90% of use cases psql/MySQL/etc is likely the better choice these days), but when it comes to choosing a DB engine when you want ACID transactions, I’d pick the one that was built to handle ACID transactions from the start, rather than one that just added it with this large caveat in the middle of the docs: > In most cases, multi-document transaction incurs a greater performance cost over single document writes, and the availability of multi-document transaction should not be a replacement for effective schema design. For many scenarios, the denormalized data model (embedded documents and arrays) will continue to be optimal for your data and use cases. That is, for many scenarios, modeling your data appropriately will minimize the need for multi-document transactions. So, to be able to have one of the core features of SQL, you lose the biggest feature of NoSQL which is super fast writes. Which, they try to remind you that if you model your data in correctly, by creating denormalized documents with embedded data, you probably won’t need them anyway. Which just makes me cringe at having to maintain these huge unstructured documents just so you get faster writes since you don’t have to worry about ACID. Or, you can use Postgres, which was built with ACID from the start, using technology and design patterns proven over decades, and has support for JSONB if you need unstructured/document storage that you can also (fairly) efficiently query. Maybe I’m just woefully uninformed, but I just can’t imagine a use case for Mongo unless your dealing with Google level data/traffic, and fortunately in their case, they have enough money to hire enough people that actually know how & when to use Mongo effectively to the point that it’s not just a fancy way to write to /dev/null. I’d actually be really interested in some non-Google scale use cases for Mongo, like where Postgres’ performance was actually an issue, and how much Mongo actually outperforms it, and the trade offs/issues switching. Most posts I’ve seen are of the opposite migration, but I want to see what all the hype for a DB engine (what I’ve always saw as one of the less _sexy_ areas of CS, at least in marketability, compared to ML/AI or Programing Language/Compilers)

bane 8y ago

Rule #1: Always start with PostgreSQL unless you have a very compelling reason not to.

revskill 8y ago
This reminds me of: Always bet on Javascript unless you have a very compelling reason not to.
liquidise 8y ago
"Very compelling reasons" means different things to different people. Which i suppose is a catch 22. The young but introspective dev may make the wrong call on a side project and learn to stick to psql in the future. The dev who is convinced "web scale" will arrive before their first AWS bill and picks Mongo as their production database will have a worse fate. As will their team and company by proxy. Rule #1: Always start with PostgreSQL. You don't have a compelling reason not to.
- webninja 8y ago
  Is there a nice web interface that is made for PostgreSQL? I’m in love with the power that PHPMyAdmin brings to MySQL. Also there are DigitalOcean Ubuntu droplet images with it all pre-configured.
  - h1d 8y ago
    It always feels way snappier to use a local GUI connected through SSH than PhpMyAdmin and its interface also feels very cluttered to me.
int_19h 8y ago
Always start with in-memory data structure with straightforward persistence (i.e. load and save it all at once, using some popular format). If you need ACID, then start with SQLite. If you need scalability as well, then PostgreSQL. Most other features aren't worth the hassle of configuration compared to a single file on disk and a library to link to. (But I suspect that most NoSQL apps these days would do just fine with SQLite, perf-wise. As an industry, we don't have a good sense of scale.)

chatterbeak 8y ago

But MongoDb is web scale! How are they going to scale on PostgrSQL?

hartator 8y ago

Not sure I understand the MongoDB hate here. Migrations and denomalization is not somthing you have to deal with in MongoDB.

Benjamin_Dobell 8y ago
You almost certainly still need to worry about migrations. MongoDB can store data in a schemaless fashion, however, chances are your application has a data model (schema). Sure, you can avoid migrations and just tack on more and more backwards compatibility for old data layouts. But performance suffers, and chances are that code quality and correctness suffer too.
- __david__ 8y ago
  From a certain perspective, adding backwards compatibility for old data layouts is a migration. A migration that gets run every the data is loaded instead of once during a classic db migration. This is inefficient in the long run.
- hartator 8y ago
  You have to deal migrations maybe 1/20 the times than with SQL. Like no migrations for new tables, new db, and new columns. Column name changes can be handled on the application side or yes with a migration.
world32 8y ago
The migration comment has already been addressed. "Denormalization is not somthing you have to deal with in MongoDB." What do you mean exactly? If you have a piece of data, lets say an address, that exists in more than one place as is the mongodb way, you will absolutely have to deal with the fact that you need to now update that data multiple times. So denormalization is very mcuh some you have to deal with in Mongodb. Just like in a relational database, denormalization has serious consequences and should only be used when the need for performance outweighs the risks of having much more complex code and hence potentially many more hard-to-debug errors.

wcchandler 8y ago

Any guesses on a timeline? Satellite 6.6? Or will it be in the next major version? 7.0?

dralley 8y ago
Pulp 2 will continue using MongoDB until EOL. Pulp 3, which is still in development, is using Postgres -- but while the platform functionality is nearly done the plugins for e.g. RPM and Docker are not yet near feature parity.

Sandra_lisa4 8y ago

zxc

peteridah 8y ago

I am quite surprised that companies still use Red Hat Satellite Server. In 2010 while I was still at Red Hat, deploying and managing it was the mainstay of the consulting business, and it was based on Oracle RAC. It was to my mind already legacy software at that time.

lunchables 8y ago
What are the good alternatives for managing your Red Hat servers?
ricklepick 8y ago
Satellite of today is not Satellite of 2010 by any stretch. Satellite 6 is based on foreman + katello + pulp
- dralley 8y ago
  And, for anyone who has experience with Satellite 6.0/6.1... it has gotten much, much, much better since then.

BossingAround 8y ago

A silly, genuine question: when I'm writing a NodeJS app, and I need to store a JSON in a DB that looks like this: { "a" : "b", "c" : { "ca":"cb", "cd":"ce" }, "d": { ... } } How do I store it? To me, NoSQL seemed like the choice in the past. This is not difficult in relational DBs, but it requires rather long SQL commands, and SQL table design. It seems to me that "MongoDB.insert(obj)" seems like the way to go, as it is much simpler. How do experienced Node devs solve this issue? Is it simply relational all the way? Is it Postgress's JSON field..?

sandGorgon 8y ago
postgres has a jsonb datatype that works pretty much like mongodb. https://blog.codeship.com/unleash-the-power-of-storing-json-in-postgres/ https://blog.codeship.com/unleash-the-power-of-storing-json-...
- willvarfar 8y ago
  And other regular RDBMS like MariaDB and MySQL have much the same functionality. I store JSON in RDBMS all the time.
  - sandGorgon 8y ago
    interesting - i never knew that. I use postgresql, so dont have first hand knowledge, but i saw this https://tableplus.io/blog/2018/09/mariadb-vs-postgresql-a-quick-comparison.html https://tableplus.io/blog/2018/09/mariadb-vs-postgresql-a-qu... > JSON support: PostgreSQL supports JSON and JSONB while MariaDB doesn’t. It supports an alias for JSON instead, which is a LONGTEXT column. Even the proposed json column, is not indexed - postgresql jsonb is indexed and queries fast.
    - willvarfar 8y ago
      Yeah MariaDB got 'dynamic columns' first, which iirc supported key values but not arrays, and then MySQL added a more complete 'json' data type, so the exact functionality differs by flavour and version. The postgres support is simply nicer to use and more powerful, e.g. the indexing. But if your RDBMS is any fairly-recent version of MariaDB or MySQL, you can use JSON too. Of the top of my head, the thing I find most griping about the MySQL syntax is you have to do thing->>"$.key" instead of postgres-style thing->>'key' Whereas what I really want to be able to do is thing.key But most of the consumers of these modern 'dynamic columns' are apps and then it all really matters less if you don't spend all your days at the sql prompt writing stuff by hand.

fooblat 8y ago

I'm so out of date with Satellite server that my first reaction was: Wow it took this long to get oracle out!

dmoreno 8y ago

This sentence, when talking about support for old enterprise versions, is a bit suspicious: > Satellite will not use newer versions of MongoDB that are licensed under SSPL. So my read is that there is not only an intention of database consolidation, but also a desire of avoiding the SSPL.

amyjess 8y ago
Nothing "suspicious" about it. Red Hat also explicitly evicted Mongo from RHEL 8 because of the SSPL [0], and they have a longstanding policy of only shipping OSS. It's no secret, and they've been very upfront about not shipping anything SSPL. [0] Previous HN discussion: https://news.ycombinator.com/item?id=18919543 https://news.ycombinator.com/item?id=18919543

AzzieElbab 8y ago

I am no fun of MongoDB, but I must admit it has gotten a lot better in the past few years. So, why drop it now?

nailer 8y ago
New license that's not OSS (MongoDB Inc saying it is doesn't make it so, the OSD determines OSS, not some company).
- ricklepick 8y ago
  The upstream project has been pretty vocal about their reasons to switch, and it's not because of the license. MongoDB just isn't the right tool for their use case, despite valiant attempts to utilize it.
  - VWWHFSfQ 8y ago
    This is very clearly because of licensing.

linkmotif 8y ago

I’ve never enjoyed Mongo for a second, not conceptually, not practically. But I don’t understand the people who act like Postgres is the end of everything. SQL never felt right for me either. It’s a fine language for business analysis, I guess, but is it actually pleasant for development? Do all of you who so enthusiastically post that it should be the first tool you reach for really never experience object-relational impedance? Do you really enjoy modeling like this?

acdha 8y ago
Yes: what you’re really doing is documenting your data structures and access patterns, which is way less work to do consciously rather than trying to bolt it on ad hoc. A really big thing is normalization which some people tend to downplay until they’ve had to write code to recurse complex structures enforcing consistency or making global changes. Similarly, atomicity and isolation are really useful characteristics to be able to take for granted without having to code around the problem everywhere. The usual arc I've seen for document stores, ISAMs, etc. is that people say this is great based on the first 20 minutes and then about a year later realize that they've spent thousands of lines of code implementing a really clunky subset of what they'd get out of the box with a SQL database and the believed ease of use or performance benefits were far less dramatic than promised or even negative.
- linkmotif 8y ago
  Right, Postgres is better than document stores, but is it actually pleasant to store your data in square tables and then process it to runtime objects from that? Even with battle tested ORMs, it’s never felt right for me.
  - chousuke 8y ago
    I find it very pleasant, but I also tend to avoid ORMs. I can just write SQL queries to get me the data I need in any form I want, and it's not always useful to stuff that into objects when I can just process it as is.