7 ms·

Loved it! I agree with most of it as well, I'm so fed up with preparing with abastractions to things that will probably never occur (changing the underlying DB

• edoceo 6y ago
  Yea, 20+ years on PostgreSQL. Why would I change my DB away from the perfect one?
  • TimTheTinker 6y ago
    One never knows what the future may bring. Long-lived OSS software is a relatively stable bet, but the point still stands.
    • rootsofallevil 6y ago
      >One never knows what the future may bring. Exactly, so make the changes when you need them, otherwise you are relying on hitting the abstraction lottery
    • jacques_chester 6y ago
      Maintaining database abstraction layers and the like is a real option used to hedge the risk of the holdup problem. But like any real option, it has a carrying cost. That means it's an economic question, there is no purely technical rule that can give you a robust answer to whether or not to abstract something away. I feel the industry has a long hangover from the 80s and 90s in terms of the holdup problem. Oracle, basically, created a massive externality of anxiety about vendor lockin that continues to impose drag to this day.
      • TimTheTinker 6y ago
        Agreed. “Our software currently runs on Oracle but can be switched to MS SQL Server or DB2 with about a day’s work” used to be an admirable and economically valuable statement. In the majority of cases, I think a simple “runs on Postgres” is even more valuable today (unless your product is a database abstraction layer).
      • fulafel 6y ago
        There are complexity limits past which we can't really build or extend applications short of big rewrites. I guess we can describe this in terms of economics but I feel it's not doing the issue justice. And it's often better to spend the complexity budget on domain problems.
    • pydry 6y ago
      Decoupling your data model from postgres because you "might" need to swap database is a bet I've seen taken many times but it's never one I've seen paid off. This is a clear example of where YAGNI applies, I think. Extra work plus extra boilerplate to maintain. No payoff. That said, extra code + boilerplate is a great way to treat riskier software.
  • rzzzt 6y ago
    Self-hosted, low volume or demo installations can benefit from a lightweight database (that is also in-process, so the user does not have to install and maintain it) like SQLite, H2 or Derby.
  • paulryanrogers 6y ago
    Also love Pg and using flavor specific features where they deliver tangible value. That said, if a team maintains multiple flavors often then an abstraction can improve quality of life.
  • majkinetor 6y ago
    IME, because client asked for the different one for whatever reason and is willing to pay for higher development cost. You can argue about that just the same as when client needs a feature X. In my case, I was very often getting away with YAGNI or 'how about we implement nightly sync from pg to oracle' but not always.
• toddh 6y ago
  I'm curious what other people think when they take over your code? We all have a bias to think our way is simple and obvious, yet it rarely works that way in practice. Abstractions are a way of trying to find a common ground conceptually. Agreed, abstractions should not be multiplied unnecessarily, but in the same way it's easier to learn Newton's Laws than read Kepler's data, a few well chosen abstractions help others organize and understand what's happening in a code base.
  • deckard1 6y ago
    > Abstractions are a way of trying to find a common ground conceptually. What's always amused me, in a tortured sort of way, is that SQL is already the abstraction. Then people go and layer an abstraction on top of it, such as some ORM flavor-of-the-day, which is obviously much less universal than SQL. In "SQL" I'm including vendor extensions too (Oracle, MySQL, etc.). It's easier to read up on vendor extensions than learn yet-another ORM. Which do you think has had the longer shelf life: MySQL or some random Ruby ORM? If you've been doing MySQL since the '90s then it's largely the same as MySQL of 2021. That Ruby ORM? Probably hasn't seen an update since 2008. I can't even remember the names of all the ORMs I've had to use over the years.
    • toddh 6y ago
      I guess I don't think of an ORM as abstraction, it's more of a complication. The abstraction would be something like: class User { static get(id) { embed your SQL here } }
• ivan_gammel 6y ago
  That’s one of the traps, every programmer will fall into at least once: copying some abstraction because someone else did it. I have not seen support of changing underlying DB as a business requirement yet (well, outside of frameworks and platforms, of course). I have seen many times when developers designed architectures, created APIs or added buttons to user interfaces, because they felt it may be useful in the future, not because someone told them it will happen. That code was very different in quality, sometimes too abstract, sometimes a big multi-screen function doing plenty of magic. All of it had nothing to do with SOLID — it was always a violation of another principle: KISS. Keeping it simple, an engineering variant of Ockham’s razor, does not contradict or replace SOLID, it complements it, defining the scope of application of other principles. If your requirements are complex enough, you may need an abstraction — if you really need it, here are some guidelines. That’s it, now keep it simple.
  • jcelerier 6y ago
    > I have not seen support of changing underlying DB as a business requirement yet (well, outside of frameworks and platforms, of course). ... and, you haven't ever seen a business requirement of what used to be a "software" to become a framework / plateform instead ? it happens all the time
    • ivan_gammel 6y ago
      Well, that - "any" software becoming a framework - does happen, but not all the time. Why does this matter?
    • gregmac 6y ago
      I've seen bits of this, and I've seen it happen in code bases that were built as "abstract" so its components could re-used. But I've never seen it happen in any way close to resembling what the original architect thought would happen, and as a result, all those abstractions and generic implementations not only added time to the mainline development, but in the end actually got in the way of the abstraction that was needed.
• youerbt 6y ago
  Yes, let's abstract the database is probably one of the most mindlessly applied rituals in an enterprise software. And if running on different databases is not a feature of your application, it is mostly harmful practice. If you can decide on a language, framework, critical libraries etc. then you should be able to decide on a database. It's probably more important than your application anyway.
  • matsemann 6y ago
    Back before devops, containers, postgres etc were mainstream (so not more than ~6 years ago in my industry), soo many were running oracle dbs. And everyone shared the same instance, and it wasn't exactly trivial to get a new personal up and running (licensing, probably required a DBAs help). So then using hsqldb or something else lightweight were golden for local development or integration tests. So abstracting the DB was the default, and absolutely needed.
    • no-s 6y ago
      >>using hsqldb or something else lightweight were golden for local development or integration tests. So abstracting the DB was the default, and absolutely needed. This also improves efficiency in operations, not necessarily development. If you used a library/framework for database access anyway, it's not an extra expense. There's ultimately a portability concern even if "vendoring", it only imposes a cut-out to permit control of necessary change). After a few unpleasant experiences I endlessly advocated we should always use an interface to access populated data objects and not interact with the database directly, not even running queries directly but always using at least lightweight IOC. I also advocated for testing where known result sets were fed through mock objects. After all, saved result sets could also be used to test/diagnose the database independently after schema/data changes. My experience predates a lot of ORM and framework responses. Unfortunately later frameworks (intended to abstract these concerns) became ends in themselves, rather than a means to an end. These were used to satiate "enterprise-y" concerns (sacrifices to the enterprise gods). If you could afford to deploy operational Oracle, you would't necessarily flinch at the cost of the extra (often pointless) layers of abstraction.
    • 6y ago
• silentbugs 6y ago
  Exactly. DRY and YAGNI are two basic concepts I've always tried to work with. Do I need this piece of code more than once? Should be extracted/created in a separate place. Would I need this in other projects? Library. YAGNI is most likely directly conflicting with most of what's mentioned, if the developer simply asks themselves: do I really need this? Will this project benefit from having separate layers of abstraction for things that are most likely never going to change? I always think twice before writing a single line of code, with the main point of focus being if future me (or anyone who reads that piece of code) will be able to understand and change things, if needed.
  • jehlakj 6y ago
    DRY seems to be one of those principles too many people take literally. Especially among junior devs from my observation (and experience). Just because there is a block of code that’s being used in two different places doesn’t mean it should always be abstracted out. There’s a subtle yet mindful consideration whether these two consumers are under the same domain. Or if it should exist as a utility function across all domains. And if that’s the case, changing the code to make it generic, simple and without side effects is ideal. I’ve seen too many of these mindless DRY patterns over time, and they eventually end up with Boolean flags to check which domain it’s being used in as the codebase becomes larger.
    • gfody 6y ago
      DRY should really be DRYUTETNRYDTP - dont repeat yourself unless the effort to not repeat yourself defeats the purpose I also propose LAWYD - look at what you've done, a mandatory moment of honest reflection after drying out some code where you compare it to the original and decide if you've really improved the situation
      • sixothree 6y ago
        I honestly thought that was one of the principles of DRY. Maybe I've been wrong this whole time. Will it be faster, safer, more maintainable if I just leave this code in here twice? At what point does that equation change?
• Lewton 6y ago
  I know of at least one company that’s slowly dying because their product is irreparably tied to oracle Literally impossible to get any new clients to agree to run on oracle
• hyko 6y ago
  Providing you’re correct about the DB choice outlasting the lifetime of the software, there is nothing about that approach that’s incompatible with SOLID. Ultimately the goal of SOLID is to be able to change any important aspect of the software independently from any other. If the DB is going to outlast the business logic you’re writing, there’s no problem having it depend on the DB concretely. Separating the what and the how does have the effect of changing how you approach reasoning about a program though, so SOLID is not without penalty. Edited for clarification.
• thatwasunusual 6y ago
  > I'm so fed up with preparing with abastractions to things that will probably never occur (changing the underlying DB for eg) [...] How can you be fed up with something that will never occur?
• herval 6y ago
  Reading these comments make me feel like a huge outlier. Practically every project I ever worked on included AT LEAST one database swap. That includes startups and big tech, for all sorts of different reasons.
  • geofft 6y ago
    I've worked on projects with database swaps, too, but I find it hard to believe that use of abstraction in advance would have helped them. There's a couple of cases. One is that you're using two SQL databases and you're not using any advanced features. You started dev on MySQL, and then the company says "Thou shalt use Postgres" (or whatever). You don't need anything fancy in your own code to handle this. You're still making SQL queries, you're just swapping out the database engine. Technically, this is an example of dependency inversion (depend on the SQL abstraction), but you also didn't set out to do it - basically any programming language you're likely to use has the common database libraries use a common abstraction for sending queries. And you didn't specifically make sure you were writing generic SQL, you just happened not to need anything. More commonly, you're switching databases for a specific feature. Maybe you realize PostGIS (or whatever) is going to solve a problem for you very well. But then you're changing how you model data, what your schemata are, and even how your code is architected and accesses things in the database. You're deciding to move certain logic from your code to the database engine - might even decide to move certain logic from the frontend into the backend, or change how request routing works, or something. This is a fantastic reason to move databases, but no amount of abstraction can prepare you for it, because you're fundamentally changing what the abstraction is. And you're deliberately abandoning SOLID because you're picking up a dependency on a concrete database. But the real case I've seen is where you're switching databases (or data storage layers, more generically) to a different model - MongoDB to not-MongoDB, a C/P database to an A/P one, a relational one to a key-value store, etc. This is the above case but even larger. There is no abstraction you could possibly write that could encompass the old and new cases. It requires rearchitecting how your code works. And then there are the most boring of cases - the ones where a database swap sounds doable in theory and the code is supposedly using an abstraction layer, but no one has ever verified that the code doesn't make assumptions about what database it's on and the code has gotten too big, so we just get an architectural exemption from "Thou shalt" and we run our own instance of the wrong database, because the overhead of running our own DB costs the business less than getting the swap wrong. (Some public examples of these sorts of database migrations that come to mind: https://slack.engineering/scaling-datastores-at-slack-with-vitess/ https://slack.engineering/scaling-datastores-at-slack-with-v... is about how Slack couldn't move away from MySQL and the architectural assumptions they made about it and had to rule out migrations to non-relational databases out of hand, and https://about.gitlab.com/blog/2018/09/12/the-road-to-gitaly-1-0/ https://about.gitlab.com/blog/2018/09/12/the-road-to-gitaly-... talks about how GitLab moved from NFS storage to an RPC service, requiring a lot of refactoring of callers.)
• no-s 6y ago
  the most important take-away from Agile is YAGNI - "you aint gonna need it." That's what I mean when I say simple. Don't obscure your code with unfalsifiable assumptions reified, as related in the typical obscure vernacular... I have had to deal with RDBMS-swapping for fairly large and transactionally intensive applications. I appreciated code that embodied SOLID to the extent that it reduced the amount of code to inspect and improved the quality of sizing up what needed to be done. However it was much easier to "lift" up systems to the new objective than "unravel" previous efforts to insure portability without a defined and testable objective of portability.
• majgr 6y ago
  I, once, removed a graph from application. Application has all these nice abstractions. Change touched 40+ files. Single Responsibility? bonkers.
• yeswecatan 6y ago
  The most useful part of changing the underlying DB, for me, seems like a way to speed up unit tests. Then again, I'm used to writing Django apps where the domain model is heavily coupled to the database, making it hard to just test simple objects without touching a database. I wonder if others here have applied some sort of domain driven design (domain models agnostic of database) without going down the whole repository route.