4 ms·
Dogma leads to shitty code no matter which way a person leans. One of the worst pieces of code I ever worked on was a query generator. Somebody noticed that the
by dkarl 3y ago
Dogma leads to shitty code no matter which way a person leans. One of the worst pieces of code I ever worked on was a query generator. Somebody noticed that there were recurring patterns in some BI-ish queries that were used to generate a dashboard for customers that wanted to see their usage, and they decided to factor out the redundant parts and eliminate the boilerplate.
What did they end up with? The few hundred lines of code expressing the BI queries shrank in half, but behind the simplicity was close to a thousand lines of dense, inscrutable magic. It was a net increase in LOC, but the value of the magic was supposed to compound as they added more queries. What happened was, the original programmer moved on, and every attempt to add more queries failed, until I joined and it was my turn to be sacrificed to the monster. (I did manage to figure it out. The key was realizing that the whole thing was stupid, from conception to execution — the other engineers had put the original programmer on a pedestal, and they were trying to make the code make sense, which it didn't.)
After making the query generator work for a few queries, I had established the credibility to say that we shouldn't use it anymore, and we should just write out all the boilerplate instead. Suddenly adding and modifying queries became something that anybody could do.
It isn't just custom code that ends up this way. I'm currently working on a project that uses SQLAlchemy, and as the glutton for punishment I am, I'm the person who cleans up all our SQLAlchemy difficulties. I virtually always have the documentation open in a tab, and I have the source code checked out to the version we use. If we just wrote raw SQL and wrote our own row mappers, we'd have twice as much database code, but we'd understand it, and anybody could write and debug it. Instead, half the team treats it as witchcraft, and I feel like I've invested more time learning SQLAlchemy in the last year than I ever spent learning SQL.
This is not to say I'm against abstraction, just that it can be done so poorly that it's counterproductive. You always have to compare -- are we better off with this, or without it? Saying that something reduces boilerplate or reduces repetition isn't the end of the conversation, even if it's true. You have to ask what the cost is.
- lovasoa 3y agoThis is very true. And this is what the blog post was advocating too ! It was not about using some smart custom ORM, but about writing dead simple raw SQL queries in SQLPage instead of hundreds of lines of python and typescript.
- dgb23 3y agoI like to go in the other direction: Write the raw SQL and then generate the boilerplate from that. This has very few surprises because it’s a bottom up approach. And even better: you can do the exact same thing by hand. There’s tools/libs that help with that like hugsql (Clojure) or sqlc (Go and other languages). Doing it top down (ORM etc.) is what can cause so many problems outside of the happy path and trivial cases. These tools basically need to reinvent SQL and map it into a procedural language. Just use SQL!
- dkarl 3y agoThere's a tool called PugSQL that looks promising for Python, but it seems that async isn't directly supported yet[0]. If I ever find time, I'd love to jump on this and make it work, but nobody should hold their breath for that. [0] https://github.com/mcfunley/pugsql/issues/44 https://github.com/mcfunley/pugsql/issues/44
- actionfromafar 3y agoQuery builders are a good golden path for me. Just SQL but extra hand holding in type coercion and syntax.
- dkarl 3y agoI think query builders can be very helpful in a language with a good type system. The only times I haven't used raw SQL and didn't feel like it was a massive mistake were when using Scala, via Slick and Quill.
- edandersen 3y agoAh, the "homemade ORM" anti-pattern. My condolences.