6 ms·

Firstly, I'd suggest the author look at this differently; perhaps "For Want of a Code Review". Especially code from a relatively recent graduate, on a piece of

• leononame 4y ago
  I agree. It took me 3 years or so to actually land in a project and learn SQL for the first time. Before it was all with ORMs. I didn't know what a join was for the first couple of years of my career. Understanding SQL and being able to work with data interactively has made me a better software engineer. This tech is important enough that it should be taught in university/coding camps.
  • miiiiiike 4y ago
    You DIDN’T learn SQL in school? Probably my most useful class. I hated it at the time, I was a desktop and embedded dev, and this was before SQLlite roamed the earth.
    • rjbwork 4y ago
      >Probably my most useful class. Ditto. My teacher was hardcore. He was a graybeard who was around before Codd's now famous paper. He worked with some of the old pre-relational hierarchical databases. We had to take SQL queries, turn them into relational calculus and algebra, turn that into a query plan, then come up with an estimate for the time the query would take to run given various hardware speed numbers and the size of the data. We had to implement our own (primitive!) database engines, including various join algorithms. To date it's one of the hardest, yet most rewarding, learning experiences I've had.
      • miiiiiike 4y ago
        That.. Sounds graduate-level. How's your PhD?
        • avidphantasm 4y ago
          That sort of course was very much par for undergrad courses at CMU when I was taking CS classes there 25 years ago. The OS course was very intense. I wasn’t a major and didn’t have time to take it, but my networks course was of similar rigor (i.e., implement a toy TCP/IP stack).
        • rjbwork 4y ago
          I took it during summer, and there were some masters and PhD students in there, but I took it as an undergrad course. It was very intense, 3 hours per day, 3 days per week, and 1.5 hours per day the other two.
          • miiiiiike 4y ago
            Man, I got downvoted into oblivion for my comment, but seriously, yeah, graduate level. That sounds fun. My most intense undergrad class series was one where we soldered together a M68HC11 computer and learned assembly one class, built an OS for it the next, and then turned it into a robot with control software running on our OS-es.
        • miiiiiike 4y ago
          This was obviously was a joke. Sounds like there's a range in SQL training that goes from "I've heard of SQL" to "I implemented a PostgreSQL compatible db my Sophomore year."
        • Izkata 4y ago
          Not necessarily. One of my undergrad courses (~2008) had us implementing a basic inverted index (think Solr or Elasticsearch), which gave me some insight into Solr that my co-workers didn't have that helped with performance issues. If we had a database course that went that in-depth, I'd've definitely taken it, too.
    • pineconewarrior 4y ago
      I took several classes on it and I still didn't really 'get' it until I had to work on challenging problems in the real world. Granted, my education was not great quality overall.
    • cfeduke 4y ago
      I just enrolled in an online CS degree course. The database class is an elective. Crazy, I know.
    • TheCapn 4y ago
      For me the first time we dabbled with SQL was in a 3rd year Software Engineering course where the focus of the class was a single group project that we managed among ourselves by splitting tasks, conducting code reviews and handling the build and release in teams. I recall one group doing the project login which went much along the lines of what the OP's article touched on. Their code was esseentially var success = false var query = SELECT * FROM users while query.read { if query(user) == input_user && query(password) == input_pass { success = true } } Yes. They selected the entire user table. Yes. They iterated over the entire result (even if first returned result was valid) Yes. That was "shipped" for the project No. My complaints notion they should be leveraging the database for all the things they're doing wrong were ignored. It was performant! Look! It logs in instantly! YEah, because there's 8 users on the database for this project, what about when it ""ships"" and there's 100,000? More? --- My first real job dealing with a database wasn't much better. We were using a MS Access database with no normalized data. Our client's primary transaction data was across a table with 70 some columns, many of which were often duplicated values in some form or utilizing very bad practices. Since joining this company I've sped up queries in almost immeasurable ways and done things my older coworkers initially derided because they couldn't understand the syntax. TL;DR SQL, for some stupid reason, is still treated as second class to core langauges and it is a god damn shame
      • marcus_holmes 4y ago
        > SQL is still treated as second class Agree so much. And if you've ever seen a real SQL wizard in action, you realise how much can be done with it. Like most of the business logic of a system can be in the database, with an interface that's a set of stored procs/functions. And fast.
        • toyg 4y ago
          > most of the business logic of a system can be in the database The problem of this approach is the tooling and lock-in. If databases had first-class versioning support for their code objects (which could easily interoperate with git), testing automation, and a parvence of standardization across the industry, then a lot of people would be very happy to work with that model. But they don't.
    • stephenhuey 4y ago
      When I was at Rice a couple decades ago, the database class was a 400-level class in which we learned relational algebra and relational calculus before SQL. The professor must have been good at teaching because I loved learning the formal underpinnings even though my memory of them has faded, but I do recall that I went from zero SQL knowledge to being very excited by its power. So many of my CS classes were very theoretical, and even though we learned some theory in the database class, it was definitely one of the single most (the single most?) pragmatic & practical of all the CS classes I had. I was so zealous about normal forms that I complained loudly at one job where they used an old D3 database with multivalue fields. It was so glaring to me because we actually used all hand-rolled SQL instead of an ORM in those days. Years later, after growing less tech-centric and more thoughtful of business needs, I realized that sparingly using multivalue fields was not a hill to die on. :) Fast forward many years to my first startup in Boston. Google App Engine was new and I wasted precious time trying to figure out how to shoehorn a typical relational data model into the early NoSQL data store available for App Engine at the time. This was just after the financial crisis and I hadn't yet heard the mantra to pick boring technologies, and I learned through sheer pain that unless you really really really need to, don't waste effort by walking away from relational databases. And also, most apps can get by with whatever the ORM does and if there's a performance issue, optimize that one query instead of trying to optimize all your SQL from the beginning. There's a lot I still don't know about pushing heavily complex queries down to the db level, but for expensive problems I'd reach for expensive assistance, because it's worth it (after trying to play with the SQL myself).
  • tracker1 4y ago
    Agreed... ORMs can be nice, but one should understand how it works. I'm a pretty big proponent of simple mappers (Dapper for .Net, template literals for JS/TS) with straight SQL over ORMs at this point.
• maratc 4y ago
  I was at a place that used sharding, so the data was scattered across 128 database servers. SELECT works there but JOIN doesn’t, as your right side may reside at another shard.
  • Jupe 4y ago
    IMO... If the query is for OLAP the data may need to be extracted to another data store. If the query is for OLTP, then the design is wrong. I don't know your problem space, but pulling data from 128 shards to resolve queries while a user is waiting is just a really bad idea.
    • paulmd 4y ago
      > If the query is for OLTP, then the design is wrong. I don't know your problem space, but pulling data from 128 shards to resolve queries while a user is waiting is just a really bad idea. well, that's the basic idea of microservices lol. forget living on a different shard, lots of times your data is going to round-trip to JSON and back a couple times and then be manually joined in some backend/service layer, or in graphql! one bad abstraction I see a lot from microservice teams (that don't really understand it past the high-level concept) is "every table is a service", or "every minimal set of tables and its codeset is a service" and that's exactly how that ends up. Microservices really ought to be chunky enough to do their business without ending up calling 27 different services under the hood just to do simple operations. Obviously there is a point where it's too chunky, but too micro is also bad too.
    • maratc 4y ago
      > pulling data from 128 shards to resolve queries while a user is waiting is just a really bad idea. To the contrary, pulling data from 128 shards can be done in parallel, and about 127 of them don't have any data to return.
  • higeorge13 4y ago
    If you had to do joins on different shards, then you have implemented sharding wrong.
  • 4y ago
• jjice 4y ago
  Our SQL course in uni left a lot to be desired. Very little time spent on join, much more on subqueries, oddly. My first job our of school there was a SQL portion and they were impressed by my overuse of subqueries. Best SQL I learned was on the first few months in a real database with real information, instead of a student-courses mock DB with 15 rows that seems to be the academic standard for teaching.
  • tracker1 4y ago
    I'm frankly surprised that some of the larger MS based data sets aren't more standard for learning. MS SQL Server isn't generally my first choice (preferring PostgreSQL for standards and portability), but it's got some pretty great example data out there.
• Beltiras 4y ago
  Oh it's taught. I have a bone to pick with how. I'd rather have spent a lot of time on the practical application of SQL than the theoretical background of column and table operations.
• lultimouomo 4y ago
  > Firstly, I'd suggest the author look at this differently; perhaps "For Want of a Code Review". Especially code from a relatively recent graduate, on a piece of code for which the engineer in question has little experience. I assume the story is made up, but if we were to take it at face value the title would be "For Want of Basic Human Decency"; the author is saying that they saw this whole easily preventable train wreck happen in slow motion and did not lift a finger to prevent it, instead laughing, taking notes and thinking of the fabulous snarky blog bost that would have come out of it.
  • axus 4y ago
    The way I read it, they accepted the decisions of those higher in the hierarchy, after providing their feedback. It wasn't clear if lots of money was lost, just lots of time. I didn't think it was made up.
  • nisegami 4y ago
    Eventually you learn it's not always a great idea to stop your employer from getting burned. Some people don't learn until it hurts.
  • Epskampie 4y ago
    From the article: > I’d definitely commented on the JOIN statements during the initial code review
    • boxed 4y ago
      How about when it became a problem? Or the second time? The third?