8 ms·
The Rise of SQL:the second programming language everyone needs to know
- namegulf 10mo agoData is the new oil or gold, SQL is the tool, the language to interact with it. Put it together, it's pure gold!
- bitexploder 10mo agoOne of the few things I have used in programming and technology consistently for over 25 years is SQL. Almost no time spent learning how to organize and query data has been a waste in my career.
- koolba 10mo agoBingo. "Bad programmers worry about the code. Good programmers worry about data structures and their relationships” Some quotes stick with you throughout your whole career.
- biophysboy 10mo agoAny recommended resources you wish you had encountered earlier?
- bitexploder 10mo agoHmm, I sort of learned ad-hoc. Joe Celko's books were good back in the day. I never read something a lot later that was an "aha" for me. I think I was a little resistant to "NoSQL" databases for a while but eventually they made sense to me. I can't think of a single resource or turning point. There are probably some very good books out there now. The key thing is not /everything/ has to be SQL. And SQL databases like Postgres and SQLite can be used for a lot more than SQL now. Also, don’t be afraid to just throw protos/JSON/whatever into a database with no or mininal schema to get going. But manage data design debt ruthlessly, it can haunt you. My biggest learnings: Don't prematurely normalize data, but if it is obvious it can always stay normalized, normalize it. Read the normal forms. Learn about indexing and how data is actually being stored on disk. Just knowing about indexes is a huge advantage even today. Understand and know when to use different styles of data storage: row oriented, column oriented, key value, bigtable style (2d key value), document (rare). Pick good systems. Spend more time than you think you should designing your data. The system is often easy if the data is right. Learn ACID and CAP theorem. Learn when and where you can trade on fundamental database principles in your data model for performance or ease of development. Honestly, a lot of this stuff senior engineers at big tech are just expected to know these days, but it still isn't really obvious and not everyone has big tech problems. Still if you know how to solve the problems at scale and you can get out of your own way it is much easier to write smaller systems (most problems people have). So in terms of resources, go learn about each of those concepts. Read papers. Ask an LLM about them. Play with databases and storage systems. Maybe try to write your own simple database. Go read about how people design massively scaled distributed systems and what systems they use to manage data. Just like with programming languages, be flexible and open minded. Read about how distributed systems work (CAP theorem). Almost all data systems make tradeoffs in that realm to meet cost/performance/implementation goals.
- biophysboy 10mo agoThanks for writing this out - I appreciate it
- baq 10mo agotable of contents of the manual of the RDBMS your project is using is a good start and this is not a joke. most senior engineers (by job title, anyway) haven't gone that far.
- FjordWarden 10mo agoThe CM DB group YT channel is good place to learn about the basics and advanced topics: https://www.youtube.com/@CMUDatabaseGroup https://www.youtube.com/@CMUDatabaseGroup
- Tostino 10mo agoI find them great for database development, but haven't seen practical "how to use SQL" type advice from Andy.
- therobots927 10mo agoThe mere existence of Pandas makes me extremely grateful for SQL, because my job would be absolute hell if I had to use pandas or a similar syntax. It’s hard to overemphasize just how perfect SQL is for the job that it does.
- derriz 10mo agoAgree that Pandas is horribly irregular - the only worse query language I’ve had to work with is Mongo’s. After about a decade of regular Pandas use, switching to Polars was such a relief. It’s not perfect since it’s slightly limited by being a Python library rather than an embedded query language but it’s so much better designed than Pandas - even ignoring the huge performance improvement. In my circle, Pandas is being abandoned en mass for Polars.
- ogogmad 10mo agoI don't think SQL is "perfect" and I'm not sure it's rational to even be saying that. For instance, why is it that the syntax for an SQL query is "select A from B" when many SQL-inspired syntaxes have switched to something like "from B select A" to make it more compositional? The relational model is pretty simple though. Pandas is an awful mess.
- setr 10mo agoFrom-before-select has nothing to do with composition as far as I can think of? That’s to solve the auto-complete issue — put the table first so the column list can be filtered. Things like allowing repeat clauses, compute select before where, etc are what solve for composition issues
- therobots927 10mo agoIs that really how it should be judged? If I need to get data from state A to state B, SQL is the most elegant and efficient way to do it. If I need to modify the states, or break them up to optimize for memory, SQL is the easiest way to do it. It’s been around for over 50 years and no one’s come up with anything better.
- 10mo ago
- petcat 10mo agoI've loved and used Django ORM and SQLAlchemy for many years. It got me a long way in my career. But at this point I've sworn-off using query-builders and ORMs. I just write real, hand-crafted SQL now. These "any db" abstractions just make for the worst query patterns. They're easy and map nicely to your application language, but they're really terrible unless you want to put in the effort to meta-program SQL using whatever constructs the builder library offers you. CTEs? Windows? Correlated subqueries? It's a lot. And they're always lazy, so you never really know when the N+1s are going to happen. Just write SQL. I figured this out when I realized that my application was written in Rust, but really it was a Postgres application. I use PG-specific features extensively. My data, and database, are the core of everything that my application does, or will ever do. Why am I caring about some convenient abstractions to make it easier to work with in Rust, or Python, or whatever? Nah. Just write the good SQL for your database.
- HillRat 10mo agoThe cargo-cult shibboleth of "never put business logic in your database" certainly didn't help, since a lot of developers just turned that into "never use stored procedures or views, your database is a dumb store with indexes."
- forgetfreeman 10mo agoThere's value in not having to hunt in several places for business logic, having it all in one language, etc. I was ambivalent on the topic until I encountered an 12 page query that contained a naive implementation of the knapsack problem. As with most things dogma comes with a whole host of issues, but in this case I think it's largely benign and likely did more good than harm.
- yobbo 10mo ago> hunt in several places for business logic But that is the result of having multiple applications needing to enforce valid states in the database. "Business logic" is a loose term. The database is the effective store for state so it must enforce states, eg by views, triggers, and procedures. Other "business logic" can happen outside of the db in different languages. When individual apps need to enforce valid states, then complexity, code, etc grows exponentially.
- rawgabbit 10mo agoSQL has been the main skill I have relied upon my entire career. Yes, I have worked with Pandas and other data libraries; my take away from working with Pandas is it is a pretty language but obfuscates the relational database with a non-relational lanuguage. Relational databases require a relational language which is what SQL is.
- TomasBM 10mo agoSomewhat tangential to the article, but why is SQL considered a programming language? I understand that's the convention according to the IEEE and Wikipedia [1], but the name itself - Structured Query Language - reveals that its purpose is limited by design. It's a computer language [2] for sure, but why programming? [1] https://en.wikipedia.org/wiki/List_of_programming_languages https://en.wikipedia.org/wiki/List_of_programming_languages [2] https://en.wikipedia.org/wiki/Computer_language https://en.wikipedia.org/wiki/Computer_language
- pjmlp 10mo agoBecause stored procedures do exist, and there isn't a single production quality RDMS that doesn't go beyond DDL and DML, adding structured programming extensions. Also, even within the standard itself, it allows for declarative programming.
- rawgabbit 10mo agoBecause "programming language" is an adjective or a descriptive term. Whatever looks like a programming language, can be called a programming language.
- Imustaskforhelp 10mo agoTo be honest, I'd like to chip in that it is technically possible to write brainf*ck, an esoteric programming language but nonetheless, its a programming language https://www.reddit.com/r/SQL/comments/81barp/i_implemented_a_brainfck_interpreter_in_pure_sql/ https://www.reddit.com/r/SQL/comments/81barp/i_implemented_a... Btw this runs in sqlite, you can try it yourself if you are interested. Source: I was thinking of creating a programming language paradigm like sqlite/smalltalk once where resumed execution/criu like possibilities were built in. Let me know if someone knows something like this too. I kinda gave up on the project but I knew that there was this one language which supported this paradigm but it was very complicated to understand and had a lot of other first time paradigm like the code itself / the ast tree is sort of like a database itself but so the tangential goes.
- yawaramin 10mo agoWhat is your definition of 'programming language'?
- pjmlp 10mo agoI am quite found of PL/SQL, and stored procedures, no need to waste network bandwith with what can be done on the database.
- yawaramin 10mo ago'The original microservices'
- pjmlp 10mo agoMicroservices are older than those, hence why Sun had the motto "The network is the computer". :) By the time you are referring to, we were already on the classical 3 tier architecture, the There are indeed Web frameworks for RDMS, that allow to expose the database as microservices, like Oracle's APEX, which grew out of Oracle's Visual Basic version, which used PL/SQL instead of BASIC.
- gcanyon 10mo ago> the second programming language everyone needs to know Do they though? I've been writing SQL for over twenty years, and my experience is that LLMs have been better at writing it than I am for at least most of 2025, for most use cases. I have zero doubt that I will only be writing SQL when I want to for fun no later than sometime 2027.
- gspr 10mo agoPeople who don't already know it still need to learn it if they wanna manage LLMs writing it. Anything else is reckless. So the original point stands.
- simlan 10mo agoAgreed with that. As with writing SQL by hand you have to be very specific with instructing an LLM. There are many ways to get to a solution in SQL all present different tradeoffs and corner cases. I found that people that don't understand SQL and the basic of a given schema produce garbage both by hand and with LLMs
- tete 10mo ago> I've been writing SQL for over twenty years, and my experience is that LLMs have been better at writing it than I am for at least most of 2025 Wow, bad career choices?
- gcanyon 10mo agoI wouldn't think so, knowing SQL has been a benefit, and knowing how to use LLMs to write SQL even more so.
- cjs_ac 10mo agoAn LLM can respond to any online discussion about <x> is a good approach for solving a particular class of problems with LLMs can do <x> better than a human better than you.
- yawaramin 10mo agoLearning SQL basically launched my career as a professional SWENG. Once I knew SQL, I found ways to apply it in even non-technical jobs.
- bikeshaving 10mo agoThis is a nice coincidence. I’ve been heads-down on publishing a JavaScript full-stack metaframework before the end of the year. However, in the past two weeks I’ve been goaded by Claude Code to extract and publish a separate database client because my vision includes Django-style admin/forms. The idea is to use Zod to define tables, and then use raw SQL fragments with JavaScript template tags. The library adds a tiny bit of magic for the annoying parts of SQL like normalizing join objects, taking care of SELECT clauses and validating writes. I’m only using it internally right now, but I think this approach is promising. Zod is fantastic for this use-case, and I’m sad I’ve only just discovered it. https://github.com/bikeshaving/zen https://github.com/bikeshaving/zen
- adamddev1 10mo agoThis looks promising!
- pavlov 10mo agoI’ve always hated SQL, but fortunately LLMs write it so well that it’s effectively become a read-only language now. You just need to know enough to check the output.
- system2 10mo agoI agree. Claude Code writes superb SQL queries for very complex data. I was dealing with PostgreSQL recently, and it improved the query from 30 seconds to 5 seconds. I couldn't figure it out myself.
- rtkwe 10mo agoHow do you present the interrelations between the tables when you're dealing with complex table structures?
- simlan 10mo agoPrompting with documentation and examples works. In an agentic tool having an MCP server for the db helps assuming it is a straightforward schema with explicitly defined relationships. Also helps if the tables correspond to entities in a natural way.
- lateforwork 10mo agoUse a tool to do that. Try https://visualdb.com https://visualdb.com it can send the relationships and table definitions to AI.
- rtkwe 10mo agoWe're pretty locked down around AI tools. Right now we can only really use GH Copilot, it's been ok so far though it's funny to see it suggest edits then suggest the opposite on the next time it review the PR if you accept them.
- sfn42 10mo agoI just go into SSMS and expand the columns for the tables I care about in the left hand explorer, screenshot it and send that to copilot.
- brikym 10mo agoI much prefer Kusto query language. SQL needs a few tweaks so that it's more type safe and supports auto completion. Some engines support From-first which is a good start.
- tdfirth 10mo agoIf you do backend web development in 99% of software companies then being very good at whatever your RDBMS is is a superpower. It's definitely worth learning SQL very well, but you also need to learn the data structures your RDBMS uses, how queries translate into operations on that data, and what those operations look like in a query plan. You can go surprisingly far with just that knowledge. A great resource is https://use-the-index-luke.com/ https://use-the-index-luke.com/
- ifh-hn 10mo agoI've always gravitated towards query languages and SQL is one of my favourites. I've never really understood the need for ORMs and other abstractions but then I'm not a software developer. If I was going to chose a "third" language I'd say regex.
- jinwoo68 10mo agoFolks, the article is from 3 years ago, 2022.
- calebm 10mo agoWith LLMs, you should be able to just query in English and have LLMs transpile from English to SQL.
- tete 10mo agoSQL largely is plain English, which is one of its design choices.
- bigstrat2003 10mo agoYou should also be able to reliably generate working code with LLMs, but you can't. They aren't a good tool until they actually work when they are supposed to.
- infogulch 10mo agoA recent article in the space: What Goes Around Comes Around... And Around... | July 1, 2024 | 30 comments | https://news.ycombinator.com/item?id=40846883 https://news.ycombinator.com/item?id=40846883 The basic thesis is that the relational model and SQL has been the prevailing choice for database management systems for decades and that won't change soon. Resubmitted because it's a good one: https://news.ycombinator.com/item?id=46359878 https://news.ycombinator.com/item?id=46359878
- schultzer 10mo agoSQL is great, but what is even better is a SOTA client. https://github.com/elixir-dbvisor/sql https://github.com/elixir-dbvisor/sql the BEAM can give you superpowers that no other platform can, handle massive concurrency with the performance that rivals bare metal and c https://erlangforums.com/t/elixir-dbvisor-sql-needs-a-sota-pool-and-i-want-to-hear-your-ideas/5145/9?u=schultzer https://erlangforums.com/t/elixir-dbvisor-sql-needs-a-sota-p...
- Kwpolska 10mo agoIEEE Spectrum is full of uninspiring blogspam, like this post.
- chasil 10mo agoI notice that the top image is of Transact SQL, the Sybase/Microsoft dialect. This is not a formal standard, and I suggest against its use. https://en.wikipedia.org/wiki/Transact-SQL https://en.wikipedia.org/wiki/Transact-SQL SQL/PSM is a general ISO standard that grew out of Oracle PL/SQL, is rooted in ADA, and is implemented by a large range of databases. https://en.wikipedia.org/wiki/SQL/PSM https://en.wikipedia.org/wiki/SQL/PSM Standards are important.
- setr 10mo agoThe SQL standard defines more of an aesthetic than an actual language. Every database just extends it arbitrarily and anything beyond rudimentary queries is borderline guaranteed to be incompatible with other databases. When it comes to procedural logic in particular… you have almost zero chance you’re dropping into that into another database and it working — even for rudimentary usage. SQL-land is utterly revolting if you have any belief in standards being important. Voting for Oracle (itself initialized as a shallowly copied dialect of IBM SQL, and deviated arbitrarily) as the thing to call “standard” is just offensive.
- chasil 10mo agoI was not aware that IBM copied Ada. I was aware that EnterpriseDB developed "deep Oracle compatibility" and sold the resulting code to IBM for Db2 several years ago. I think you are [more than] a bit behind the times? https://www.cnet.com/culture/ibm-puts-oracle-to-the-sword-with-enterprisedb/ https://www.cnet.com/culture/ibm-puts-oracle-to-the-sword-wi...
- stevefan1999 10mo agowithout CTE, is SQL a programming language?
- CommenterPerson 10mo agoKnowing SQL, and knowing how to optimize queries has pretty much paid my paycheck for 20 years.
- marcusfrex 10mo agoSQL is not a programming language.
- canyp 10mo agoI wonder how this guy implement a ray tracer, then. https://github.com/chunky/sqlraytracer https://github.com/chunky/sqlraytracer
- avmich 10mo agoThe problem of SQL is that is is ubiquitous. The advice to learn SQL is because it's everywhere. Now it has inertia, and a better way of doing things has an uphill battle. In 1980-s somebody compared working with SQL to drinking data through a straw, comparing to swimming in data with APL. We had a recent article comparing modern SSD delays and RAM sizes to those which defined architectures of SQL systems of recent decades - we can perhaps reengineer our databases better for modern opportunities. But all these approaches have to deal with SQL as an entrenched technology. Probably a task for a company of a Google class.
- deepsun 10mo agoMy only problem with SQL is it was designed for human input (same as shell commands), not for machines. Hence the SQL Injection attacks and other peculiarities and inefficiencies. IMO for machine-to-machine talk we should not be using a long text that needs to be parsed carefully and securely on the other side, but rather a structured structure that's easy to de-serialize (like AST packed into a protobuf, but for SQL). So that any invalid SQL parameters like "1;DELETE FROM users;--" would just fail to parse. It may be called ABI, although it may be textual as well (like json), not necessarily binary (like protobuf). PostgreSQL already supports binary wire protocol, but I never seen it's being used, instead people prefer sending raw SQL strings designed for humans even from applications.
- SoftTalker 10mo agoluckily you can use parameterized queries and completely avoid this problem.
- deepsun 10mo agoOnly for simple queries. E.g. it's hard to parameterize table names. Also it makes an extra round-trip to server to prepare the query.
- brettgriffin 10mo agoYou'd have to be using very antiquated (by nearly two decades!) patterns or practices for SQL injection to be a concern.
- deepsun 10mo agoAgree, but for example, migration scripts are still often just a bunch of long .sql files (unless it's Liquibase with its own cross-DBMS XML syntax), or test/staging/benchmark schemas. Even today. And subling commenters say that all you need is raw SQL and results mapping to your code. Which I did for a while, but found that mapping is a lot of copy-pasta with minor diffs, a burden to maintain. So it's easier to use a thin library like JOOQ for mapping, or use only the mapper part of a bigger ORM framework like Django/Hibernate. And my argument is that it's easier to map to/from a concise strongly-typed ABI/API structs instead of one raw SQL string with its structure designed for human reading/writing, like SELECT before FROM. There are such ABI-s, but they are DBMS-specific, while SQL is less so.
- gjvc 10mo agohttps://blog.codinghorror.com/object-relational-mapping-is-the-vietnam-of-computer-science/ https://blog.codinghorror.com/object-relational-mapping-is-t...
- mpalmer 10mo agoGetting more than a little tired of "news" stories like this, that read as though the author is learning about the subject matter for the first time. That in itself is fine, of course. Less fine is the author's thoughtless assumption that their readers are all learning about it too.