3 ms·

I love sql myself. I'm not sure why I love it so much and other people hate it. Sometimes I think that coming up with reasons is just after the fact justifica

by geebee 10y ago

I love sql myself. I'm not sure why I love it so much and other people hate it. Sometimes I think that coming up with reasons is just after the fact justification (do I really like it because it has a solid mathematical foundation in the relational calculus, or does that just sound like a good reason to say I like it)? Pandas is an excellent contribution to data analysis, but I gotta tell you, I was absolutely delighted to learn about pandasql and realize I could write selects, group by, order by, and so forth, converting data frame to data frame through sql. What can I say? I truly encourage people to give SQL a real chance. I'm happy to use ORMs that save me some typing on simple queries, and certainly I use a combination of pandas and sql, there are some things much better accomplished through a programming language. But I'm willing to drop to SQL pretty quickly if I sense that the programming language or library is really just forcing me to re-learn a new implementation of sql.

danso 10y ago
I used to dislike it myself, and in the context of building within a framework like Rails, I'll defer to an ORM. But my appreciation for SQL deepened as I became a more experienced programmer and began valuing explicitness in code. The reason why I do most data work in Python is because most of my work is exploratory, and I need a language that is well-suited for gluing together services and non-analysis-specific functions. But when it comes to doing work in which time and consistency and portability is Key, SQL is my choice for structured data.
collyw 10y ago
I agree, I was always so so about SQL until I had to use a lot of it. The syntax isn't amazing, but the relational model works really well once you can think in sets. The declarative nature of it means that if I push stuff from the app layer to the database layer its usually less buggy and has better performance.
ams6110 10y ago
IF you start to think in a set orientation, SQL makes a lot more sense. If you try to think procedurally you will struggle with SQL.
- iwintermute 10y ago
  Couldn't agree more - I often see this pattern in programmers - they try to think in cursors instead of sets and end up really fighting the instrument - not using it as intended.
- geebee 10y ago
  That's interesting. I didn't learn SQL until taking a course on it in grad school. I had some programming background, mainly from math classes and a bit of CS. the concept of a query really seemed odd, it took some mental adjusting. My only programming background was largely in writing procedural code. My brain really wanted to think in terms of methods, functions, inputs, and return values. It's been so long (18 years since that course), that it is difficult to remember the mindset I was in. It all seems pretty natural now. But you are absolutely right, you do need to start thinking in terms of sets and operations on sets, not in terms of methods with inputs, procedures, and return values
  - danso 10y ago
    I was in a similar boat. I studied computer engineering in school but didn't take any database course. My first encounter with SQL was years later as an education reporter. Someone in the newsroom threw out a "Access 97 for Dummies" and I picked it up and realized it was just what I needed to do the kind of analysis I wanted on California schools and test scores (CA.gov is quite good about posting raw data [0]) I eventually moved from Access to MySQL/SQLite, but if it weren't for Access's interactive query designer, where the tables are presented as lists of columns and you draw/drag lines to declare a JOIN between columns [1], I honestly don't think I would have ever grokked SQL. I actually think that without SQL, I would have never understood the power and purpose of a JOIN, even though that concept of comparing lists is the foundation of most investigative research. So I justify forcing SQL on to journalism students not merely because it's a useful skill, but because it presents the best vocabulary and grammar for describing the idea of joins. Also, SQL's general lack of data-munging functions or conveniences really drives home that data is just characters and numbers, and doesn't magically have meaning or come into existence. That is an extremely difficult concept to convey with Excel -- e.g. how it automagically converts `9-11` into `11-Sep` and confers a datetime context onto what was just plaintext. [0] http://www.cde.ca.gov/ds/dd/ http://www.cde.ca.gov/ds/dd/ [1] http://www.opengatesw.net/ms-access-tutorials/Access-Articles/Microsoft-Access-Query.htm http://www.opengatesw.net/ms-access-tutorials/Access-Article...

by geebee 10y ago

danso 10y ago

I used to dislike it myself, and in the context of building within a framework like Rails, I'll defer to an ORM. But my appreciation for SQL deepened as I became a more experienced programmer and began valuing explicitness in code. The reason why I do most data work in Python is because most of my work is exploratory, and I need a language that is well-suited for gluing together services and non-analysis-specific functions. But when it comes to doing work in which time and consistency and portability is Key, SQL is my choice for structured data.

collyw 10y ago

I agree, I was always so so about SQL until I had to use a lot of it. The syntax isn't amazing, but the relational model works really well once you can think in sets. The declarative nature of it means that if I push stuff from the app layer to the database layer its usually less buggy and has better performance.

ams6110 10y ago

IF you start to think in a set orientation, SQL makes a lot more sense. If you try to think procedurally you will struggle with SQL.

iwintermute 10y ago
Couldn't agree more - I often see this pattern in programmers - they try to think in cursors instead of sets and end up really fighting the instrument - not using it as intended.
geebee 10y ago
That's interesting. I didn't learn SQL until taking a course on it in grad school. I had some programming background, mainly from math classes and a bit of CS. the concept of a query really seemed odd, it took some mental adjusting. My only programming background was largely in writing procedural code. My brain really wanted to think in terms of methods, functions, inputs, and return values. It's been so long (18 years since that course), that it is difficult to remember the mindset I was in. It all seems pretty natural now. But you are absolutely right, you do need to start thinking in terms of sets and operations on sets, not in terms of methods with inputs, procedures, and return values
- danso 10y ago
  I was in a similar boat. I studied computer engineering in school but didn't take any database course. My first encounter with SQL was years later as an education reporter. Someone in the newsroom threw out a "Access 97 for Dummies" and I picked it up and realized it was just what I needed to do the kind of analysis I wanted on California schools and test scores (CA.gov is quite good about posting raw data [0]) I eventually moved from Access to MySQL/SQLite, but if it weren't for Access's interactive query designer, where the tables are presented as lists of columns and you draw/drag lines to declare a JOIN between columns [1], I honestly don't think I would have ever grokked SQL. I actually think that without SQL, I would have never understood the power and purpose of a JOIN, even though that concept of comparing lists is the foundation of most investigative research. So I justify forcing SQL on to journalism students not merely because it's a useful skill, but because it presents the best vocabulary and grammar for describing the idea of joins. Also, SQL's general lack of data-munging functions or conveniences really drives home that data is just characters and numbers, and doesn't magically have meaning or come into existence. That is an extremely difficult concept to convey with Excel -- e.g. how it automagically converts `9-11` into `11-Sep` and confers a datetime context onto what was just plaintext. [0] http://www.cde.ca.gov/ds/dd/ http://www.cde.ca.gov/ds/dd/ [1] http://www.opengatesw.net/ms-access-tutorials/Access-Articles/Microsoft-Access-Query.htm http://www.opengatesw.net/ms-access-tutorials/Access-Article...