3 ms·

I'll take a different tack. In my world (data) there is a lack of developers - or just people of any kind - with sound understanding of the relational model an

by crdb 12y ago

I'll take a different tack. In my world (data) there is a lack of developers - or just people of any kind - with sound understanding of the relational model and its (only relatively correct) implementation, SQL. SQL is a very declarative language, high level and intuitive and easy to write correctly. With a good RDBMS, it's also easy to optimize when you are resource constrained. And literally EVERY company out there has a database that needs to be queried, reports that need to be put out, databases that need to be normalized, data warehouse schemas that need to be drawn up, data sources to be joined together, questions that need to be answered and relatively few people to do the work. I would say start with bases in set theory and logic in parallel with C. J. Date's books (which are easier than Codd's original paper as a starting point). An additional benefit of starting there is he's less likely to shoot himself in the foot with ORMs later on. Regarding where to look for work, talking only about Singapore and ASEAN companies, I get contacted several times a week by agents, headhunters, and executives from local companies who have these problems. I can't speak for Europe, but here, a cold approach or a well formed reply to a job ad with "data analyst" in the title offering cheap ("because 15") but high quality remote work might work.

Vivtek 12y ago
Yeah, I was planning on drilling him on database concepts. You can't start that too early. The trick is getting him focused on a real problem first and showing him databases in context.
- tixocloud 12y ago
  Absolutely agree on this point. What got me interested in software development was having some problem to solve/goal to accomplish.
weavie 12y ago
SQL is a very practical skill for every developer to have, but I think it would take quite an exceptional 15 year old to be able to stay interested in it long enough for them to become useful. I would at least encourage them to learn another language alongside it.
- Vivtek 12y ago
  You got that right.
- BWStearns 12y ago
  I learned it alongside learning about how Rails' ORM worked and it made it much easier to stay interested (I still hate hand writing SQL though). It had the nice benefit of helping to really understand what was going on in the ORM and at the DB layer. At that point you can safely avoid abusing the ORM and creating terrible queries that might otherwise look readable and concise at the application layer.
nickysielicki 12y ago
I've never taken SQL seriously, I always find it to be a huge PITA and for the small projects I do, I find it makes more sense to just use an ORM and if I ever write something with thousands of concurrent requests I figure I can flesh that out as it comes. Why is what you're saying the better solution for someone just starting? Isn't it better to build up, rather than jump into something for which he will see little reward for in his personal projects? His son might not work in an environment where his skills have tangible benefits until 10 years from now.
- crdb 12y ago
  I wasn't a programmer (finance background), and I learnt enough SQL to be useful in perhaps 20 minutes. Reading just one Date book [1] took me a further 3 days and allowed me to solve pretty much everything I've encountered since, with the occasional googling for syntax. Aside from perhaps statistical learning (reading through ISL [2] takes perhaps half a day to a day if you are committed, another day for the labs, and then you are immediately useful to most SMEs), I can't think of a field in computer science that has such a fast ROI. Both being able to query well, and being able to build simple but accurate statistical models, are valued quite highly by the marketplace right now and naturally lend themselves to a "contractor" relationship. The OP mentioned making money... SQL is declarative which means if you can word it in your head, you can write the query (unlike with OO where you need to define what happens to your record). If you learn it properly, from the set theoretic perspective (aka the relational model, not using the RDBMS as a data dump without FKs, and treating rows as records the OO way), you can keep working like that almost indefinitely, and the problem solving is a lot of fun - "how can I define my output in terms of my input in one go". Most importantly, you KNOW your code is correct (except logical errors) whereas the ORM does not guarantee as much (my first step when at a new company that uses an OO framework is enforcing all the implicit FK constraints and keeping track of the 1-3% of tuples missing bits). Most of the time it (sort of) doesn't matter (click through rate being off by 0.2% isn't going to change your life); when you're calculating how much money you donate customers via discounts or matching your orders with cash flow, it does, and this is when companies start paying good money. Unfortunately, based on interviewing 200+ applicants over a couple of years to data-related positions, I've started to think that the relational model is simply not taught anymore, and many popular frameworks actively encourage patterns that break the model (e.g. polymorphic associations in RoR [3]). I'm now happy just to see someone aware of why constraints might be useful, let alone being able to discuss 2 vs 3 or 4 valued logic or the usefulness of prenex normal form. [1] SQL and Relational Theory, C. J. Date 2011 [2] http://www-bcf.usc.edu/~gareth/ISL/ http://www-bcf.usc.edu/~gareth/ISL/ [3] http://stackoverflow.com/questions/922184/why-can-you-not-have-a-foreign-key-in-a-polymorphic-association http://stackoverflow.com/questions/922184/why-can-you-not-ha...
Dwolb 12y ago
My girlfriend does work for her data management team in an industry which doesn't appear to pay well. Could I ask your advice on which companies or keywords she should search for to potentially find a new data management position?
- crdb 12y ago
  I've seen only three ways to make decent money in a still-technical position: 1. management (deciding what gets done and supervising it), in this case being the interface between the pure managers with MBAs and the developers, like the gearing mechanism between the stick and the engine (this is pretty much where I sit); 2. specialization, your unix-bearded database sysadmins who can do wonders with the most constrained resources, think someone who actually worked on building Postgres/Ingres since the 80s, because some institutions like banks do need that kind of talent and they can name their price; I mention beards because when I did look for such a profile, every applicant with one exception was in his 40s or 50s and wore a large grey beard; 3. crossing over to "data science" (which is the bubble word of the day, and where a lot of the fast money is), i.e. picking up skills in statistical learning and applying them to company datasets. It's still 90% getting the data into the shape you want. I used to think this field was the preserve of the PhDs, but most companies I talked to at least in Asia have almost no capability on that side of things (not even for questioning the assumptions and significance of their financial models). I also saw that Andrew Ng said much the same thing in his Coursera lectures, something along the lines of "by the time you finish this course you'll be more competent at machine learning than most data scientists I've met in the Valley" so I suspect this extends to the US. They're sort of related: you want to know about things like the innards of database engines anyway for anything beyond trivial select operations (2.); you need to be aware of what upstairs wants in order to do a good job as an analyst (1.) and you'll naturally start questioning the requests that come to you ("why are you asking me for this data, what problem are you trying to solve, what assumptions are you making") which is where 3. comes in. That's the most valuable part to the company, but also the hardest politically. I think mixing these things in an SME setting where you can have a fast impact is where you have the highest ROI as an employee (and thus highest likelihood of good money). Big corps, despite being able to offer great salaries, are so slow and bureaucratic that it is very hard to make a meaningful impact - cue in stories of no company email account for months, datasets shared in Excel because database access can't be given for "security reasons", and the utter lack of accountability that comes from 7-layer middle management structures. Startups (pre-revenue/profitability) are too small to be able to afford, or get much return, on a data scientist type.

by crdb 12y ago

Vivtek 12y ago

Yeah, I was planning on drilling him on database concepts. You can't start that too early. The trick is getting him focused on a real problem first and showing him databases in context.

tixocloud 12y ago
Absolutely agree on this point. What got me interested in software development was having some problem to solve/goal to accomplish.

weavie 12y ago

SQL is a very practical skill for every developer to have, but I think it would take quite an exceptional 15 year old to be able to stay interested in it long enough for them to become useful. I would at least encourage them to learn another language alongside it.

Vivtek 12y ago
You got that right.
BWStearns 12y ago
I learned it alongside learning about how Rails' ORM worked and it made it much easier to stay interested (I still hate hand writing SQL though). It had the nice benefit of helping to really understand what was going on in the ORM and at the DB layer. At that point you can safely avoid abusing the ORM and creating terrible queries that might otherwise look readable and concise at the application layer.

nickysielicki 12y ago

I've never taken SQL seriously, I always find it to be a huge PITA and for the small projects I do, I find it makes more sense to just use an ORM and if I ever write something with thousands of concurrent requests I figure I can flesh that out as it comes. Why is what you're saying the better solution for someone just starting? Isn't it better to build up, rather than jump into something for which he will see little reward for in his personal projects? His son might not work in an environment where his skills have tangible benefits until 10 years from now.

crdb 12y ago
I wasn't a programmer (finance background), and I learnt enough SQL to be useful in perhaps 20 minutes. Reading just one Date book [1] took me a further 3 days and allowed me to solve pretty much everything I've encountered since, with the occasional googling for syntax. Aside from perhaps statistical learning (reading through ISL [2] takes perhaps half a day to a day if you are committed, another day for the labs, and then you are immediately useful to most SMEs), I can't think of a field in computer science that has such a fast ROI. Both being able to query well, and being able to build simple but accurate statistical models, are valued quite highly by the marketplace right now and naturally lend themselves to a "contractor" relationship. The OP mentioned making money... SQL is declarative which means if you can word it in your head, you can write the query (unlike with OO where you need to define what happens to your record). If you learn it properly, from the set theoretic perspective (aka the relational model, not using the RDBMS as a data dump without FKs, and treating rows as records the OO way), you can keep working like that almost indefinitely, and the problem solving is a lot of fun - "how can I define my output in terms of my input in one go". Most importantly, you KNOW your code is correct (except logical errors) whereas the ORM does not guarantee as much (my first step when at a new company that uses an OO framework is enforcing all the implicit FK constraints and keeping track of the 1-3% of tuples missing bits). Most of the time it (sort of) doesn't matter (click through rate being off by 0.2% isn't going to change your life); when you're calculating how much money you donate customers via discounts or matching your orders with cash flow, it does, and this is when companies start paying good money. Unfortunately, based on interviewing 200+ applicants over a couple of years to data-related positions, I've started to think that the relational model is simply not taught anymore, and many popular frameworks actively encourage patterns that break the model (e.g. polymorphic associations in RoR [3]). I'm now happy just to see someone aware of why constraints might be useful, let alone being able to discuss 2 vs 3 or 4 valued logic or the usefulness of prenex normal form. [1] SQL and Relational Theory, C. J. Date 2011 [2] http://www-bcf.usc.edu/~gareth/ISL/ http://www-bcf.usc.edu/~gareth/ISL/ [3] http://stackoverflow.com/questions/922184/why-can-you-not-have-a-foreign-key-in-a-polymorphic-association http://stackoverflow.com/questions/922184/why-can-you-not-ha...

Dwolb 12y ago

My girlfriend does work for her data management team in an industry which doesn't appear to pay well. Could I ask your advice on which companies or keywords she should search for to potentially find a new data management position?

crdb 12y ago
I've seen only three ways to make decent money in a still-technical position: 1. management (deciding what gets done and supervising it), in this case being the interface between the pure managers with MBAs and the developers, like the gearing mechanism between the stick and the engine (this is pretty much where I sit); 2. specialization, your unix-bearded database sysadmins who can do wonders with the most constrained resources, think someone who actually worked on building Postgres/Ingres since the 80s, because some institutions like banks do need that kind of talent and they can name their price; I mention beards because when I did look for such a profile, every applicant with one exception was in his 40s or 50s and wore a large grey beard; 3. crossing over to "data science" (which is the bubble word of the day, and where a lot of the fast money is), i.e. picking up skills in statistical learning and applying them to company datasets. It's still 90% getting the data into the shape you want. I used to think this field was the preserve of the PhDs, but most companies I talked to at least in Asia have almost no capability on that side of things (not even for questioning the assumptions and significance of their financial models). I also saw that Andrew Ng said much the same thing in his Coursera lectures, something along the lines of "by the time you finish this course you'll be more competent at machine learning than most data scientists I've met in the Valley" so I suspect this extends to the US. They're sort of related: you want to know about things like the innards of database engines anyway for anything beyond trivial select operations (2.); you need to be aware of what upstairs wants in order to do a good job as an analyst (1.) and you'll naturally start questioning the requests that come to you ("why are you asking me for this data, what problem are you trying to solve, what assumptions are you making") which is where 3. comes in. That's the most valuable part to the company, but also the hardest politically. I think mixing these things in an SME setting where you can have a fast impact is where you have the highest ROI as an employee (and thus highest likelihood of good money). Big corps, despite being able to offer great salaries, are so slow and bureaucratic that it is very hard to make a meaningful impact - cue in stories of no company email account for months, datasets shared in Excel because database access can't be given for "security reasons", and the utter lack of accountability that comes from 7-layer middle management structures. Startups (pre-revenue/profitability) are too small to be able to afford, or get much return, on a data scientist type.