34 ms·

And that is (part of) why RDB "failed" to keep people even for use-cases where they should be better then NoSQL. - To much hidden complexity which can leaf to

And that is (part of) why RDB "failed" to keep people even for use-cases where they should be better then NoSQL. - To much hidden complexity which can leaf to bugs if you don't know about the hidden complexity. - To messy and/or misunderstood concurrency. - Many People having fundamental misunderstandings about how to do things and how far transactions are useful for given (often implicit default) options. One thing I had to learn the hard way is that many programmers which supposedly "know" about RDMS/SQL don't even understand isolation levels properly. Also is it just me or is this way easier in postgres (with on conflict ...).

teh_klev 6y ago
It makes me weep when interviewing and I ask developers about fundamental stuff such as "what is ACID?"[0] None of this stuff is hard to learn, and I'm not exactly Mr Ace Developer. But time and again I've been asked to debug/fix data inconsistencies that were caused simply because the developer didn't know what a transaction was, whilst running multiple inserts and updates, and one of them failed along the way. [0]: https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID
- ragnese 6y ago
  I don't know if it's worth a damn, but I have no idea what the acronym stands for. I'm sure I knew it at some point and I know it's a database compliance standard kinda thing. But I use the shit out of transactions. I rather have contention issues than non-atomic operations done to corrupt our data. When I get to Facebook scale I'll let you know what I think about eventually-consistent and NoSQL, etc. ;)
  - dathinab 6y ago
    The problem is people not even knowing that if you use the default isolation level (on pg, read-commited) that you can't do something like "read cell X, modify X in code (e.g. X+=1), write X back" without running into race conditions which you can either avoid by using a lock (e.g. a row lock) or a different isolation level. No need to know what ACID is an acronym for (I forget it all the time) or how the isolation levels are called, but you need to know what to look out for (mainly data dependencies between read an writes in the same transaction having race conditions you need to "team" and multiple read inconsistencies)
    - teh_klev 6y ago
      > No need to know what ACID is an acronym for Sure, but if you're mentoring a junior developer then I'd like to think you'd pass this knowledge on and make them understand its importance in a formal way, rather than some partial hand-wavey "this kinda means this". Get them to go and learn these fundamentals and test them on it, which is good for you because it keeps it fresh in your mind to just roll off the tongue at parties :)
    - ragnese 6y ago
      Right. Good point. I use row locks in my transactions as well. I always have to refresh myself on the isolation levels.
- master-litty 6y ago
  Anecdata: I don’t know what that acronym is off the top of my head, and I’ve worked with databases in some form consistently for over ten years.
  - dathinab 6y ago
    Funny think, I had the same issue just now. I did know all the important thinks it stands for but not the exact definition. But I think the he/she meant that people didn't understand ACID and not that they didn't know the exact definition. EDIT: I would never expect someone to know the exact definition of ACID or SOLID or similar. But if they have a lot of experience with db they should always know the ideas behind it and even if they are not an expert (but will have to write SQL) I would expect them to know what problems rough to look out for and when they should make sure they are currently not running into such an problem by e.g. checking the postgres documentation. It's good.
- deleted 6y ago
  [deleted]
- teh_klev 6y ago
  This is a general response to a bunch of comments below saying they'd no awareness of ACID, that genuinely surprised me. ACID wasn't new to me, I've been around databases since the 80's in their various forms. But I learned the solids about ACID through, believe it or not, being sent on a jolly to learn me some Oracle back in '98 and then the concept being hammered home whilst taking the SQL Server module when acquiring my MCSD (that was a jolly-free self study to earn more bucks - which paid off). Now I know a bunch of people poo-poo corporate certifications e.g. the MCP and MCSD etc, but I've yet to encounter in any modern code-school/bootcamp type educational material that covers these important fundamentals. And that's a shame because as developers it makes you think more carefully about decisions you make when interacting with a database product such as MS SQL, MySQL, Postgres etc, especially via ORM's. i.e. proper training material that sets up the knowledge fundamentals before being let loose. I'm not having a go at the competency of folks that aren't aware of ACID, but it's a fundamental concept you need to be aware of when dealing with databases such as Oracle, SQL Server, MySQL, Postgres etc. Knowing this stuff allows you to build reliable and consistent scalable databases by balancing trade-offs around the ACID concept.
noobar 6y ago
How are these issues any less for NoSQL databases? They don't have non-obvious idiosyncracies? You don't need to understand their concurrency models?
- dathinab 6y ago
  The problem is if you don't know about a lot of SQL complexity and you are not aware of you not knowing it really easy to end up with a major problem with "corrupted" (here incorrect) data while the main reason which is often cited for RDB is to prevent exactly that. This can easily lead to an situation where people (wrongly) come to the conclusion that RDBs don't work anyway and they should just go to NoSQL DBs as it's "just easier" to not have to handle a scheme. Which yes is completely flawed as a argument as doing NoSQL without any implicitly schema can bit you very hard and you still want to do transactions but now your DBMS might not support the transactions you need (instead of you just doing them incorrectly) so you "somehow" bolt them on top of it and if it fails now it's "clearly your fault". Except there is a good chance it isn't as some NoSQL DBMS are much more tricky to handle then they seem (like Mongo not necessary keeping to the Durability part in ACID |EDIT: In the past I don't know about now). I have seen this fallacy or parts of it more then one time both in "real live" and comments on Hacker News.

teh_klev 6y ago

It makes me weep when interviewing and I ask developers about fundamental stuff such as "what is ACID?"[0] None of this stuff is hard to learn, and I'm not exactly Mr Ace Developer. But time and again I've been asked to debug/fix data inconsistencies that were caused simply because the developer didn't know what a transaction was, whilst running multiple inserts and updates, and one of them failed along the way. [0]: https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID

ragnese 6y ago
I don't know if it's worth a damn, but I have no idea what the acronym stands for. I'm sure I knew it at some point and I know it's a database compliance standard kinda thing. But I use the shit out of transactions. I rather have contention issues than non-atomic operations done to corrupt our data. When I get to Facebook scale I'll let you know what I think about eventually-consistent and NoSQL, etc. ;)
- dathinab 6y ago
  The problem is people not even knowing that if you use the default isolation level (on pg, read-commited) that you can't do something like "read cell X, modify X in code (e.g. X+=1), write X back" without running into race conditions which you can either avoid by using a lock (e.g. a row lock) or a different isolation level. No need to know what ACID is an acronym for (I forget it all the time) or how the isolation levels are called, but you need to know what to look out for (mainly data dependencies between read an writes in the same transaction having race conditions you need to "team" and multiple read inconsistencies)
  - teh_klev 6y ago
    > No need to know what ACID is an acronym for Sure, but if you're mentoring a junior developer then I'd like to think you'd pass this knowledge on and make them understand its importance in a formal way, rather than some partial hand-wavey "this kinda means this". Get them to go and learn these fundamentals and test them on it, which is good for you because it keeps it fresh in your mind to just roll off the tongue at parties :)
  - ragnese 6y ago
    Right. Good point. I use row locks in my transactions as well. I always have to refresh myself on the isolation levels.
master-litty 6y ago
Anecdata: I don’t know what that acronym is off the top of my head, and I’ve worked with databases in some form consistently for over ten years.
- dathinab 6y ago
  Funny think, I had the same issue just now. I did know all the important thinks it stands for but not the exact definition. But I think the he/she meant that people didn't understand ACID and not that they didn't know the exact definition. EDIT: I would never expect someone to know the exact definition of ACID or SOLID or similar. But if they have a lot of experience with db they should always know the ideas behind it and even if they are not an expert (but will have to write SQL) I would expect them to know what problems rough to look out for and when they should make sure they are currently not running into such an problem by e.g. checking the postgres documentation. It's good.
deleted 6y ago
[deleted]
teh_klev 6y ago
This is a general response to a bunch of comments below saying they'd no awareness of ACID, that genuinely surprised me. ACID wasn't new to me, I've been around databases since the 80's in their various forms. But I learned the solids about ACID through, believe it or not, being sent on a jolly to learn me some Oracle back in '98 and then the concept being hammered home whilst taking the SQL Server module when acquiring my MCSD (that was a jolly-free self study to earn more bucks - which paid off). Now I know a bunch of people poo-poo corporate certifications e.g. the MCP and MCSD etc, but I've yet to encounter in any modern code-school/bootcamp type educational material that covers these important fundamentals. And that's a shame because as developers it makes you think more carefully about decisions you make when interacting with a database product such as MS SQL, MySQL, Postgres etc, especially via ORM's. i.e. proper training material that sets up the knowledge fundamentals before being let loose. I'm not having a go at the competency of folks that aren't aware of ACID, but it's a fundamental concept you need to be aware of when dealing with databases such as Oracle, SQL Server, MySQL, Postgres etc. Knowing this stuff allows you to build reliable and consistent scalable databases by balancing trade-offs around the ACID concept.

noobar 6y ago

How are these issues any less for NoSQL databases? They don't have non-obvious idiosyncracies? You don't need to understand their concurrency models?

dathinab 6y ago
The problem is if you don't know about a lot of SQL complexity and you are not aware of you not knowing it really easy to end up with a major problem with "corrupted" (here incorrect) data while the main reason which is often cited for RDB is to prevent exactly that. This can easily lead to an situation where people (wrongly) come to the conclusion that RDBs don't work anyway and they should just go to NoSQL DBs as it's "just easier" to not have to handle a scheme. Which yes is completely flawed as a argument as doing NoSQL without any implicitly schema can bit you very hard and you still want to do transactions but now your DBMS might not support the transactions you need (instead of you just doing them incorrectly) so you "somehow" bolt them on top of it and if it fails now it's "clearly your fault". Except there is a good chance it isn't as some NoSQL DBMS are much more tricky to handle then they seem (like Mongo not necessary keeping to the Durability part in ACID |EDIT: In the past I don't know about now). I have seen this fallacy or parts of it more then one time both in "real live" and comments on Hacker News.