3 ms·

> I cannot imagine anyone using a database effectively on any significant amount of data without understanding indexes, how different joins work, why join order

by tome 7y ago

> I cannot imagine anyone using a database effectively on any significant amount of data without understanding indexes, how different joins work, why join order is important, what effect join orders have on performance, etc. But conversely you probably didn't have to understand what filesystem the database runs on, whether it is in a RAID array, whether the network connection was over Ethernet or T1, etc. All abstractions leak. The question is how leaky they are. In my experience Haskell abstractions are much less leaky than most.

barrkel 7y ago
I did and do; when performance at the database level isn't right, you need to look into OS stats, I/O stats, disk stats (the SAN is an area ripe for unexpected contention, RAID5/6 act like a single spindle and inhibit random access in ways that RAID1 or RAID10 don't, stripe length in RAID5/6 bloats the size of small writes, etc.), but I had to stop somewhere :)
- yakshaving_jgt 7y ago
  > but I had to stop somewhere Why? You seem unsatisfied with any of the previous layers of abstraction, so where does it end? When can I truly call myself a user of a database?
  - ukj 7y ago
    When you don't have to fix it when it stops servicing requests, or worry about scaling workloads. When you treat it as a black-to-grey box, not a grey-to-white box.
- bryanrasmussen 7y ago
  So I'm confused - how many years did you have to study before you felt confident enough in your grasp of all the underlying concepts to write your first "Hello World"?
  - barrkel 7y ago
    Hello World doesn't push much to the limits.
vnorilo 7y ago
I’m not much of a haskeller, but friends’ war stories indicate that laziness leaks quite a bit.
- tome 7y ago
  Aha, yes! You are right. But then so does strictness. See https://augustss.blogspot.com/2011/05/more-points-for-lazy-evaluation-in.html https://augustss.blogspot.com/2011/05/more-points-for-lazy-e... for some examples. Laziness is a definitely a double-edged sword, with sharp edges.
- whateveracct 7y ago
  Thunk leaks happen, but they aren't that scary. They all pretty much have one of a handful of root causes. It's definitely a wart of laziness, but it's also pretty easy to avoid. As far as abtraction goes, laziness doesn't leak. In fact, it's a big reason Haskell performance composes. The head . sort example is contrived, but it holds true in more complicated & useful examples as well!
ukj 7y ago
I am in the Database-as-a-Service world. I have to understand all the intricacies of the system, so people who choose to treat databases as black boxes don't have to. There is only so far this abstraction holds. Stateless (immutable) code scales horizontally. ACID[1] doesn't. [1] https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID
magicalhippo 7y ago
Fun aside, coworker recently had a fun issue where the database server would halt due to running out of disk space... except the disk had over 100 GB left. Turns out NTFS has a maximum number of file fragments a file can have, and if it exceeds that it will refuse to grow the file (for obvious reasons). Hardly everyday stuff though.

by tome 7y ago

barrkel 7y ago

I did and do; when performance at the database level isn't right, you need to look into OS stats, I/O stats, disk stats (the SAN is an area ripe for unexpected contention, RAID5/6 act like a single spindle and inhibit random access in ways that RAID1 or RAID10 don't, stripe length in RAID5/6 bloats the size of small writes, etc.), but I had to stop somewhere :)

yakshaving_jgt 7y ago
> but I had to stop somewhere Why? You seem unsatisfied with any of the previous layers of abstraction, so where does it end? When can I truly call myself a user of a database?
- ukj 7y ago
  When you don't have to fix it when it stops servicing requests, or worry about scaling workloads. When you treat it as a black-to-grey box, not a grey-to-white box.
bryanrasmussen 7y ago
So I'm confused - how many years did you have to study before you felt confident enough in your grasp of all the underlying concepts to write your first "Hello World"?
- barrkel 7y ago
  Hello World doesn't push much to the limits.

vnorilo 7y ago

I’m not much of a haskeller, but friends’ war stories indicate that laziness leaks quite a bit.

tome 7y ago
Aha, yes! You are right. But then so does strictness. See https://augustss.blogspot.com/2011/05/more-points-for-lazy-evaluation-in.html https://augustss.blogspot.com/2011/05/more-points-for-lazy-e... for some examples. Laziness is a definitely a double-edged sword, with sharp edges.
whateveracct 7y ago
Thunk leaks happen, but they aren't that scary. They all pretty much have one of a handful of root causes. It's definitely a wart of laziness, but it's also pretty easy to avoid. As far as abtraction goes, laziness doesn't leak. In fact, it's a big reason Haskell performance composes. The head . sort example is contrived, but it holds true in more complicated & useful examples as well!

ukj 7y ago

I am in the Database-as-a-Service world. I have to understand all the intricacies of the system, so people who choose to treat databases as black boxes don't have to. There is only so far this abstraction holds. Stateless (immutable) code scales horizontally. ACID[1] doesn't. [1] https://en.wikipedia.org/wiki/ACID https://en.wikipedia.org/wiki/ACID

magicalhippo 7y ago

Fun aside, coworker recently had a fun issue where the database server would halt due to running out of disk space... except the disk had over 100 GB left. Turns out NTFS has a maximum number of file fragments a file can have, and if it exceeds that it will refuse to grow the file (for obvious reasons). Hardly everyday stuff though.