6 ms·

I have spent quite a lot of time building a database in Java. It's not a small undertaking. I've gone through as many books and papers as I can, and there is o

by ccleve 8y ago

I have spent quite a lot of time building a database in Java. It's not a small undertaking. I've gone through as many books and papers as I can, and there is only one, one, that gives an overview and actual examples of building a system that processes SQL. Everything else is theoretical or so general that you don't get enough direction to get started. It's "Database Design and Implementation" by Sciore. It's a little dated, but not by much. It is an underappreciated resource.

fgonzag 8y ago
You could have read the source to the processor of one of the many open source SQL databases though (off the top of my head: MySQL, PostgreSQL, SQLite, Derby, H2, HSQLDB). Many of them have very clean code, SQLite would be my choice if you know C otherwise one of the java databases (though I've never actually read their source)
- macspoofing 8y ago
  This. Here's an anecdote: one my friends worked for a startup that at the height of NoSQL hype (circa 2012) decided to build their product on Cassandra. The data they dealt with was highly relational with hard requirements on consistency. He told me working on a NoSQL database taught him a lot about relational databases, because he scoured the Postgres source code to learn how to implement consistent and durable commits across distributed/sharded nodes.
  - haggy 8y ago
    OK I'll say it... Your friend chose wrong. HORRIBLY wrong. I have a lot of experience with both Postgres and Cassandra and the fact that your friend chose Cassandra despite their data model being relational AND having high consistency requirements shows that they chose Cassandra for the wrong reason(s). Before you come back saying something about scalability, I've run both databases at way above average scale so what you choose should come down to what you need, which is not what was done in this case (your friend)
    - mekoka 8y ago
      I think his comment was mostly about the fact that you can learn a lot about relational databases by reading Postgresql's source. He did say at the height of NoSQL hype. To me, that choice of words gives subtle hints as to the historical context and a disclaimer about the pertinence of that decision. Thank you for your input though, it provides food for thought for the next person mulling over a similar decision.
    - int_19h 8y ago
      It sounds like his friend didn't choose anything, his bosses did.
    - macspoofing 8y ago
      >Your friend chose wrong. HORRIBLY wrong. Yeah, obviously. Also, he didn't choose anything. The tech stack was mandated on him.
    - nickpeterson 8y ago
      Sometimes the best way to learn the value of something is to try really hard not to use it.
- afarrell 8y ago
  For some, reading a body of source code without any guidence about what path to walk or where to focus your attention is really quite a hard way to learn.
  - fgonzag 8y ago
    It is an essential skill for our job though, being able to dive into a new code base, form a mental model of it, and modify it either because of new feature requirements or simply bug fixing. And it is a skill, I also had a lot of trouble with it initially, but diving into several open source projects was the perfect way to practice it, and it helped me tremendously in my professional development.
    - rnnr 8y ago
      Agreed. A good tip for this is to try to find the earliest version/commit of the project you are interested into.
    - PretzelFisch 8y ago
      Reading the source doesn't tell you why they chose that route or what the tradeoffs are.
      - ldng 8y ago
        Well, depends on the quality of the documentation I guess.
    - dokem 8y ago
      Most code bases are business logic and not nearly as complex or formal as a db. DBs, OSs and compilers are systems that should be studied to be effective in the code.
    - notafrog 8y ago
      I agree with you, in that it's an essential skill for our job. However, from my experience at least, I've found that going head first into the source code of something you don't really understand how it works, is not the best course of action. For example, I would make sure to read about B+ trees first, before reading the implementation of indexes in SQLite or PostgreSQL.
HappyJoy 8y ago
I've been looking for this type of book. Thanks!
zerr 8y ago
Agree! I've accidentally found this gem. It is really written from the actual hands-on engineering/programming perspective which covers all aspects, from the front-end to the actual storage implementation.
charlysl 8y ago
You may find MIT's Database Systems course helpful, the labs consist of building several db components in Java (and the prof is Stonebraker, of Postgres fame): https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/ https://ocw.mit.edu/courses/electrical-engineering-and-compu... https://youtu.be/F3XGUPll6Qs https://youtu.be/F3XGUPll6Qs http://blog.marcua.net/post/117671929 http://blog.marcua.net/post/117671929
- hestefisk 8y ago
  Stonebraker is the Donald Knuth of databases.
billxreynolds 8y ago
Sciore is a great teacher! I had him at BC 20 years ago for both database systems and object oriented programming. I didn't know he put out a few books, I'll have to check them out.

by ccleve 8y ago

fgonzag 8y ago

You could have read the source to the processor of one of the many open source SQL databases though (off the top of my head: MySQL, PostgreSQL, SQLite, Derby, H2, HSQLDB). Many of them have very clean code, SQLite would be my choice if you know C otherwise one of the java databases (though I've never actually read their source)

macspoofing 8y ago
This. Here's an anecdote: one my friends worked for a startup that at the height of NoSQL hype (circa 2012) decided to build their product on Cassandra. The data they dealt with was highly relational with hard requirements on consistency. He told me working on a NoSQL database taught him a lot about relational databases, because he scoured the Postgres source code to learn how to implement consistent and durable commits across distributed/sharded nodes.
- haggy 8y ago
  OK I'll say it... Your friend chose wrong. HORRIBLY wrong. I have a lot of experience with both Postgres and Cassandra and the fact that your friend chose Cassandra despite their data model being relational AND having high consistency requirements shows that they chose Cassandra for the wrong reason(s). Before you come back saying something about scalability, I've run both databases at way above average scale so what you choose should come down to what you need, which is not what was done in this case (your friend)
  - mekoka 8y ago
    I think his comment was mostly about the fact that you can learn a lot about relational databases by reading Postgresql's source. He did say at the height of NoSQL hype. To me, that choice of words gives subtle hints as to the historical context and a disclaimer about the pertinence of that decision. Thank you for your input though, it provides food for thought for the next person mulling over a similar decision.
  - int_19h 8y ago
    It sounds like his friend didn't choose anything, his bosses did.
  - macspoofing 8y ago
    >Your friend chose wrong. HORRIBLY wrong. Yeah, obviously. Also, he didn't choose anything. The tech stack was mandated on him.
  - nickpeterson 8y ago
    Sometimes the best way to learn the value of something is to try really hard not to use it.
afarrell 8y ago
For some, reading a body of source code without any guidence about what path to walk or where to focus your attention is really quite a hard way to learn.
- fgonzag 8y ago
  It is an essential skill for our job though, being able to dive into a new code base, form a mental model of it, and modify it either because of new feature requirements or simply bug fixing. And it is a skill, I also had a lot of trouble with it initially, but diving into several open source projects was the perfect way to practice it, and it helped me tremendously in my professional development.
  - rnnr 8y ago
    Agreed. A good tip for this is to try to find the earliest version/commit of the project you are interested into.
  - PretzelFisch 8y ago
    Reading the source doesn't tell you why they chose that route or what the tradeoffs are.
    - ldng 8y ago
      Well, depends on the quality of the documentation I guess.
  - dokem 8y ago
    Most code bases are business logic and not nearly as complex or formal as a db. DBs, OSs and compilers are systems that should be studied to be effective in the code.
  - notafrog 8y ago
    I agree with you, in that it's an essential skill for our job. However, from my experience at least, I've found that going head first into the source code of something you don't really understand how it works, is not the best course of action. For example, I would make sure to read about B+ trees first, before reading the implementation of indexes in SQLite or PostgreSQL.

HappyJoy 8y ago

I've been looking for this type of book. Thanks!

zerr 8y ago

Agree! I've accidentally found this gem. It is really written from the actual hands-on engineering/programming perspective which covers all aspects, from the front-end to the actual storage implementation.

charlysl 8y ago

You may find MIT's Database Systems course helpful, the labs consist of building several db components in Java (and the prof is Stonebraker, of Postgres fame): https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010/ https://ocw.mit.edu/courses/electrical-engineering-and-compu... https://youtu.be/F3XGUPll6Qs https://youtu.be/F3XGUPll6Qs http://blog.marcua.net/post/117671929 http://blog.marcua.net/post/117671929

hestefisk 8y ago
Stonebraker is the Donald Knuth of databases.

billxreynolds 8y ago

Sciore is a great teacher! I had him at BC 20 years ago for both database systems and object oriented programming. I didn't know he put out a few books, I'll have to check them out.