6 ms·

Show HN: NebulaDB, my first attempt at a database

30 points by jhedwards 12y ago

danbruc 12y ago
How Does it Work? NebulaDB manages a c file that is full of structs that have arrays of pointers to links. When you save a query new structs are added to the file along with c functions that configure the links. When you query the data, the function that corresponds to your query is appended to the c file, the file is compiled and the output is parsed into JSON downstream. Seriously?
- Mahn 12y ago
  That doesn't sound very efficient. OP, what is the rationale behind this?
  - ChiperSoft 12y ago
    If OP just made it for fun, it doesn't have to be efficient
    - Mahn 12y ago
      Sure, but it's such an odd design choice that one can't help but wonder why, even if it was for fun only.
      - alphapapa 12y ago
        People come up with all kinds of "odd choices" when they haven't been indoctrinated with existing dogma (I don't necessarily mean that in a negative way, despite the connotations). Sometimes that's how new breakthroughs happen, by bringing fresh eyes to bear on an old problem. Whether sharing this particular example here was brave or stupid is up for grabs though... ;) (No offense to the OP intended. I think it's an interesting example of thinking outside the box, and I'm sure you're learning a lot more than you would going through a bunch of online tutorials!)
  - jhedwards 12y ago
    Well, I've only been programming for two years and I'm new to advanced concepts like compiling etc. There aren't really simple tutorials on how to make a database afaik, so I figured I'd just think something up and try it out. I realized it was an inefficient design, but I didn't know if it would be unusably inefficient or just slow compared to programs written by teams of CS majors. Only one way to find out, as they say.
    - theseoafs 12y ago
      > Well, I've only been programming for two years and I'm new to advanced concepts like compiling etc. What exactly have you been doing for two years if you think compiling is an "advanced concept"? By the way, your functions go in the .c file, not in the .h file.
      - jhedwards 12y ago
        Weird, most programmers I know consider compilers to be at least somewhat advanced. To answer your question, two years ago I knew nothing about programming, I was poor, living in China, with no marketable skills. I had to teach myself everything in my spare time. In that time I mostly studied frameworks, algorithms, data structures, the quirks of JS etc. I managed to do well enough that I'm now working 9-5 in silicon valley making UI for medical software. Unfortunately, I have never had the opportunity to learn about compilers or database internals.
        danbruc 12y ago
        I guess he referred to using a compiler, not writing one. The later is obviously one of the more advanced topics in computer science.
    - danbruc 12y ago
      There are a lot of books on database system design like »Fundamentals of Database Systems« [1] by Elmasri and Navathe. The reason there is no such thing as a simple tutorial is because any serious attempt to build a database system will consume on the order of tens or hundreds of men-years of effort if you start from scratch. [1] http://www.amazon.com/Fundamentals-Database-Systems-Ramez-Elmasri/dp/0136086209 http://www.amazon.com/Fundamentals-Database-Systems-Ramez-El...
- xeromal 12y ago
  Probably just a fun project. Don't get your panties in a wad.
  - danbruc 12y ago
    I don't. But I interpret posting something here as a request for comments and this was the obvious one for me because there is no clear indication that this is just a fun project. The only thing I sometimes worry about - and this is a quite good example - is if it is really a good idea to have all the weekend fun projects in the public. You want to know if there are some interesting new databases or libraries for something? Search for it and you get an ever increasing list of results with 99 % useless weekend hacks. I am not sure how it will turn out in the long run but there might be some real danger of polluting the internet with evermore useless stuff and bad code.
    - omni 12y ago
      You think the Internet isn't already 99% useless garbage? Sifting through it is the entire point of intelligent search engines.
      - danbruc 12y ago
        Of course, it is. But when it comes to source code it is a pretty recent development that everything ends up in the public.
        hellbanner 12y ago
        Maybe someone should setup a database linking Key: <type of library> with Value: <list of libraries that do some thing>, sorted by rank voted on by the community.
        alphapapa 12y ago
        This isn't exactly what you had in mind, but if you follow enough of the links... https://github.com/asciimoo/ListOfGithubLists https://github.com/asciimoo/ListOfGithubLists
- xrstf 12y ago
  Oh how I love it to fetch a package via NPM, thinking it's gonna be JS-only and then being greeted by compiler errors because I don't have a C compiler setup on my Windows machine. Without falling on my face or reading the very end of the README, I would not know that I need C to run this Node package. Especially since the README says > NebulaDB runs on a Node server. I wish there was a way to declare non-Node dependencies in the package.json.
monk_e_boy 12y ago
Your 'syntax' is very hard to read: ['* ','->','admin'] Couldn't you write a little parser to understand this: " * -> admin" I doubt the overhead would be much. [edit] a * is hard to get into a HN comment
- jhedwards 12y ago
  Good idea, I think I'll do that. Thanks!
amirouche 12y ago
It looks like a "datalog database"
yid 12y ago
Ignore the negativity in this thread, OP. The rough design you've come up with is called a "triple store". You might find it interesting to learn about how powerful the representation is. Start here: http://en.m.wikipedia.org/wiki/Triplestore http://en.m.wikipedia.org/wiki/Triplestore

30 points by jhedwards 12y ago

How Does it Work? NebulaDB manages a c file that is full of structs that have arrays of pointers to links. When you save a query new structs are added to the file along with c functions that configure the links. When you query the data, the function that corresponds to your query is appended to the c file, the file is compiled and the output is parsed into JSON downstream. Seriously?

Mahn 12y ago
That doesn't sound very efficient. OP, what is the rationale behind this?
- ChiperSoft 12y ago
  If OP just made it for fun, it doesn't have to be efficient
  - Mahn 12y ago
    Sure, but it's such an odd design choice that one can't help but wonder why, even if it was for fun only.
    - alphapapa 12y ago
      People come up with all kinds of "odd choices" when they haven't been indoctrinated with existing dogma (I don't necessarily mean that in a negative way, despite the connotations). Sometimes that's how new breakthroughs happen, by bringing fresh eyes to bear on an old problem. Whether sharing this particular example here was brave or stupid is up for grabs though... ;) (No offense to the OP intended. I think it's an interesting example of thinking outside the box, and I'm sure you're learning a lot more than you would going through a bunch of online tutorials!)
- jhedwards 12y ago
  Well, I've only been programming for two years and I'm new to advanced concepts like compiling etc. There aren't really simple tutorials on how to make a database afaik, so I figured I'd just think something up and try it out. I realized it was an inefficient design, but I didn't know if it would be unusably inefficient or just slow compared to programs written by teams of CS majors. Only one way to find out, as they say.
  - theseoafs 12y ago
    > Well, I've only been programming for two years and I'm new to advanced concepts like compiling etc. What exactly have you been doing for two years if you think compiling is an "advanced concept"? By the way, your functions go in the .c file, not in the .h file.
    - jhedwards 12y ago
      Weird, most programmers I know consider compilers to be at least somewhat advanced. To answer your question, two years ago I knew nothing about programming, I was poor, living in China, with no marketable skills. I had to teach myself everything in my spare time. In that time I mostly studied frameworks, algorithms, data structures, the quirks of JS etc. I managed to do well enough that I'm now working 9-5 in silicon valley making UI for medical software. Unfortunately, I have never had the opportunity to learn about compilers or database internals.
      - danbruc 12y ago
        I guess he referred to using a compiler, not writing one. The later is obviously one of the more advanced topics in computer science.
  - danbruc 12y ago
    There are a lot of books on database system design like »Fundamentals of Database Systems« [1] by Elmasri and Navathe. The reason there is no such thing as a simple tutorial is because any serious attempt to build a database system will consume on the order of tens or hundreds of men-years of effort if you start from scratch. [1] http://www.amazon.com/Fundamentals-Database-Systems-Ramez-Elmasri/dp/0136086209 http://www.amazon.com/Fundamentals-Database-Systems-Ramez-El...
xeromal 12y ago
Probably just a fun project. Don't get your panties in a wad.
- danbruc 12y ago
  I don't. But I interpret posting something here as a request for comments and this was the obvious one for me because there is no clear indication that this is just a fun project. The only thing I sometimes worry about - and this is a quite good example - is if it is really a good idea to have all the weekend fun projects in the public. You want to know if there are some interesting new databases or libraries for something? Search for it and you get an ever increasing list of results with 99 % useless weekend hacks. I am not sure how it will turn out in the long run but there might be some real danger of polluting the internet with evermore useless stuff and bad code.
  - omni 12y ago
    You think the Internet isn't already 99% useless garbage? Sifting through it is the entire point of intelligent search engines.
    - danbruc 12y ago
      Of course, it is. But when it comes to source code it is a pretty recent development that everything ends up in the public.
      - hellbanner 12y ago
        Maybe someone should setup a database linking Key: <type of library> with Value: <list of libraries that do some thing>, sorted by rank voted on by the community.
        alphapapa 12y ago
        This isn't exactly what you had in mind, but if you follow enough of the links... https://github.com/asciimoo/ListOfGithubLists https://github.com/asciimoo/ListOfGithubLists
xrstf 12y ago
Oh how I love it to fetch a package via NPM, thinking it's gonna be JS-only and then being greeted by compiler errors because I don't have a C compiler setup on my Windows machine. Without falling on my face or reading the very end of the README, I would not know that I need C to run this Node package. Especially since the README says > NebulaDB runs on a Node server. I wish there was a way to declare non-Node dependencies in the package.json.

monk_e_boy 12y ago

Your 'syntax' is very hard to read: ['* ','->','admin'] Couldn't you write a little parser to understand this: " * -> admin" I doubt the overhead would be much. [edit] a * is hard to get into a HN comment

jhedwards 12y ago
Good idea, I think I'll do that. Thanks!

amirouche 12y ago

It looks like a "datalog database"

yid 12y ago

Ignore the negativity in this thread, OP. The rough design you've come up with is called a "triple store". You might find it interesting to learn about how powerful the representation is. Start here: http://en.m.wikipedia.org/wiki/Triplestore http://en.m.wikipedia.org/wiki/Triplestore