5 ms·

I don't. But I interpret posting something here as a request for comments and this was the obvious one for me because there is no clear indication that this is

by danbruc 12y ago

I don't. But I interpret posting something here as a request for comments and this was the obvious one for me because there is no clear indication that this is just a fun project. The only thing I sometimes worry about - and this is a quite good example - is if it is really a good idea to have all the weekend fun projects in the public. You want to know if there are some interesting new databases or libraries for something? Search for it and you get an ever increasing list of results with 99 % useless weekend hacks. I am not sure how it will turn out in the long run but there might be some real danger of polluting the internet with evermore useless stuff and bad code.

omni 12y ago
You think the Internet isn't already 99% useless garbage? Sifting through it is the entire point of intelligent search engines.
- danbruc 12y ago
  Of course, it is. But when it comes to source code it is a pretty recent development that everything ends up in the public.
  - hellbanner 12y ago
    Maybe someone should setup a database linking Key: <type of library> with Value: <list of libraries that do some thing>, sorted by rank voted on by the community.
    - alphapapa 12y ago
      This isn't exactly what you had in mind, but if you follow enough of the links... https://github.com/asciimoo/ListOfGithubLists https://github.com/asciimoo/ListOfGithubLists
      - hellbanner 12y ago
        Hey that's great! Bookmarked (and GH starred!)
jhedwards 12y ago
I highly doubt people will have trouble distinguishing my project from production-ready databases.
- danbruc 12y ago
  Of course not and I didn't want to imply that. But I tried to survey the database landscape a year or so ago and a lot of the effort went into figuring out whether a system deserves a closer look or is an abandoned weekend hack. But there is obviously no obligation for anyone to make my work easy. And the comment was not specifically aimed at your project, but is a very general thought about potentially negative effects of having an ever increasing amount of code in the public. Similar to the situation for satellites and space junk.
robertfw 12y ago
"Seriously" is hardly a constructive comment
- danbruc 12y ago
  While it has a negative connotation it was a serious question because it was not obvious to me whether this is a serious project or not. But besides that what should a constructive comment to this look like? The design is just bad and I really hate watering down opinions. I prefer people telling me that I am the worst singer in the world a thousand times over ending up at American Idol with illusion that I am a good singer. Be as constructive as possible but don't let this make you dishonest.
  - jhedwards 12y ago
    I would have preferred you said something like "hey nice try, but the overhead of compiling a c file every time you query will quickly make the database unusable" or "this design limits the database to very small data sets" or anything, really, that could help me understand what is so bad about the design. It was an experiment, it seemed to work well, so I posted it here for feedback. I wouldn't mind if someone told be it was terrible as long they did it respectfully and gave me some clue as to why they thought so.
    - danbruc 12y ago
      Let me try to show you my point of view. I see a link on Hacker News about a new database. Now I think there is someone who is serious about developing a database and is confident enough about his work to present it to the public and to this forum in particular. I naturally expect that author has a good understanding of database systems because I think he will have researched the topic before he started building something new. And there is really a lot of literature on databases. The dominant standard, the relational model now often more or less synonymous with SQL, is 45 years old, the first »modern« database systems are more than 50 years old like IDS [1] released in 1964. But even those are usually referred to as 3rd generation systems with a decade or so of development on their back. And if you want to you could go even further back, at least as far as 1884, 130 years, and the Hollerith machines which are at the roots of IBM. And with this expectation I looked at your project and found something that is probably the most convoluted ways to implement an database that I have seen. Really, don't take it personal, but there is not much that resembles a usual database system design and there is nothing where I honestly could have said that it is a nice idea but it could be improved by doing this or that. I could have suggested to read the Wikipedia article on databases [2] and work from there, but would this really have sounded less rude? [1] http://en.wikipedia.org/wiki/Integrated_Data_Store http://en.wikipedia.org/wiki/Integrated_Data_Store [2] http://en.wikipedia.org/wiki/Database http://en.wikipedia.org/wiki/Database
jmadsen 12y ago
Is that your idea of a "comment"? "Seriously?" Perhaps going beyond the first word and explaining your thoughts would have been worth more of everyone's time?
- danbruc 12y ago
  See my other comment [1], I don't think I could have done it much better. [1] https://news.ycombinator.com/item?id=9403780 https://news.ycombinator.com/item?id=9403780
  - jhedwards 12y ago
    OK I guess I see your point. I understand that you dislike seeing an experiment like this walking around like a real database, but on my end I don't see the downside of trying something that's radically different from the established dogma. I'll probably read that book you recommended and try something more orthodox next time for good measure, but in the meantime I'm still really interested in exploring the limitations of this design.
    - danbruc 12y ago
      Here are some things you will probably notice. Start with a really small database, maybe 10,000 facts. How long does a query take? Half a second? One second? I am just guessing here. It will take less than a millisecond on a usual SQL database. Now increase the size of the database to say 100,000 facts. This will make essentially no difference for a normal database because they will likely be able to answer the query using an index in time logarithmic in the number of facts but it will take NebulaDB ten times longer to answer a query because GCC has to compile a ten times larger file. If there is some quadratic code somewhere in the system it may actually take a hundred times longer. Every query requires processing all the data while a usual database will only touch the data relevant to answer the query. Another thing are updates and deletions. I did not read the code in enough detail but can you support that? If yes it sounds a bit like a nightmare to me to find and remove or update the relevant lines in the C source code file. And it will be very likely a slow process because if the length of a fact changes you will have to move all the later facts.
ketralnis 12y ago
Sorry about all of that free labour and code being offered to you. We'll all try to tone down the creativity so you don't have to think critically in the future

by danbruc 12y ago

omni 12y ago

You think the Internet isn't already 99% useless garbage? Sifting through it is the entire point of intelligent search engines.

danbruc 12y ago
Of course, it is. But when it comes to source code it is a pretty recent development that everything ends up in the public.
- hellbanner 12y ago
  Maybe someone should setup a database linking Key: <type of library> with Value: <list of libraries that do some thing>, sorted by rank voted on by the community.
  - alphapapa 12y ago
    This isn't exactly what you had in mind, but if you follow enough of the links... https://github.com/asciimoo/ListOfGithubLists https://github.com/asciimoo/ListOfGithubLists
    - hellbanner 12y ago
      Hey that's great! Bookmarked (and GH starred!)

jhedwards 12y ago

I highly doubt people will have trouble distinguishing my project from production-ready databases.

danbruc 12y ago
Of course not and I didn't want to imply that. But I tried to survey the database landscape a year or so ago and a lot of the effort went into figuring out whether a system deserves a closer look or is an abandoned weekend hack. But there is obviously no obligation for anyone to make my work easy. And the comment was not specifically aimed at your project, but is a very general thought about potentially negative effects of having an ever increasing amount of code in the public. Similar to the situation for satellites and space junk.

robertfw 12y ago

"Seriously" is hardly a constructive comment

danbruc 12y ago
While it has a negative connotation it was a serious question because it was not obvious to me whether this is a serious project or not. But besides that what should a constructive comment to this look like? The design is just bad and I really hate watering down opinions. I prefer people telling me that I am the worst singer in the world a thousand times over ending up at American Idol with illusion that I am a good singer. Be as constructive as possible but don't let this make you dishonest.
- jhedwards 12y ago
  I would have preferred you said something like "hey nice try, but the overhead of compiling a c file every time you query will quickly make the database unusable" or "this design limits the database to very small data sets" or anything, really, that could help me understand what is so bad about the design. It was an experiment, it seemed to work well, so I posted it here for feedback. I wouldn't mind if someone told be it was terrible as long they did it respectfully and gave me some clue as to why they thought so.
  - danbruc 12y ago
    Let me try to show you my point of view. I see a link on Hacker News about a new database. Now I think there is someone who is serious about developing a database and is confident enough about his work to present it to the public and to this forum in particular. I naturally expect that author has a good understanding of database systems because I think he will have researched the topic before he started building something new. And there is really a lot of literature on databases. The dominant standard, the relational model now often more or less synonymous with SQL, is 45 years old, the first »modern« database systems are more than 50 years old like IDS [1] released in 1964. But even those are usually referred to as 3rd generation systems with a decade or so of development on their back. And if you want to you could go even further back, at least as far as 1884, 130 years, and the Hollerith machines which are at the roots of IBM. And with this expectation I looked at your project and found something that is probably the most convoluted ways to implement an database that I have seen. Really, don't take it personal, but there is not much that resembles a usual database system design and there is nothing where I honestly could have said that it is a nice idea but it could be improved by doing this or that. I could have suggested to read the Wikipedia article on databases [2] and work from there, but would this really have sounded less rude? [1] http://en.wikipedia.org/wiki/Integrated_Data_Store http://en.wikipedia.org/wiki/Integrated_Data_Store [2] http://en.wikipedia.org/wiki/Database http://en.wikipedia.org/wiki/Database

jmadsen 12y ago

Is that your idea of a "comment"? "Seriously?" Perhaps going beyond the first word and explaining your thoughts would have been worth more of everyone's time?

danbruc 12y ago
See my other comment [1], I don't think I could have done it much better. [1] https://news.ycombinator.com/item?id=9403780 https://news.ycombinator.com/item?id=9403780
- jhedwards 12y ago
  OK I guess I see your point. I understand that you dislike seeing an experiment like this walking around like a real database, but on my end I don't see the downside of trying something that's radically different from the established dogma. I'll probably read that book you recommended and try something more orthodox next time for good measure, but in the meantime I'm still really interested in exploring the limitations of this design.
  - danbruc 12y ago
    Here are some things you will probably notice. Start with a really small database, maybe 10,000 facts. How long does a query take? Half a second? One second? I am just guessing here. It will take less than a millisecond on a usual SQL database. Now increase the size of the database to say 100,000 facts. This will make essentially no difference for a normal database because they will likely be able to answer the query using an index in time logarithmic in the number of facts but it will take NebulaDB ten times longer to answer a query because GCC has to compile a ten times larger file. If there is some quadratic code somewhere in the system it may actually take a hundred times longer. Every query requires processing all the data while a usual database will only touch the data relevant to answer the query. Another thing are updates and deletions. I did not read the code in enough detail but can you support that? If yes it sounds a bit like a nightmare to me to find and remove or update the relevant lines in the C source code file. And it will be very likely a slow process because if the length of a fact changes you will have to move all the later facts.

ketralnis 12y ago

Sorry about all of that free labour and code being offered to you. We'll all try to tone down the creativity so you don't have to think critically in the future