4 ms·

Is this a serious article ? The state of the art of knowledge ? There are at least two (2) existing prior art implementation that have done this for years, onl

by rakoo 2y ago

Is this a serious article ? The state of the art of knowledge ? There are at least two (2) existing prior art implementation that have done this for years, only better (as in, with better tools): - recutils: https://www.gnu.org/software/recutils/ https://www.gnu.org/software/recutils/ - ndb: https://9fans.github.io/plan9port/man/man7/ndb.html https://9fans.github.io/plan9port/man/man7/ndb.html Please, developers of everywhere, I beg you: please learn what came before you before reinventing the wheel, only triangular this time. Please take the time to appreciate that if it's so obvious maybe it's because of your ignorance and not your genius.

breck 2y ago
Thank you for bringing up recutils and ndb. I had seen them years ago but didn't make the connection when writing this paper. But there are some great connections, and I will definitely be updating the post with a section and links to them. I am reading through the source and will have more to say soon. If anyone has any links to massive plain text datasets based on these (or other similar tools), I would appreciate more pointers. I can tell you now (subject to change), based on my preliminary read through the source is that the two systems you mentioned missed some highly important details that I have presented in my paper, with order of magnitude impacts. Not to discredit them at all, rather I think my work gives them credit, in that they were on the right track, and we just have the benefit of some recent innovations, and get to stand on their shoulders (and the shoulders of others). Edit: I have updated the paper with a reference to Recutils. Thanks rakoo! https://github.com/breck7/breckyunits.com/commit/71b706d296ecfa1caa3dae05d746258be520cbf5 https://github.com/breck7/breckyunits.com/commit/71b706d296e... The added text: GNU Recutils^recutils deserves credit as the closest precursor to our system. If Recutils were to adopt some designs from our system it would be capable of supporting larger databases. https://www.gnu.org/software/recutils/ ^recutils: GNU Recutils: Jose E. Marchesi https://www.gnu.org/software/recutils/ - Recutils and our system have debatable syntactic differences, but our system solves a few clear problems described in the Recutils docs: - "difficult to manage hierarchies". Hierarchies are painless in our system through nested parsers, parser inheritance, parser mixins, and nested measurements. - "tedious to manually encode...several lines". No encoding is needed in our system thanks to the indentation trick. - In Recutils comments are "completely ignored by processing tools and can only be seen by looking at the recfile itself". Our system supports first class comments which are bound to measurements using the indentation trick. - "It is difficult to manually maintain the integrity of data stored in the data base." In our system advances parsers provides unlimited capabilities for maintaining data integrity.
- rakoo 2y ago
  If your research has added value over the existing state of the art I would love to read more about it !
  - breck 2y ago
    Thanks to your pointer, I was able to explain a bit more about the advances over the SOTA. Thank you! This is the speed at which peer review should happen. If I'm lucky, I'll wake up tomorrow to someone else pointing out another precursor I overlooked.
    - rakoo 2y ago
      Thanks for the comparison ! I see there are some advances compared to recutils, those would benefit being put to the front !
stavros 2y ago
Maybe the author just independently came up with this, thought it was cool, and wanted to share? I don't know if "please do a thorough literature review before showing me things" is the right sentiment here.
- rkangel 2y ago
  > Maybe the author just independently came up with this, thought it was cool, and wanted to share? Except that the title ("A New Way to Store Knowledge") is leaning heavily on NEW.
- rakoo 2y ago
  I'm not sure the paper-like presentation of the article shows that the author was in pure discovery mode, eager to share something new and interesting. My message is an echo to earlier comments of earlier posts that talked about a similar point: nothing is ever new, everything has already been done before. If we tell ourselves we're engineers, we should be studying what came before in order to prove that the new thing is indeed better. That being said, recutils is the standard method of recording data in GNU, and ndb is the standard method of configuring stuff in Plan 9, a system that any proponent of UNIX mindset should know about. I'm not exactly talking about obscure stuff here.
  - chipdart 2y ago
    > I'm not sure the paper-like presentation of the article shows that the author was in pure discovery mode, eager to share something new and interesting. If the author was following a paper-like presentation, the author somehow skipped the section listing relevant prior work. This is something every single journal enforces, as researching prior work is the very first step any author does when they come up with something.
- vineyardmike 2y ago
  Considering the article has a “prior art” section, I assume a literature review would be appropriate. My confidence is shaken considering the sparse “prior art” section links to Apple M1 as an example of “fast file systems”.
deleted 2y ago
[deleted]

by rakoo 2y ago

breck 2y ago

Thank you for bringing up recutils and ndb. I had seen them years ago but didn't make the connection when writing this paper. But there are some great connections, and I will definitely be updating the post with a section and links to them. I am reading through the source and will have more to say soon. If anyone has any links to massive plain text datasets based on these (or other similar tools), I would appreciate more pointers. I can tell you now (subject to change), based on my preliminary read through the source is that the two systems you mentioned missed some highly important details that I have presented in my paper, with order of magnitude impacts. Not to discredit them at all, rather I think my work gives them credit, in that they were on the right track, and we just have the benefit of some recent innovations, and get to stand on their shoulders (and the shoulders of others). Edit: I have updated the paper with a reference to Recutils. Thanks rakoo! https://github.com/breck7/breckyunits.com/commit/71b706d296ecfa1caa3dae05d746258be520cbf5 https://github.com/breck7/breckyunits.com/commit/71b706d296e... The added text: GNU Recutils^recutils deserves credit as the closest precursor to our system. If Recutils were to adopt some designs from our system it would be capable of supporting larger databases. https://www.gnu.org/software/recutils/ ^recutils: GNU Recutils: Jose E. Marchesi https://www.gnu.org/software/recutils/ - Recutils and our system have debatable syntactic differences, but our system solves a few clear problems described in the Recutils docs: - "difficult to manage hierarchies". Hierarchies are painless in our system through nested parsers, parser inheritance, parser mixins, and nested measurements. - "tedious to manually encode...several lines". No encoding is needed in our system thanks to the indentation trick. - In Recutils comments are "completely ignored by processing tools and can only be seen by looking at the recfile itself". Our system supports first class comments which are bound to measurements using the indentation trick. - "It is difficult to manually maintain the integrity of data stored in the data base." In our system advances parsers provides unlimited capabilities for maintaining data integrity.

rakoo 2y ago
If your research has added value over the existing state of the art I would love to read more about it !
- breck 2y ago
  Thanks to your pointer, I was able to explain a bit more about the advances over the SOTA. Thank you! This is the speed at which peer review should happen. If I'm lucky, I'll wake up tomorrow to someone else pointing out another precursor I overlooked.
  - rakoo 2y ago
    Thanks for the comparison ! I see there are some advances compared to recutils, those would benefit being put to the front !

stavros 2y ago

Maybe the author just independently came up with this, thought it was cool, and wanted to share? I don't know if "please do a thorough literature review before showing me things" is the right sentiment here.

rkangel 2y ago
> Maybe the author just independently came up with this, thought it was cool, and wanted to share? Except that the title ("A New Way to Store Knowledge") is leaning heavily on NEW.
rakoo 2y ago
I'm not sure the paper-like presentation of the article shows that the author was in pure discovery mode, eager to share something new and interesting. My message is an echo to earlier comments of earlier posts that talked about a similar point: nothing is ever new, everything has already been done before. If we tell ourselves we're engineers, we should be studying what came before in order to prove that the new thing is indeed better. That being said, recutils is the standard method of recording data in GNU, and ndb is the standard method of configuring stuff in Plan 9, a system that any proponent of UNIX mindset should know about. I'm not exactly talking about obscure stuff here.
- chipdart 2y ago
  > I'm not sure the paper-like presentation of the article shows that the author was in pure discovery mode, eager to share something new and interesting. If the author was following a paper-like presentation, the author somehow skipped the section listing relevant prior work. This is something every single journal enforces, as researching prior work is the very first step any author does when they come up with something.
vineyardmike 2y ago
Considering the article has a “prior art” section, I assume a literature review would be appropriate. My confidence is shaken considering the sparse “prior art” section links to Apple M1 as an example of “fast file systems”.

deleted 2y ago

[deleted]