3 ms·

Are you aware of https://github.com/bup/bup https://github.com/bup/bup? It's a git file format based backup incremental backup tool. Do you also use rolling ch

Are you aware of https://github.com/bup/bup https://github.com/bup/bup? It's a git file format based backup incremental backup tool. Do you also use rolling checksums (like bup) to prevent re-storing data when only a few bytes have changed? --> What are the major use cases you imagine for noms? I've read the section on Github, but can you give some specific examples that you see as good use cases?

aboodman 10y ago
Yes, we credited bup in various places, such as the design overview: https://github.com/attic-labs/noms/blob/master/doc/intro.md https://github.com/attic-labs/noms/blob/master/doc/intro.md We were also heavily influenced by camlistore (which I hacked on for awhile), irmin, ipfs, and others who have done a lot of interesting work in this space. We do use rolling checksums, but I think we have done some novel work here: https://github.com/attic-labs/noms/blob/master/doc/intro.md#prolly-trees-probabilistic-b-trees https://github.com/attic-labs/noms/blob/master/doc/intro.md#...
- pinko 10y ago
  FYI, the "HTTP protocol" link[1] in the intro page above is broken. PS: I love the name "Prolly Tree". [1] <https://github.com/attic-labs/noms/blob/master/datas/database_server.go> https://github.com/attic-labs/noms/blob/master/datas/databas...
  - aboodman 10y ago
    :) Thanks, we like it too.
- ripa 10y ago
  bup is focused on backups. Would you say that noms is useful as a backup utility? Do you have any mechanisms for securing integrity, specifically repairing the store in case of inconsistencies? Is there any plans to support any data retention policy/functionality?
  - aboodman 10y ago
    Noms should be useful as a backup utility, but I'd say it's especially useful for backing up data which is not files. Think about backing up data which you only have access to via API. You can take the JSON output of an API and drop it into Noms, then do the same thing tomorrow, and Noms will automatically deduplicate the data as well as give you a nice structured API to read and interact with it. We have an example of this here: https://github.com/attic-labs/noms/tree/master/samples/js/flickr https://github.com/attic-labs/noms/tree/master/samples/js/fl... but it's not working atm due to a bug introduced right before launch. You can look at the code though.

aboodman 10y ago

Yes, we credited bup in various places, such as the design overview: https://github.com/attic-labs/noms/blob/master/doc/intro.md https://github.com/attic-labs/noms/blob/master/doc/intro.md We were also heavily influenced by camlistore (which I hacked on for awhile), irmin, ipfs, and others who have done a lot of interesting work in this space. We do use rolling checksums, but I think we have done some novel work here: https://github.com/attic-labs/noms/blob/master/doc/intro.md#prolly-trees-probabilistic-b-trees https://github.com/attic-labs/noms/blob/master/doc/intro.md#...

pinko 10y ago
FYI, the "HTTP protocol" link[1] in the intro page above is broken. PS: I love the name "Prolly Tree". [1] <https://github.com/attic-labs/noms/blob/master/datas/database_server.go> https://github.com/attic-labs/noms/blob/master/datas/databas...
- aboodman 10y ago
  :) Thanks, we like it too.
ripa 10y ago
bup is focused on backups. Would you say that noms is useful as a backup utility? Do you have any mechanisms for securing integrity, specifically repairing the store in case of inconsistencies? Is there any plans to support any data retention policy/functionality?
- aboodman 10y ago
  Noms should be useful as a backup utility, but I'd say it's especially useful for backing up data which is not files. Think about backing up data which you only have access to via API. You can take the JSON output of an API and drop it into Noms, then do the same thing tomorrow, and Noms will automatically deduplicate the data as well as give you a nice structured API to read and interact with it. We have an example of this here: https://github.com/attic-labs/noms/tree/master/samples/js/flickr https://github.com/attic-labs/noms/tree/master/samples/js/fl... but it's not working atm due to a bug introduced right before launch. You can look at the code though.