4 ms·

Hi everyone, I'm one of the core contributors to Dat, @joeahand. Happy to answer any questions. It's an interesting time to see this posted because I've been wo

Hi everyone, I'm one of the core contributors to Dat, @joeahand. Happy to answer any questions. It's an interesting time to see this posted because I've been working on a new datproject.org site recently =). Dat Project started with a focus on increasing access to research & public data. To support the data tools we built a peer-to-peer protocol. People are doing some really cool stuff on top of Dat (such as Beaker Browser), we're really excited about it and want to make sure to support all the neat use cases. We'll be launching an updated site soon to highlight more of the work around the protocol and what the community is building. Our main use case will still be data management but most of what you see on the current site will shift to a new domain.

batbomb 9y ago
How are you going to beat Globus for research data of appreciable size?
- filiwickers 9y ago
  We'd love to work with them! We really like what they've done and we have a few partners that use Globus. Underneath all the transfers in Globus use GridFTP. Using Dat could help distribute bandwidth and speed up transfers. It'll also add version control for free, which (I think) Globus does not have yet.
rakoo 9y ago
I've glanced at the docs before already, and dat seemed super cool except for one little thing: it seems that there can only be one source that is allowed to modify the dataset. Is this by design or am I missing something ?
- tbv 9y ago
  That’s correct. But the protocol author mafintosh is currently working on a major new release which introduces multi-writer capabilities.
baldfat 9y ago
Care to explain how this is different then Resilio https://www.resilio.com/ https://www.resilio.com/ I use this with encryption for my data folders on my projects.
- filiwickers 9y ago
  Ya, there are a few other related questions below. Resilio is BitTorrent-based. But I'm not 100% familiar with how Resilio differs from BitTorrent. The core difference is in our approach. We're all open source and a non-profit. We're also really focused on the research data use case where BitTorrent is less easily deployed. We hope that making an open and easy to use p2p protocol will enable other developers to build applications on top, and something like Resilio could be one.
molszanski 9y ago
It looks like to me, from the sources, that you see plain http/tcp connection. BitTorrent is using UDP that could be a huge gain in many situations. Do you plan to add some Realiable UDP overlay to speed things up? Something like Aspera but more open? Maybe QUIC is mature enough?
- filiwickers 9y ago
  We use TCP or UDP for all the connections. UDP is especially helpful for hole-punching as well. The direct HTTP support is not fully supported yet. But we're excited for that because it'll allow you to use S3 or other static file servers as peers. Dat works over any protocol, so it's just a matter of implementing it.
  - molszanski 9y ago
    Thanks! I usseme i isimplemented in mafintosh/hyperdrive?

batbomb 9y ago

How are you going to beat Globus for research data of appreciable size?

filiwickers 9y ago
We'd love to work with them! We really like what they've done and we have a few partners that use Globus. Underneath all the transfers in Globus use GridFTP. Using Dat could help distribute bandwidth and speed up transfers. It'll also add version control for free, which (I think) Globus does not have yet.

rakoo 9y ago

I've glanced at the docs before already, and dat seemed super cool except for one little thing: it seems that there can only be one source that is allowed to modify the dataset. Is this by design or am I missing something ?

tbv 9y ago
That’s correct. But the protocol author mafintosh is currently working on a major new release which introduces multi-writer capabilities.

baldfat 9y ago

Care to explain how this is different then Resilio https://www.resilio.com/ https://www.resilio.com/ I use this with encryption for my data folders on my projects.

filiwickers 9y ago
Ya, there are a few other related questions below. Resilio is BitTorrent-based. But I'm not 100% familiar with how Resilio differs from BitTorrent. The core difference is in our approach. We're all open source and a non-profit. We're also really focused on the research data use case where BitTorrent is less easily deployed. We hope that making an open and easy to use p2p protocol will enable other developers to build applications on top, and something like Resilio could be one.

molszanski 9y ago

It looks like to me, from the sources, that you see plain http/tcp connection. BitTorrent is using UDP that could be a huge gain in many situations. Do you plan to add some Realiable UDP overlay to speed things up? Something like Aspera but more open? Maybe QUIC is mature enough?

filiwickers 9y ago
We use TCP or UDP for all the connections. UDP is especially helpful for hole-punching as well. The direct HTTP support is not fully supported yet. But we're excited for that because it'll allow you to use S3 or other static file servers as peers. Dat works over any protocol, so it's just a matter of implementing it.
- molszanski 9y ago
  Thanks! I usseme i isimplemented in mafintosh/hyperdrive?