6 ms·

Kind of stagnant, but I liked the IDEA of what was being done with the Beaker Browser using the DAT protocol to run a self-hosted P2P web. I just wish someone w

by JakeAl 6y ago

Kind of stagnant, but I liked the IDEA of what was being done with the Beaker Browser using the DAT protocol to run a self-hosted P2P web. I just wish someone would take the project to completion/usability. It would really disrupt things. https://beakerbrowser.com https://beakerbrowser.com

karissa 6y ago
They are working on a new version, it's almost done. It's going to be a lot better! https://github.com/beakerbrowser/beaker/pull/1435 https://github.com/beakerbrowser/beaker/pull/1435
Taek 6y ago
Skynet is a new protocol that has been working on something like this. You can see some of the webapps built on Skynet here: https://skynethub.io/_B19P18DEI4Y_37wg6yXtfulEq2U8AcIO_lWM7sKncC1rw/index.html#/apps/all https://skynethub.io/_B19P18DEI4Y_37wg6yXtfulEq2U8AcIO_lWM7s... Main project website: https://siasky.net https://siasky.net
- karissa 6y ago
  I don't see a github repository or any link to the open source code there. Is it open source?
  - soupbowl 6y ago
    I found this on their website: https://github.com/NebulousLabs/skynet-webportal https://github.com/NebulousLabs/skynet-webportal
- koeng 6y ago
  Is there anyway to pin skynet files using storage rather than Sia, or a way to download skynet files using the command line (independent of a curl request from a portal)?
  - Taek 6y ago
    For your first question: technically yes, but it's not really the way it's meant to be done. If you want to pin something with storage, you can become a host on the Sia network and then make that content available through your host. But this also means the content is subject to going offline if you go offline (unless someone has re-pinned it to other hosts), and Skynet is really meant to be this high reliability platform where you don't need to worry about particular users going offline. If you want to download skynet files using the command line, you can run 'siad' in portal mode as a daemon, and then use 'siac' to query the daemon. 'siac skynet download [skylink] [destination]' is the command to perform the download. Doing it this way means you gain direct decentralized access to Skynet instead of needing to funnel through a portal.
    - koeng 6y ago
      Is there a way to be a host and then pin other people's content that I like? I'd love to be a host that pins public content without an explicit financial transaction.
- faizshah 6y ago
  For me one of the main positives of dat is there isn’t an attached cryptocurrency.
  - Taek 6y ago
    Attaching a cryptocurrency to Sia allows us to draw a distinction between producers and consumers. In typical p2p networks, you are expected to be both in roughly equal proportions, which means your network gets polluted by a lot of low-quality users who are just trying to do their part. Sia has the consumers pay the producers, and has a marketplace mechanism that selects the highest performing producers to perform the jobs and receive the revenue. This allows the network as a whole to be much, much more efficient.
    - theamk 6y ago
      Wait, consumers pay the producers? As in, “you need to pay to download files”?
      - anaganisk 6y ago
        Doesn't not paying make us end up rebuilding the internet again with ads and tracking? Free is not always an option.
        theamk 6y ago
        Doesn't paying encourage useless sites highly optimized to grab user attention, low quality content, shock sites, cloned sites, site bloat (if payment is per KB), and so on? That said, I think there's definitely a space for paid internet. I was just surprised because siasky homepage says "Build a Free Internet" - perhaps it should say "Build a pay per view internet" instead?
        Taek 6y ago
        Free as in freedom, not free as in beer. I would argue that the current model is more friendly to attention grabbing content (clickbait, etc) because advertising has the limitation that all views are worth the same value. In a pay-as-you-go Internet, users can whitelist high quality content sources as being okay to charge 10x or 100x what you'd typical accept to view a webpage. This would incentive content creators to build a brand and reputation that makes users comfortable putting them on the 'high quality' list, so that their content can see a massive revenue multiple relative to the number of eyeballs.
pfraze 6y ago
We decided to take a year to go heads down and rework a lot of stuff, so we did get publicly stagnant but we're fulltime on the next version and should release a public beta soon. I tweet a lot about our progress if you can sort through my sillier tweets (@pfrazee). Past year has had a lot of improvements - Protocol moved to a hole-punching DHT for peer discovery (hyperswarm). - Protocol now scales # of writes and # of files much better. We were able to put a Wikipedia export, which is millions of files in 2 flat dirs, into a single "drive" and get good read performance. This performance bump came from an indexing structure that's built into every log entry (hypertrie). - Protocol now supports "mounting" which is a way to basically symlink drives onto each other. Good composition tool, esp useful for mounting deps in a /vendor directory. - Browser now has a builtin editor that splits the screen for live editing. Feels similar to hackmd. - Browser added a bash-like terminal for working with the protocol's filespace. It's glued to the current page so you can drive around the web using `cd`. - Browser added a basic identity system. Every user has a profile site that's created automatically and maintains an address book of other users. - We built out application tooling a fair amount. It's fairly easy to build multi-user apps now, where previously it was a bit of rocket surgery. Some of the year was spent prototyping ideas and throwing them away as well. A bit inefficient, but helped us learn.
- grng3r 6y ago
  I've been reading about dat and doing some basic projects for about a year in my spare time, stagnating a bit lately as am finishing my degree. As I remember there was some talk about Python and Rust implementations at the time, is that still under development as well? I am not as enthusiastic with Js but really like the protocol and the idea behind it.... Thanks
  - pfraze 6y ago
    Yeah I do know there's a Rust impl that people are working on
- koeng 6y ago
  I’ve been thinking of deploying dat for Nanopore DNA sequencing, but recall something about dat not being great at handling that kind of data. Does the newest version of dat handle large files well (10gb)? Does it handle tons of files nested in a few directories well? Are there any issues I should know about there? What is the command line support like for multi-writer? Do you have any metrics for how much Dat is currently being used? Thanks!
anchpop 6y ago
You say you like the idea, do you have a problem with the implementation?
__ka 6y ago
The Cliqz browser implements the Dat protocol. Here's a writeup on the implementation: https://0x65.dev/blog/2020-03-02/implementing-the-dat-protocol-in-cliqz.html https://0x65.dev/blog/2020-03-02/implementing-the-dat-protoc... Disclosure: I work at Cliqz.
- rakoo 6y ago
  Amazing ! Dat is probably one of the most interesting technologies out there, it's good to see more people step in, especially if it's a growing company. Does that mean that Cliqz will start indexing dat sites ? The way I see it, the biggest issue of decentralized anything is always discoverability. If Cliqz is interested in it, it could have a leverage in making dat more widespread. Dat even helps you with the indexing, because it's the sites themselves that tell you when they are updated and when they are updated, it's like an indexer's dream come true. And seeing how dat operates, Cliqz could even be a peer in the swarm of the sites it indexes to give them more chance to be reached (at the cost of Cliqz having a view on who visits what, which seems contrary to your DNA if I understand correctly)

by JakeAl 6y ago

karissa 6y ago

They are working on a new version, it's almost done. It's going to be a lot better! https://github.com/beakerbrowser/beaker/pull/1435 https://github.com/beakerbrowser/beaker/pull/1435

Taek 6y ago

Skynet is a new protocol that has been working on something like this. You can see some of the webapps built on Skynet here: https://skynethub.io/_B19P18DEI4Y_37wg6yXtfulEq2U8AcIO_lWM7sKncC1rw/index.html#/apps/all https://skynethub.io/_B19P18DEI4Y_37wg6yXtfulEq2U8AcIO_lWM7s... Main project website: https://siasky.net https://siasky.net

karissa 6y ago
I don't see a github repository or any link to the open source code there. Is it open source?
- soupbowl 6y ago
  I found this on their website: https://github.com/NebulousLabs/skynet-webportal https://github.com/NebulousLabs/skynet-webportal
koeng 6y ago
Is there anyway to pin skynet files using storage rather than Sia, or a way to download skynet files using the command line (independent of a curl request from a portal)?
- Taek 6y ago
  For your first question: technically yes, but it's not really the way it's meant to be done. If you want to pin something with storage, you can become a host on the Sia network and then make that content available through your host. But this also means the content is subject to going offline if you go offline (unless someone has re-pinned it to other hosts), and Skynet is really meant to be this high reliability platform where you don't need to worry about particular users going offline. If you want to download skynet files using the command line, you can run 'siad' in portal mode as a daemon, and then use 'siac' to query the daemon. 'siac skynet download [skylink] [destination]' is the command to perform the download. Doing it this way means you gain direct decentralized access to Skynet instead of needing to funnel through a portal.
  - koeng 6y ago
    Is there a way to be a host and then pin other people's content that I like? I'd love to be a host that pins public content without an explicit financial transaction.
faizshah 6y ago
For me one of the main positives of dat is there isn’t an attached cryptocurrency.
- Taek 6y ago
  Attaching a cryptocurrency to Sia allows us to draw a distinction between producers and consumers. In typical p2p networks, you are expected to be both in roughly equal proportions, which means your network gets polluted by a lot of low-quality users who are just trying to do their part. Sia has the consumers pay the producers, and has a marketplace mechanism that selects the highest performing producers to perform the jobs and receive the revenue. This allows the network as a whole to be much, much more efficient.
  - theamk 6y ago
    Wait, consumers pay the producers? As in, “you need to pay to download files”?
    - anaganisk 6y ago
      Doesn't not paying make us end up rebuilding the internet again with ads and tracking? Free is not always an option.
      - theamk 6y ago
        Doesn't paying encourage useless sites highly optimized to grab user attention, low quality content, shock sites, cloned sites, site bloat (if payment is per KB), and so on? That said, I think there's definitely a space for paid internet. I was just surprised because siasky homepage says "Build a Free Internet" - perhaps it should say "Build a pay per view internet" instead?
        Taek 6y ago
        Free as in freedom, not free as in beer. I would argue that the current model is more friendly to attention grabbing content (clickbait, etc) because advertising has the limitation that all views are worth the same value. In a pay-as-you-go Internet, users can whitelist high quality content sources as being okay to charge 10x or 100x what you'd typical accept to view a webpage. This would incentive content creators to build a brand and reputation that makes users comfortable putting them on the 'high quality' list, so that their content can see a massive revenue multiple relative to the number of eyeballs.

pfraze 6y ago

We decided to take a year to go heads down and rework a lot of stuff, so we did get publicly stagnant but we're fulltime on the next version and should release a public beta soon. I tweet a lot about our progress if you can sort through my sillier tweets (@pfrazee). Past year has had a lot of improvements - Protocol moved to a hole-punching DHT for peer discovery (hyperswarm). - Protocol now scales # of writes and # of files much better. We were able to put a Wikipedia export, which is millions of files in 2 flat dirs, into a single "drive" and get good read performance. This performance bump came from an indexing structure that's built into every log entry (hypertrie). - Protocol now supports "mounting" which is a way to basically symlink drives onto each other. Good composition tool, esp useful for mounting deps in a /vendor directory. - Browser now has a builtin editor that splits the screen for live editing. Feels similar to hackmd. - Browser added a bash-like terminal for working with the protocol's filespace. It's glued to the current page so you can drive around the web using `cd`. - Browser added a basic identity system. Every user has a profile site that's created automatically and maintains an address book of other users. - We built out application tooling a fair amount. It's fairly easy to build multi-user apps now, where previously it was a bit of rocket surgery. Some of the year was spent prototyping ideas and throwing them away as well. A bit inefficient, but helped us learn.

grng3r 6y ago
I've been reading about dat and doing some basic projects for about a year in my spare time, stagnating a bit lately as am finishing my degree. As I remember there was some talk about Python and Rust implementations at the time, is that still under development as well? I am not as enthusiastic with Js but really like the protocol and the idea behind it.... Thanks
- pfraze 6y ago
  Yeah I do know there's a Rust impl that people are working on
koeng 6y ago
I’ve been thinking of deploying dat for Nanopore DNA sequencing, but recall something about dat not being great at handling that kind of data. Does the newest version of dat handle large files well (10gb)? Does it handle tons of files nested in a few directories well? Are there any issues I should know about there? What is the command line support like for multi-writer? Do you have any metrics for how much Dat is currently being used? Thanks!

anchpop 6y ago

You say you like the idea, do you have a problem with the implementation?

__ka 6y ago

The Cliqz browser implements the Dat protocol. Here's a writeup on the implementation: https://0x65.dev/blog/2020-03-02/implementing-the-dat-protocol-in-cliqz.html https://0x65.dev/blog/2020-03-02/implementing-the-dat-protoc... Disclosure: I work at Cliqz.

rakoo 6y ago
Amazing ! Dat is probably one of the most interesting technologies out there, it's good to see more people step in, especially if it's a growing company. Does that mean that Cliqz will start indexing dat sites ? The way I see it, the biggest issue of decentralized anything is always discoverability. If Cliqz is interested in it, it could have a leverage in making dat more widespread. Dat even helps you with the indexing, because it's the sites themselves that tell you when they are updated and when they are updated, it's like an indexer's dream come true. And seeing how dat operates, Cliqz could even be a peer in the swarm of the sites it indexes to give them more chance to be reached (at the cost of Cliqz having a view on who visits what, which seems contrary to your DNA if I understand correctly)