7 ms·

I really like the idea but reading this in the FAQ disappoints me: "Gemini has no support for caching, compression, or resumption of interrupted downloads" Bac

I really like the idea but reading this in the FAQ disappoints me: "Gemini has no support for caching, compression, or resumption of interrupted downloads" Back in the day in the time of BBS with Fidonet or Usenet, I really liked the offline first mentality. With zmodem you had compressed and resumable downloads, with Echomail and Fidonet and POP3/Usenet you had offline first forums. But the same FAQ says Gemini is also for those: "Interested in low-power computing and/or low-speed networks, either by choice or necessity" This seems to be contradictory. Have compression, caching and resumable downloads omitted by default for the sake of simplicity or can it still be added to the protocol? What's the rationale? A more formal seperation of simplicity of presentation and use within slow and frequently disconnected networks? That if you for example want zstd compression you can use a VPN? Or that if you want resumable downloads or caching you can use a local proxy? With the right proxy or browser one could emulate Gemini experience with the regular www, similar to using the old Opera Mini or force Reader View, block all images, javascript and ads on all sites. Wouldn't that be 'better'?

zozbot234 5y ago
> with Echomail and Fidonet and POP3/Usenet you had offline first forums. One of the side effects of ActivityPub is to standardize a "post new content" endpoint that might be used by offline-first clients with intermittent connectivity. This is in addition to getting content via ActivityStreams format, which can be implemented very easily, even by a cheap, statically-hosted site with no reliance on JS webtech.
- mananaysiempre 5y ago
  That’s the other direction, isn’t it? The point of UUCP, NNTP, and Echonet (and SMTP, for that matter) is message distribution between servers with intermittent connectivity by batching and relaying messages. Can I host an ActivityPub node that both publishes and receives (and, if we’re talking Echomail, relays) stuff if I only have an Internet connection for an hour per day? Can the whole federation function like that? (An ActivityPub-NNTP interconnect sounds like something that public-inbox really should support, now that I think about it.)
  - zozbot234 5y ago
    AIUI, ActivityPub allows for both POSTing content and GETting it in either direction, that is, a node can be acting as an "inbox", an "outbox" or both.
- pmlnr 5y ago
  Activity*: let's overcomplicate xml by making it json!
AnIdiotOnTheNet 5y ago
I'm not a part of the Gemini community, but if I had to guess: * caching is like alcohol: it is both the cause of and solution to all of life's problems * nothing Gemini serves should be so large as to need resumable downloads or compression In other words they seem like deliberate choices to prevent web-app-ification of Gemini.
- api 5y ago
  Cryptographic content-addressability (with inherent data deduplication) is superior to simple caching in every single way: efficiency, facilitation of distributed storage, and security. A link should point to a hash. Now you can go get that hash from the same host you used to get the link or any other host that stores it anywhere in the world. That hash could unpack to data or to a list of other hashes. Go get those. Where you get them doesn't matter one bit. You can verify the correctness of the data trivially. This is how BitTorrent works but it could work equally well on top of a simpler protocol.
  - selfhoster11 5y ago
    Would IPFS about cover this use case?
    - api 5y ago
      IPFS uses this, but it's more complex than necessary and has a ton of added features that are not core to the principle. (Not saying they're all bad... just that the core principle doesn't require thousands of lines of code to implement.) Here's the bottom line: The cryptographic content addressing I described sounds complex but is actually super simple and can be implemented in a page or two of code that will almost never need to change because it is "correct." It could be added to the gemini protocol in an afternoon. Caching sounds simple but is horrifically complex and an endless source of bugs and failure modes.
      - selfhoster11 5y ago
        > It could be added to the gemini protocol in an afternoon. I'd actually like this to be a separate protocol that's co-hosted with Gemini. It would make Gemini simpler, and it would enable other protocols to reuse content addressable storage.
DrBazza 5y ago
I'm not sure about how 'low' low power computing is, but consider this: a jpeg is compressed, and on a 1990 Acorn Archimedes (ARM, 8Mhz 32bit, and the fastest home computer at the time), you could watch a jpeg image being rendered in real time. Uncompressed images, unsurprisingly, rendered instantly.
- foobar33333 5y ago
  Gemini has no media support so its only text. I'm not sure what computer people are using which struggles to uncompress text.
  - fsiefken 5y ago
    Yes, but suppose I'm on a 200 bytes per second long distance radio link. One average UTF8 encoded page would perhaps take 10 seconds to load. With zstd or brotli compression it would take half the time. For ultra low power computing (for solar power) with zstd or brotli decompression support in a gemini client it would for example be doable on a raspberry pico running fuzix driving an e-ink display with keyboard. https://liliputing.com/2021/05/picomputer-kit-turns-a-raspberry-pi-pico-into-a-pocket-sized-computer-with-screen-and-keyboard.html https://liliputing.com/2021/05/picomputer-kit-turns-a-raspbe... https://learn.adafruit.com/adafruit-qt-py-2040 https://learn.adafruit.com/adafruit-qt-py-2040 https://www.tomshardware.com/reviews/adafruit-qt-py-rp2040-review https://www.tomshardware.com/reviews/adafruit-qt-py-rp2040-r...
    - efitz 5y ago
      This is a legit scenario. Many sailors use SSB [1] for data access and have ultra low bandwidth. [1] https://www.practical-sailor.com/marine-electronics/a-second-look-at-marine-ssb https://www.practical-sailor.com/marine-electronics/a-second...
  - selfhoster11 5y ago
    Some clients can be configured to load images inline. And if they don't, it's not a big problem to click on an image link for illustration.
mananaysiempre 5y ago
... Or simply a text-mode browser, yes. To be fair, full HTTP is ludicrously complicated on the server and almost nobody actually does or uses it. I don’t know of a single web app framework or library that properly and conveniently does all of content encodings, transfer encodings, content negotiation, caching, ranges, ETags, and so on; what’s worse, if you’re doing anything dynamic much of this is application-dependent and so I don’t really know how it should be done. The parsing (on both sides) also kind of sucks, if not as much as for 822. Gopher (on which Gemini is based) is almost laughably simple by comparison, you can do it manually in shell script and not feel like a dirty hack. I do not understand why the Gemini people don’t do NNTP, though. The relative inaccessibility of Usenet in the modern world? Still, you don’t have to federate with it if you don’t want to, and the tech is quite nice, if somewhat difficult due to the inherent complexity of the problem.
- pmlnr 5y ago
  Lack of guides for setting up a private nntp, I'd say. If you have some resources, please share them.
- selfhoster11 5y ago
  Gemini is symbiotic with other "old web" protocols, so Usenet fits. If one is running Gemini, then their gemlog will likely list an email address (SMTP) and host large files over FTP, HTTP or IPFS.
- projektfu 5y ago
  I’m don’t remember Usenet being federated so much as based on replication. It was really organized around large centers with many users at each center, and the need to store and forward everything faithfully.
  - mananaysiempre 5y ago
    I’m ready to defer to your expertise (or any expertise, really), but aren’t we just disagreeing over terminology? By a “federated system” I mean that there is a relatively small number of independently operated first-class entities (servers, nodes) that are more permanently connected and host a large number of second-class entities (users, points) that are more intermittently connected and generally correspond to individuals. Internet mail, IRC, Jabber and Matrix are federated; Slack (centralized) and BitTorrent with DHT (peer-to-peer) are not. The Internet itself is complex enough that it probably doesn’t usefully fit into this classification. In this sense Usenet and FidoNet are both federated store-and-forward mail systems. Or am I wrong?
1vuio0pswjnm7 5y ago
"With the right proxy or browser one could emulate Gemini experience with the regular www, similar to using the old Opera Mini or force Reader View, block all images, javascript and ads on all sites. Wouldn't that be 'better'?" One of the neat things about Gemini is how quickly and easily people started writing clients for it. Perhaps this illustrates how we can influence client design, e.g., limit complexity (and perhaps decrease insecurity), through protocol design. I use a text-only browser to read HTML. I often make HTTP requests from the commmand-line using a variety of different TCP clients in order to utilise a feature of HTTP/1.1 called pipelining. I have been using the web this way for decades. Needless to say, "new" protocols like HTTP/2 and HTTP/3 come across as being 100% commercial in their vision of what the web is; they are designed to acommodate a web full of advertising. Their sponsors justify them based on problems that one might encounter when trying to use the web for commerce. However, they fail to account for non-commercial uses. For example, there's no "HOL blocking" problem when all the resoources are coming from the same server and the site operator is not trying to "build complex, interactive [graphical] web pages." There is an underlying bias with these protocols for enabling a commercial, advertising-filled web. As one would expect considering who is paying the salaries of their implementors. Then we have something like QUIC whose sponsors similarly make justifcation arguments that assume every site's goals are "complex [graphical] web pages", likely to be supported through advertising. I have read articles arguing in favour of QUIC that suggest TCP never achieves its true potential for speed because we are only ever making large numbers of small TCP "connections". That usage pattern is indeed something we see with "complex, interactive [graphical] webpages".FN1 The kind that serve advertising. However, with HTTP/1.1 pipelining, as I use it, I have different goals. I make a single, longer-lived TCP connection and make 10s to 100s of HTTP requests to the same server. I am not using the web as an interactive, graphical interface. I am doing [text] records retrieval. I am using the web as an information source. Again, what I am seeing with these "next gen" protocols are an underlying bias and assumptions in favour of a commercial web filled with ads. As such, I agree the "limitations" of Gemini which make it cool to some of us, i.e., it is inherently not advertising-friendly, could be achieved using HTTP either by 1. users choice of client, i.e., stop using graphical browsers that run Javascript, written by companies that rely on "surveillance capitalism" or 2. a new standard that websites could adopt whereby they (also) served content in non-advertising-friendly way. #2 is outside the control of the user. It is squarely under the control of website operators and those with the power ($$) to influence website operators. Indeed, it can also be brought about through use of a proxy. I have also become a die-hard localhost-bound proxy user. It is seemingly limitless what a one can do with a good proxy, however it can quickly lead to some rather complex configurations and much of the results can alternatively be achieved by #1, simply using the "right" client. I use both #1 and #2 together. 1. Not sure that Gemini supports pipelining. If not, how does it allow for efficient, bulk text retrieval. Does it encourage short, successive TCP connections. The mistake that the original lobster.rs "Gemini is useless" commenter made is that he assumed Gemini is meant to "replace" the web. Yet no one, certainly not the creator of the protocol, ever claimed anything of the sort. In fact, Gemini documentation explicitly disclaims this as a goal. The commenter has constructed a strawman. He also assumed the only worthwhile use for an internet is a web, and the only worthwhile use for a web is to engage in commerce. To me, that looks like myopia. The internet has historically been multi-use, as the parent comment describes. The "www" is but one use. Alas, it has been co-opted by those seeking to use it as an advertising vector.
canadianfella 5y ago
> Have compression, caching and resumable downloads omitted by default for the sake of simplicity or can it still be added to the protocol? You’re missing a word, which made this hard to read. “Have compression, caching and resumable downloads been omitted by default for the sake of simplicity or can it still be added to the protocol?”
bawolff 5y ago
I guess if you dont have css or javascript/other subresources that stay constant, caching is less important.

zozbot234 5y ago

> with Echomail and Fidonet and POP3/Usenet you had offline first forums. One of the side effects of ActivityPub is to standardize a "post new content" endpoint that might be used by offline-first clients with intermittent connectivity. This is in addition to getting content via ActivityStreams format, which can be implemented very easily, even by a cheap, statically-hosted site with no reliance on JS webtech.

mananaysiempre 5y ago
That’s the other direction, isn’t it? The point of UUCP, NNTP, and Echonet (and SMTP, for that matter) is message distribution between servers with intermittent connectivity by batching and relaying messages. Can I host an ActivityPub node that both publishes and receives (and, if we’re talking Echomail, relays) stuff if I only have an Internet connection for an hour per day? Can the whole federation function like that? (An ActivityPub-NNTP interconnect sounds like something that public-inbox really should support, now that I think about it.)
- zozbot234 5y ago
  AIUI, ActivityPub allows for both POSTing content and GETting it in either direction, that is, a node can be acting as an "inbox", an "outbox" or both.
pmlnr 5y ago
Activity*: let's overcomplicate xml by making it json!

AnIdiotOnTheNet 5y ago

I'm not a part of the Gemini community, but if I had to guess: * caching is like alcohol: it is both the cause of and solution to all of life's problems * nothing Gemini serves should be so large as to need resumable downloads or compression In other words they seem like deliberate choices to prevent web-app-ification of Gemini.

api 5y ago
Cryptographic content-addressability (with inherent data deduplication) is superior to simple caching in every single way: efficiency, facilitation of distributed storage, and security. A link should point to a hash. Now you can go get that hash from the same host you used to get the link or any other host that stores it anywhere in the world. That hash could unpack to data or to a list of other hashes. Go get those. Where you get them doesn't matter one bit. You can verify the correctness of the data trivially. This is how BitTorrent works but it could work equally well on top of a simpler protocol.
- selfhoster11 5y ago
  Would IPFS about cover this use case?
  - api 5y ago
    IPFS uses this, but it's more complex than necessary and has a ton of added features that are not core to the principle. (Not saying they're all bad... just that the core principle doesn't require thousands of lines of code to implement.) Here's the bottom line: The cryptographic content addressing I described sounds complex but is actually super simple and can be implemented in a page or two of code that will almost never need to change because it is "correct." It could be added to the gemini protocol in an afternoon. Caching sounds simple but is horrifically complex and an endless source of bugs and failure modes.
    - selfhoster11 5y ago
      > It could be added to the gemini protocol in an afternoon. I'd actually like this to be a separate protocol that's co-hosted with Gemini. It would make Gemini simpler, and it would enable other protocols to reuse content addressable storage.

DrBazza 5y ago

I'm not sure about how 'low' low power computing is, but consider this: a jpeg is compressed, and on a 1990 Acorn Archimedes (ARM, 8Mhz 32bit, and the fastest home computer at the time), you could watch a jpeg image being rendered in real time. Uncompressed images, unsurprisingly, rendered instantly.

foobar33333 5y ago
Gemini has no media support so its only text. I'm not sure what computer people are using which struggles to uncompress text.
- fsiefken 5y ago
  Yes, but suppose I'm on a 200 bytes per second long distance radio link. One average UTF8 encoded page would perhaps take 10 seconds to load. With zstd or brotli compression it would take half the time. For ultra low power computing (for solar power) with zstd or brotli decompression support in a gemini client it would for example be doable on a raspberry pico running fuzix driving an e-ink display with keyboard. https://liliputing.com/2021/05/picomputer-kit-turns-a-raspberry-pi-pico-into-a-pocket-sized-computer-with-screen-and-keyboard.html https://liliputing.com/2021/05/picomputer-kit-turns-a-raspbe... https://learn.adafruit.com/adafruit-qt-py-2040 https://learn.adafruit.com/adafruit-qt-py-2040 https://www.tomshardware.com/reviews/adafruit-qt-py-rp2040-review https://www.tomshardware.com/reviews/adafruit-qt-py-rp2040-r...
  - efitz 5y ago
    This is a legit scenario. Many sailors use SSB [1] for data access and have ultra low bandwidth. [1] https://www.practical-sailor.com/marine-electronics/a-second-look-at-marine-ssb https://www.practical-sailor.com/marine-electronics/a-second...
- selfhoster11 5y ago
  Some clients can be configured to load images inline. And if they don't, it's not a big problem to click on an image link for illustration.

mananaysiempre 5y ago

... Or simply a text-mode browser, yes. To be fair, full HTTP is ludicrously complicated on the server and almost nobody actually does or uses it. I don’t know of a single web app framework or library that properly and conveniently does all of content encodings, transfer encodings, content negotiation, caching, ranges, ETags, and so on; what’s worse, if you’re doing anything dynamic much of this is application-dependent and so I don’t really know how it should be done. The parsing (on both sides) also kind of sucks, if not as much as for 822. Gopher (on which Gemini is based) is almost laughably simple by comparison, you can do it manually in shell script and not feel like a dirty hack. I do not understand why the Gemini people don’t do NNTP, though. The relative inaccessibility of Usenet in the modern world? Still, you don’t have to federate with it if you don’t want to, and the tech is quite nice, if somewhat difficult due to the inherent complexity of the problem.

pmlnr 5y ago
Lack of guides for setting up a private nntp, I'd say. If you have some resources, please share them.
selfhoster11 5y ago
Gemini is symbiotic with other "old web" protocols, so Usenet fits. If one is running Gemini, then their gemlog will likely list an email address (SMTP) and host large files over FTP, HTTP or IPFS.
projektfu 5y ago
I’m don’t remember Usenet being federated so much as based on replication. It was really organized around large centers with many users at each center, and the need to store and forward everything faithfully.
- mananaysiempre 5y ago
  I’m ready to defer to your expertise (or any expertise, really), but aren’t we just disagreeing over terminology? By a “federated system” I mean that there is a relatively small number of independently operated first-class entities (servers, nodes) that are more permanently connected and host a large number of second-class entities (users, points) that are more intermittently connected and generally correspond to individuals. Internet mail, IRC, Jabber and Matrix are federated; Slack (centralized) and BitTorrent with DHT (peer-to-peer) are not. The Internet itself is complex enough that it probably doesn’t usefully fit into this classification. In this sense Usenet and FidoNet are both federated store-and-forward mail systems. Or am I wrong?

1vuio0pswjnm7 5y ago

"With the right proxy or browser one could emulate Gemini experience with the regular www, similar to using the old Opera Mini or force Reader View, block all images, javascript and ads on all sites. Wouldn't that be 'better'?" One of the neat things about Gemini is how quickly and easily people started writing clients for it. Perhaps this illustrates how we can influence client design, e.g., limit complexity (and perhaps decrease insecurity), through protocol design. I use a text-only browser to read HTML. I often make HTTP requests from the commmand-line using a variety of different TCP clients in order to utilise a feature of HTTP/1.1 called pipelining. I have been using the web this way for decades. Needless to say, "new" protocols like HTTP/2 and HTTP/3 come across as being 100% commercial in their vision of what the web is; they are designed to acommodate a web full of advertising. Their sponsors justify them based on problems that one might encounter when trying to use the web for commerce. However, they fail to account for non-commercial uses. For example, there's no "HOL blocking" problem when all the resoources are coming from the same server and the site operator is not trying to "build complex, interactive [graphical] web pages." There is an underlying bias with these protocols for enabling a commercial, advertising-filled web. As one would expect considering who is paying the salaries of their implementors. Then we have something like QUIC whose sponsors similarly make justifcation arguments that assume every site's goals are "complex [graphical] web pages", likely to be supported through advertising. I have read articles arguing in favour of QUIC that suggest TCP never achieves its true potential for speed because we are only ever making large numbers of small TCP "connections". That usage pattern is indeed something we see with "complex, interactive [graphical] webpages".FN1 The kind that serve advertising. However, with HTTP/1.1 pipelining, as I use it, I have different goals. I make a single, longer-lived TCP connection and make 10s to 100s of HTTP requests to the same server. I am not using the web as an interactive, graphical interface. I am doing [text] records retrieval. I am using the web as an information source. Again, what I am seeing with these "next gen" protocols are an underlying bias and assumptions in favour of a commercial web filled with ads. As such, I agree the "limitations" of Gemini which make it cool to some of us, i.e., it is inherently not advertising-friendly, could be achieved using HTTP either by 1. users choice of client, i.e., stop using graphical browsers that run Javascript, written by companies that rely on "surveillance capitalism" or 2. a new standard that websites could adopt whereby they (also) served content in non-advertising-friendly way. #2 is outside the control of the user. It is squarely under the control of website operators and those with the power ($$) to influence website operators. Indeed, it can also be brought about through use of a proxy. I have also become a die-hard localhost-bound proxy user. It is seemingly limitless what a one can do with a good proxy, however it can quickly lead to some rather complex configurations and much of the results can alternatively be achieved by #1, simply using the "right" client. I use both #1 and #2 together. 1. Not sure that Gemini supports pipelining. If not, how does it allow for efficient, bulk text retrieval. Does it encourage short, successive TCP connections. The mistake that the original lobster.rs "Gemini is useless" commenter made is that he assumed Gemini is meant to "replace" the web. Yet no one, certainly not the creator of the protocol, ever claimed anything of the sort. In fact, Gemini documentation explicitly disclaims this as a goal. The commenter has constructed a strawman. He also assumed the only worthwhile use for an internet is a web, and the only worthwhile use for a web is to engage in commerce. To me, that looks like myopia. The internet has historically been multi-use, as the parent comment describes. The "www" is but one use. Alas, it has been co-opted by those seeking to use it as an advertising vector.

canadianfella 5y ago

> Have compression, caching and resumable downloads omitted by default for the sake of simplicity or can it still be added to the protocol? You’re missing a word, which made this hard to read. “Have compression, caching and resumable downloads been omitted by default for the sake of simplicity or can it still be added to the protocol?”

bawolff 5y ago

I guess if you dont have css or javascript/other subresources that stay constant, caching is less important.