39 ms·

Engineering for Slow Internet

• FerretFred 2y ago
  A fascinating article and I need to revisit this when I have more time. So, all I'd say now is that there's way too much emphasis on GUI. Also, check out some of the web sites on https://1mb.club https://1mb.club - it's amazing what can be achieved in less than 1Mb of HTML ...
  • nicbou 2y ago
    1 MB is comically large. 150 kb is a much more interesting target.
    • FerretFred 2y ago
      Haha .. I just rechecked mine and it's coming in as just under 20Kb - must be the pie-chart graphic bloating it out
• MagicMoonlight 2y ago
  As I was reading this I realised that everything here had already been solved - by torrents. Everything is split into chunks. Downloads and uploads happen whenever connections can happen. If someone local has a copy, they can seed it to you without you needing an external connection. You can have a cache server that downloads important stuff for everyone.
• warpech 2y ago
  I assume it was considered, but I don’t see it mentioned: Would it be a terrible idea to use a cloud computer and Remote Desktop/VNC to it? Your slow internet only needs to stream the compressed pixels to your thin client.
• hi-v-rocknroll 2y ago
  The presumption that "every user" has 20 ms latency, 200 Mbps bandwidth, and unlimited data limits is fundamentally inconsiderate to other cases such as great distances, local congestion, or where accessibility issues exist. This problem will return with a vengeance once humans occupy the Moon and Mars. PSA: Please optimize your website, web apps for caching and efficiency, and offer slow/graceful fallback versions instead of 8K fullscreen video as your homepage for all users.
• agarwa90 2y ago
  very well written post!
• SergeAx 2y ago
  Telegram messenger is fantastic. It works over GPRS (AKA 2.5G) connection. I love sailing, and the moment we see nearby island and get data connection - Telegram immediately starts working. WhatsApp tries, but actually works only over 3G.
• tirey 2y ago
  https://github.com/tireymorris/hyperwave https://github.com/tireymorris/hyperwave hyperwave is great for slow connections - in my testing, even a 2G throttled connection is still usable with load times in the 5s range.
• dosourcenotcode 2y ago
  Definitely agree with the article that engineers should be more aware of scenarios where those interacting with the systems they build have slow internet. Another thing I think people should think about is scenarios with intermittent connectivity where there is literally no internet for periods ranging from minutes to days. Sadly in both these regards I believe we're utterly screwed. Even the Offline First and Local First movements who you'd think would handle these issues in at least a semi-intelligent manner don't actually practice what they preach. Look at Automerge or frankly the vast majority of the other projects that came out of those movements. Logically you'd think they have offline documentation that allows people to study them in a Local First fashion. Sadly that's not the case. The hypocrisy is truly a marvel to behold. You'd think that if they can get hard stuff like CRDTs right they'd get simple stuff right like actually providing offline / local first docs in a trivial to obtain way. Again sadly not. The following two links are yet another example of a similar kind of hypocrisy: https://twitter.com/mitchellh/status/1781840288300097896 https://twitter.com/mitchellh/status/1781840288300097896 https://github.com/hashicorp/vagrant/issues/1052#issuecomment-16025549 https://github.com/hashicorp/vagrant/issues/1052#issuecommen... Again at this point the jokes are frankly writing themselves. Like bro make it possible for people to follow your advice. Also if you directly state or indirectly insinuate that your tool is ANY/ALL OF Local First, or Open Source, or Free As In Freedom you better have offline docs. If you don't have offline docs your users and collaborators don't have Freedom 1. If you can't exercise Freedom 1 you are severely hampered in your ability to exercise Freedoms 0, 2, or 3 for any nontrivial FOSS system. The problem has gotten so bad the I started the Freedom Respecting Technology movement which I'm gonna plug here: https://makesourcenotcode.github.io/freedom_respecting_technology.html https://makesourcenotcode.github.io/freedom_respecting_techn...
• prng2021 2y ago
  I’m curious if using these sites through a remote desktop session would be a lot better experience. Keep a computer running at home. Connect to it from Antarctica and do what you gotta do. I don’t know how well VNC/Windows Remote Desktop do with such poor network conditions though.
• SlowBro 2y ago
  I've been looking into how to survive low bandwidth/high latency/frequent disconnecting internet, because we may need to save money at home and temporarily switch to an unlimited 2G speed (64kbit) cell data plan as our primary internet for a few months. (Red Pocket's $7/mo. 1GB GSMA plan paid annually.) Not only is Red Pocket notoriously bad at limiting both bandwidth and are horrible with latency, we also live far from the closest cell tower, so our reception is also poor and frequently disconnects. A one-two-three punch for a bad internet experience. This is probably about as close to experiencing an Antarctic data satellite connection within the continental US as is possible, and would be even worse than a 56k modem; While the bandwidth is roughly the same, Red Pocket users often report latencies in excess of 1000+ms, whereas 56k modem latency was typically under 200ms. The frequent disconnects (because we're far from the tower) would simulate Antarctic conditions, as well. So how do internet-connected people living in 2024 survive such an atrocity? First, we would leverage SSH and CLI tools as much as possible. Brow.sh and Carbonyl browser for web and Mutt for email. Mosh.sh to keep the SSH connection alive and reconnect automatically on reconnect--even if your IP changes. (Mosh is incredible.) Some terminals have a zoom feature (ctrl++ or - in kitty) which may help if some web browser item is just too difficult to see in its blocky resolution. See also the Carbonyl --bitmap and --zoom flags: https://github.com/fathyb/carbonyl/releases/tag/v0.0.3 https://github.com/fathyb/carbonyl/releases/tag/v0.0.3 The youtube-dl command can fetch tiny versions of a video on a server (even audio-only if we don't care about the video portion such as with long-form interviews), then transcode them on the server to the smallest-tolerable bitrate. rsync them home overnight and watch them the next day. rsync can retry on failure: rsync --partial --progress --compress-choice=STR, --zc=zlib --compress-level=9 --rsh=ssh user@host:remote_file local_file Edit: On second thought, rsync doesn't perform well in high-latency environments due to TCP acks, but something like uftp which uses UDP should do better, and offers resume. https://stackoverflow.com/questions/38014501/what-is-the-fastest-way-to-transfer-files-over-high-latency-and-high-bandwidth-l https://stackoverflow.com/questions/38014501/what-is-the-fas... https://uftp-multicast.sourceforge.net/ https://uftp-multicast.sourceforge.net/ https://sourceforge.net/p/uftp-multicast/discussion/general/thread/5c899011/ https://sourceforge.net/p/uftp-multicast/discussion/general/... Needs a remote Linux server, but ServerHunter has servers adequate for the task for less than $3/mo. Focus on RAM; Most any plan has sufficient hard drive space and speed, sufficient bandwidth and CPU for what you're doing. You're not serving large content to many demanding customers, and in my personal testing I have found that even 1GB RAM should be enough--but obviously more is better. And many companies only charge for data egress--which should be smaller than ingress. Data ingress to the server is usually free, and when you're downloading large amounts of data to the server, then compressing them, egress is much smaller. Choose companies that have a domain age of at least 1 year old; I've had a fly-by-night company take my money and run. (PayPal reimbursed me.) To download large files, try first compressing them; Although, depending on the original file, compression can either give you no advantage or even _increase_ the file size. Try different compression methods; bzip2, zpaq, xz, lzma, etc. ls -l the before and after to see if it helped any. Also, we'd rely on phones apps for things like Gmail/Facebook/Messenger/WhatsApp/Telegram/Twitter/Discord/etc. This saves always redownloading JavaScript for each visit. While there are Windows/Linux desktop clients for these, they usually are only thin layer over a browser, which means they still download JavaScript and large images on every use. But a phone app has everything it needs all the time--until the next app update, of course. Can temporarily disable app updates, but I wouldn't do that for long. The good part about app updates is they can tolerate slow links and frequent disconnects. And we often take our phones with us to high-bandwidth locations, so we can do app updates there and disable them over cell data. The article said on some days Antarctica has better bandwidth availability, so you could halt updates until conditions improve. Same with Windows or Linux OS auto-updates; Disable them until bandwidth conditions improve. (But you're probably already doing that.) Edit: After reading the article again, yes they definitely are doing that, and are still having problems. Bummer. Installing Bluestacks is an option, for the above reason. Using phone apps on the desktop that don't need to download JavaScript on every use could bridge the gap somewhat. Disable its auto-updates as well. Gmail can be set up for IMAP or POP3. Try both; POP should give a better experience, but by default it removes messages on the server, so look for that setting to disable it if you want to keep them there. I believe you can disable downloading attachments and images automatically, which probably works better for IMAP? Not sure if POP can handle this. Doing it this way means you don't need to load the web interface any time you check mail, and you'll only be reading the text content of emails. It should therefore work through Mutt as well. Then there's browser tricks for local rendering: Disable graphic images and disable media autoplay. uBlock Origin to reduce overall download size. Turn off JavaScript until truly needed. Pain in the butt, but it works. In Windows, set your network connection to metered. This should help. (And back in the US, we can set our phones to low bandwidth mode--but that would only help users on cell networks; I'm sure the south pole lacks cell reception.) You're not going to be gaming with large downloads and patches, nor will you be competing with low-latency gamers around the globe. But not all games need these. Play against the computer instead of other people. Play only pre-downloaded or DVD-based games. Or go back to the 90s and set up a LAN party and play against one another in the same room. We love Unreal Tournament 1999 GOTY; Plays blisteringly fast on any computer, is cheap to buy on DVD or Steam, has no in-game paid-for upgrades, has oodles and oodles of maps for download, and gameplay still feels fresh even 25 years later. Card games and offline phone games are king in low bandwidth environments. You're probably not going to make many audio or video calls, so learn to send mp4s or mp3s to others back and forth by email. Use the lowest-tolerable bitrate. A 6Kbps OPUS/32Kbps AV1 video is about 5KB/sec and is clear enough to follow along, and would certainly be acceptable for a video message. A 0.5Kbps OPUS-encrypted audio file is surprisingly understandable, if not very enjoyable. But to get the message across, it works. A 6Kbps OPUS audio sounds like telephone quality to my ears. https://heavydeck.net/post/64k-is-enough-for-video/ https://heavydeck.net/post/64k-is-enough-for-video/ Remember that 24-64kbit audio streams are still a thing. Search low bandwidth streaming radio. Sort by bitrate: https://directory.shoutcast.com/ https://directory.shoutcast.com/ https://directory.shoutcast.com/?q=299_Talk_297 https://directory.shoutcast.com/?q=299_Talk_297 I don't know if podcast apps have a low-bandwidth option, but it's worth investigating. Apparently the Player FM Pro app compresses pods on their servers before sending. There are a few text-only news sites: https://blog.wturrell.co.uk/text-only-news-websites/ https://blog.wturrell.co.uk/text-only-news-websites/ RSS feeds can fetch news over a slow link, and they're still a thing: https://rss.feedspot.com/world_news_rss_feeds/ https://rss.feedspot.com/world_news_rss_feeds/ Finally, there's the possibility of running a remote desktop session: Something like RDP over SSH, VNC over SSH, NoMachine's NX, TeamViewer, etc. Can give clearer detail than brow.sh and can sometimes be faster than browsing locally. I tested it by intentionally limiting my desktop's speed to 5KB/sec and while it wasn't fun, it got the job done. Took about ten seconds to paint a page on a 1GB VM, but that can often be faster than rendering a JavaScript-heavy page locally. Seems to be a good compromise for certain situations when the rest of the options fail you. Typing is too laggy (5 seconds before characters appeared) but one way around that is to type into a file via SSH then cat the file on the server, paste that into the web form. Or write your notes to a file on your local system and copy+paste that directly in. I've found that even a server with 1GB RAM is sufficient for even this task, as long as you're only looking at one web page at a time. No multiple tabs. I ran XFCE4 on Ubuntu 22.04 and Google's official Chrome package. Resized the connection to 1024x768, which I consider to be the minimum for useful browsing. Changed the quality settings to lowest, color depth to lowest, etc. Disabled audio and printing. To read large pages offline, print to PDF on the server inside RDP/VNC and download that with rsync. The combination of all of these should help us get through a season of needing to save money, and it may very well save the day in Antarctica, as well.
• arnaudsm 2y ago
  Big tech probably loses millions of customers from developing countries because of their bloated frontend engineering. Not everyone has an M3 with gigabit fiber. My favorite setup when working remotely with slow internet is the following : - Build tasks & large data operations run on my home server via SSH - Markdown notes, Notion being unusable - 100% CLI apps, which are usually the lightest and most efficient
• fuzzfactor 2y ago
  Give me a break. The purpose of broadband is so that nothing has any latency when you're not on dial-up. Except user-initiated mega downloads, and they occur at the maximum connection rate the hardware can handle. At bandwidth costs anybody can afford. Just like with dial-up, why settle for less instantaneous performance than the hardware can handle? Just to make it so lesser geeks can participate? The hallmark of insufficient web "engineering" has always been the completeness of the failure to pre-deploy on dial-up.
• dghlsakjg 2y ago
  At the beginning of learning to code in 2017 I was living in the Caribbean. I had to get Xcode command line tools installed, <1gb of download. Something trivial to get done in the US. It took me several days of hunting down an internet connection that was fast and stable enough to handle it since most internet in the country was cell based. The solution was a friendly computer repair shop with a decent hardwired connection that let me leave the computer for a few hours to download. In a place like that there are some sites that just don’t work sometimes because it is clear that the devs have never thought about what happens when a significant portion of your requests fail.
• fragmede 2y ago
  http://hn.zip http://hn.zip is one such web page where it downloads as much as it can to be usable on a flakey connection.
• hunterhod 2y ago
  Phoenix LiveView.
• 0xWTF 2y ago
  I still think engineering for slow internet is really important, and massively under appreciated by most software developers, but ... LEO systems (like Starlink, especially StarLink) essentially solve the core problems now. I did an Arctic transit (Alaska to Norway) in September and October of 2023, and we could make FaceTime video calls from the ship, way above the Arctic Circle, despite cloud cover, being quite far from land, and ice. This was at the same time OP was in Antartica. Whatever that constraint was, it's just contracting for the service and getting terminals to the sites. The polar coverage is relatively sparse, but still plenty, due to the extraordinarily low population. https://satellitemap.space/ https://satellitemap.space/
  • NelsonMinar 2y ago
    Starlink has its own networking issues thanks to a lot of latency jitter and 0.5% or more packet loss. See the discussion from last month: https://news.ycombinator.com/item?id=40384959 https://news.ycombinator.com/item?id=40384959 The biggest issue for Starlink at the poles is, as you say, very sparse coverage. Also I suspect Starlink has to usually relay polar packets between satellites, not just a simple bent pipe relaying to a ground station.
  • Aerbil313 2y ago
    LEO internet won't be practically usable for another decade for loads of us who are not living in a country the US DOD favors.
  • giantrobot 2y ago
    Slow Internet isn't just remote places, it also crops up in heavily populated urban areas. It's sad that you had better connectivity above the Arctic circle than the typical connectivity with hotel WiFi. Bad connectivity also happens with cellular connections all over the place.
  • el_benhameen 2y ago
    Not really the point of your post, but that sounds like a really cool trip. What were you doing up there?
  • 2y ago
• grishka 2y ago
  IMO author shouldn't have censored the app names. The world deserves to know.
  • Tepix 2y ago
    I agree! They call out Apple, Samsung and a few others, but not app vendors.
    • grishka 2y ago
      The messaging app with an updater is clearly Signal. The checkmark icons gave it away for me.
  • koito17 2y ago
    Querying an exact match of a few strings on Google shows me that Slack is the very first example given in the blog post. For additional confirmation, the "6-byte message" screenshot lists an xoxc token and rich_text object, both of which you will frequently encounter in the Slack API. To be honest, I was expecting it to be Jira at first since I was unaware of Slack's size. Searching for an exact match of "PRL_ERR_WEB_PORTAL_UNEXPECTED" gives away Parallels as the first example of a hard-coded HTTPS timeout. So on, so forth.
  • future10se 2y ago
    Those I recognize (from interface, branding, strings, etc.): * Slack -- https://brr.fyi/media/engineering-for-slow-internet/load-error-01.png https://brr.fyi/media/engineering-for-slow-internet/load-err... * Signal (main screen) -- https://brr.fyi/media/engineering-for-slow-internet/in-app-downloader-progress-01.png https://brr.fyi/media/engineering-for-slow-internet/in-app-d... * 1Password (config/about page) -- https://brr.fyi/media/engineering-for-slow-internet/in-app-downloader-02.png https://brr.fyi/media/engineering-for-slow-internet/in-app-d... * Zoom (updater screen on Mac) -- https://brr.fyi/media/engineering-for-slow-internet/in-app-downloader-03.png https://brr.fyi/media/engineering-for-slow-internet/in-app-d... * Parallels Desktop ("prl_client_app" is the binary name) -- https://brr.fyi/media/engineering-for-slow-internet/hardcoded-timeout-02.png https://brr.fyi/media/engineering-for-slow-internet/hardcode...
• rlv-dan 2y ago
  > Please keep in mind that I wrote the majority of this post ~7 months ago, so it’s likely that the IT landscape has shifted since then. Not sure if this is serious or intended as a joke. It made me giggle nonetheless. Which is kind of sad.
  • modeless 2y ago
    They're being obtuse. What "it's likely the IT landscape has shifted" actually means is "they got Starlink and their connection is fast now, and I know this for certain but I want to downplay it as much as possible because I'm trying to make a point".
    • sham1 2y ago
      Or they could be making a joke about how quickly trends shift in IT. It's like how people joke (or at least used to joke) that you'd get a dozen new JavaScript frameworks daily. Exaggeration for comedic effect, in other words.
  • kortilla 2y ago
    It’s serious presumably because Starlink coverage includes the poles now. 7 months ago was around the time they did a demo with the McMurdo base IIRC.
    • bizzyb 2y ago
      McMurdo has starlink. South Pole doesn't, but not due to technical reasons from starlink's side. From what I understand when they tested at Pole they noticed interference with some of the science experiments, its possible they will engineer around that at some point but for now starlink is a low priority compared to ensuring the science goes on. I forget the exact distance, but its something like 5 miles from pole that they ask groups traversing to turn off their starlink.
      • fragmede 2y ago
        that is to say, the starlink terminals radiating EM sufficient to mess with the sensitive sensors at the south pole, which is fascinating since they're supposed to have passed compliance testing that they're not doing too much of that. but the south pole has a different definition of too much, it seems. fascinating!
• danpalmer 2y ago
  Having a lot of experience commuting on underground public transport (intermittent, congested), and living/working in Australia (remote), I can safely say that most services are terrible for people without "ideal" network conditions. On the London Underground it's particularly noticeable that most apps are terrible at handling network that comes and goes every ~2 minutes (between stops), and which takes ~15s to connect to each AP as a train with 500 people on it all try to connect at the same time. In Australia you're just 200ms from everything most of the time. That might not seem like much, but it really highlights which apps trip up on the N+1 request problem. The only app that I am always impressed with is WhatsApp. It's always the first app to start working after a reconnect, the last to get any traffic through before a disconnect, and even with the latency, calls feel pretty fast.
  • pimeys 2y ago
    I guess in London you get wifi only on stops, it's the same in Berlin. In Helsinki the wifi connection is available inside the trains, and in the stations. So you never get a connection loss when moving. I never understood the decision in Berlin to do this, why not just provide internet inside the train... And yeah, most of the internet works very badly when you drop the network all the time...
    • pintxo 2y ago
      Berlin did not have mobile connections inside the tunnels until very recently (this year, I believe). This included the trains not being connected to any outside network. Thus wifi on the subway was useless to implement.
      • saagarjha 2y ago
        I was in Berlin earlier this month and the cellular connections underground were quite good now. So maybe this is less of a problem?
        • nicbou 2y ago
• initramfs 2y ago
  awesome. for the other 6 continents, check out: https://solar.lowtechmagazine.com/2015/10/how-to-build-a-low-tech-internet https://solar.lowtechmagazine.com/2015/10/how-to-build-a-low...
• oefrha 2y ago
  This is why I find it dreadful that evangelists here are heavily promoting live$whatever technology where every local state change requires at least one server roundtrip, or “browsers support esm now, bundling is a thing of the past!” etc. You don’t need to be at Antarctica to feel the latencies caused by the waterfall of roundtrips, or roundtrip on every click, as long as you’re a mere 200ms from the server, or in a heavily congested place.
  • deleted 2y ago
    [deleted]
• xobs 2y ago
  An example of a program that's atrocious about unreliable connectivity is `git` -- it has no way to resume downloads, and will abort all progress if it fails mid-transfer. The only way I've found to reliably check out a git repository over an unreliable link is to check it out somewhere reliable and `rsync` the .git directory over.
  • o11c 2y ago
    Usually `git clone --depth 1 URL` works, then you can incrementally deepen it. This does cause extra load on the servers, but if it's that big a problem for them, they can write the incremental patches themselves. (I suspect that the "dumb http" transport are also incremental if you squint hard enough at them, but I've never had reason to investigate that closely)
• snthpy 2y ago
  Have a look at https://offlinefirst.org/ https://offlinefirst.org/ and https://www.inkandswitch.com/local-first/ https://www.inkandswitch.com/local-first/ .
• 0xbadcafebee 2y ago
  > a lot of the end-user impact is caused by web and app engineering which fails to take slow/intermittent links into consideration. Technology today is developed by, and for, privileged people. Has been that way for a while. Ever since you had to upgrade your computer in order to read the news, there has been a slow, steady slog of increasing resource use and conspicuous consumption. I remember using the 9600 baud modem to get online and do the most basic network transactions. It felt blazing fast, because it was just some lines of text being sent. I remember the 2.5KBps modem, allowing me to stream pictures and text in a new World Wide Web. I remember the 5KBps modem making it possible to download an entire movie! (It took 4 days, and you had to find special software to multiplex and resume cancelled downloads, because a fax on the dialup line killed the connection) I remember movies growing to the size of CDROMs, and later DVDROMs, so those who could afford these newer devices could fit the newer movies, and those who couldn't afford them, didn't. I remember the insane jump from 5KBps to 1.5Mbps, when the future arrived. Spending days torrenting hundreds of songs to impress the cool kids at school, burning them CDs, movies, compiling whole libraries of media [hey, 15 year olds can't afford retail prices!]. I remember when my poor friends couldn't use the brand new ride-sharing services Uber and Lyft because you had to have an expensive new smartphone to hail them. They'd instead have to call and then pay for a full fare taxi, assuming one would stop for them in the poor neighborhood, or wait an hour and a half to catch the bus. I remember when I had to finally ditch my gaming laptop, with the world-class video card you could've done crypto mining on, because opening more than 5 browser tabs would churn the CPU and hard-drive, max out the RAM, and crash the browser. I remember having to upgrade my operating system, because it could no longer run a new enough browser, that was now required to load most web pages. I remember buying smartphone after smartphone after smartphone - not because the previous one stopped working, but because more apps required more cpu and more memory and more storage. I remember trying to download and run a chat app on my local machine, and running out of memory, because the chat app had an embedded web browser. I remember running out of my data cap on my cell phone because some app decided it wanted to stream a load of data as if it was just unlimited. I remember running out of space on my smartphone because 70% of the space was being used just to store the Operating System files. I'm not complaining, though. It's just how the world works. Humanity grows and consumes ever more resources. The people at the top demand a newer, better cake, and they get it; everyone else picks up the crumbs, until they too get something resembling cake. I sure ate my share. Lately I try to eat as little cake as possible. Doesn't change the world, but does make me feel better. Almost like the cake is a lie.
• self_awareness 2y ago
  What are the hopes for "engineering for slow internet" to happen, when people have engineered applications for "fast internet" when all we had was "slow internet". Nice thought in theory but unnecessarily gives false hope. Confluence/Jira sometimes need to download 20 megabytes in order to show a page with only text and some icons. I have a friend that tells me that in their company they had two jiras, one for developers, one for the rest of the company, because it was that dead slow. I've lost all faith already that this will change for better.
  • finack 2y ago
    [flagged]
    • mattgreenrocks 2y ago
      The beautiful thing about engineering is that the skill differential is apparent all the way from conception to implementation. But our industry is allergic to actually calling out actual skill issues that impact end users.
      • finack 2y ago
        [flagged]
• Tepix 2y ago
  I had a similar experience as the author on a boat in the south pacific. Starlink was available but often wasn't used because of its high power usage (60+ watts). So we got local SIM cards instead which provided 4G internet in some locations and EDGE (2G) in others. EDGE by itself isn't too bad on paper - you get a couple dozen kilobits per second. In reality, it was much worse. I ran into apps with short timeouts that would have worked just fine, if the authors had taken into account that loading can take minutes instead of milliseconds. Low bandwith, high latency connections need to be part of the regular testing of software. For Linux, there's netem (https://wiki.linuxfoundation.org/networking/netem https://wiki.linuxfoundation.org/networking/netem) that will let you do this. An issue that the anonymous blog author didn't have was metered connections. Doing OS or even app upgrades was pretty much out of the question for cost reasons. Luckily, every few weeks or so, we got to a location with an unmetered connection to perform such things. But we got very familiar with the various operating systems' ways to mark connections as metered/unmetered disable all automatic updates and save precious bandwidth.
  • chipdart 2y ago
    > Low bandwith, high latency connections need to be part of the regular testing of software. One size does not fit all. It would be a waste of time and effort to architect (or redesign) an app just because a residual subset of potential users might find themselves on a boat in the middle of the Pacific. Let's keep things in perspective: some projects even skip testing WebApps on more than one browser because they deem that wasteful and an unjustified expense, even though it's trivial to include them on a test matrix, and this is a UI-only.
    • fragmede 2y ago
      it's not a total redesign, it's just raising a timeout from 30 to 3000
    • Nevolihs 2y ago
      Websites regularly break because I don't have perfect network coverage on my phone every single day. In a lot of places, I don't even have decent reception. This in Germany in and around a major city. Why do you think this only applies to people on a boat?
• deleted 2y ago
  [deleted]
• throw46365 2y ago
  As a web developer I actually resisted much faster internet for ages. Until 2022 I had a rock-solid, never-failed 7 megabit/s-ish down, 640k up connection and I found it very easy to build sites that others describe as blazing fast. This was slow really by the standards of much of the UK population even by 2015. So all I had to do was make it fast for me. A change of provider for practical reasons gave me an ADSL2+ connection that is ten times faster; still arguably slower than a lot of residential broadband in the UK but not so helpfully. So now I test speed on mobile; even in the south east of England it is not that difficult to find poor mobile broadband. And when it’s poor, it’s poor in arguably more varied ways.
  • oefrha 2y ago
    As a web developer you can just throttle your connection in developer tools though, no self-limiting required. But nobody does that in big corporations building most of the sites needed by people with slow connections.
    • throw46365 2y ago
      Yeah, though it doesn’t quite capture all of the experience of working with slower broadband. For example if you have a website that is meant to be used alongside a video call or while watching video, it’s difficult to really simulate all of that “feel”. Using a link that is slow in practice is an invaluable experience.
      • xandrius 2y ago
        You can install programs at software level to emulate it as a whole. I remember using one for OSX and it working pretty well.
        • Aachen 2y ago
          In fact, you probably already have that program and just need to tell the OS what parameters you want
    • 2y ago
• deleted 2y ago
  [deleted]
• alicewilma52 2y ago
  [dead]
• parentheses 2y ago
  I recall using a device called the "Mini Maxwell" by this company: https://www.iwl.com/ https://www.iwl.com/ It enabled you to simulate network slow-downs, packet-loss, packet corruption, packet reordering and more. It was so critical in testing our highly network sensitive software.
  • RedShift1 2y ago
    For browser apps, a network connection simulator is included in the Chrome developer tools' Network tab.
    • christophilus 2y ago
      It doesn’t do a great job, though. A good simulator will have random drops, jitter, bursts, etc.
• KronisLV 2y ago
  I don't think any of our apps are built with slow connections in mind at all. Most of our web libraries and frameworks are indeed quite bloated (with features of convenience), downloading 20 MB of JS and 50 MB of content in total to render a page is insane when you think about it. We'd need to be able to turn off most images or visual elements to focus purely on the elements and their functionality, except for cases where displaying an image is critical for the function (and even then give the choice of showing a low quality version with a smaller file size). Things like web safe fonts that are already present in the browser/OS, most likely no libraries like React or Vue either (maybe Preact or Svelte). We'd need to allow for really long request (say, fetch) timeout values, maybe even to choose how to set them based on the connection quality (if a user has a really fast connection but a request suddenly hangs and is taking upwards of a minute, something has probably gone wrong and it'd make sense to fail that request, vs a user in a remote area for whom all requests are similarly slow), assuming that the server doesn't mind connections that linger around for a long time at slow speeds. We'd also need to allow for configuring an arbitrary HTTP cache/proxy for any site visited and file requested (say, store up to 1 TB on some local server based on file hashes and return the same for any user that requests that), but obviously things don't usually work that way over privacy/security concerns (nowadays different sites even download duplicate copies of the same files due to changes in the browsers: https://www.peakhour.io/blog/cache-partitioning-firefox-chrome/ https://www.peakhour.io/blog/cache-partitioning-firefox-chro... ). Maybe even for any web request that the OS might want to do, like system updates, basically a full on MitM for the whole system. Speaking of which, no more Electron or large software packages. Only native software with like Win32/WPF or maybe something like GTK/Qt, but nowadays it seems like even phone apps, not just desktop software, often don't use the system GUI frameworks, but instead ship a bunch of visual fluff, which might look nice and work well, but also takes up a bunch of space. I don't think there are incentives out there to guide us towards a world like that, which doesn't quite make sense to me. Lightweight websites should lead to better customer/user retention, but in practice that doesn't seem like something that anyone is optimizing for - ads everywhere, numerous tracking scripts, even autoplay videos, for everything from news sites to e-commerce shops. People who do optimize for that sort of stuff, seem to be a part of a smaller niche enthusiast community (which is still nice to see), like: https://1mb.club/ https://1mb.club/ https://512kb.club/ https://512kb.club/ Admittedly, even I'm guilty of bloating my homepage size from ~150 KB to ~ 600 KB due to wanting to use a custom set of fonts (that I host myself), even WOFF2 didn't save me there.
• padolsey 2y ago
  A lot of this resonates. I'm not in Antartica, I'm in Beijing, but still struggle with the internet. Being behind the great firewall means using creative approaches. VPNs only sometimes work, and each leaves a signature that the firewall's hueristics and ML can eventually catch onto. Even state-mandated ones are 'gently' limited at times of political sensitivity. It all ends up meaning that, even if I get a connection, it's not stable, and it's so painful to sink precious packets into pointless web-app-react-crap roundtrips. I feel like some devs need to time-travel back to 2005 or something and develop for that era in order to learn how to build things nimbly. In deficit of time travel, if people could just learn to open web tools and use its throttling tool: turn it to 3g, and see if their webapp is resilient. Please!
  • arvinsim 2y ago
    Devs only build for the requirements they are given. You want performance? Then include it in the requirements and give it the necessary time budget in a project.
    • sieabahlpark 2y ago
      [dead]
  • j45 2y ago
    The amount of complexity to generate HTML/JS is a little staggering sometimes for the majority of simple use cases. Using Facebook level architectures for actually pretty basic needs can be like hitting an ant-sized problem with a sledgehammer and wondering why the sledgehammer is so heavy and awkward to swing for little things.
  • watusername 2y ago
    > It all ends up meaning that, even if I get a connection, it's not stable I feel you, but the experience can actually be very good if you invest a bit of time looking at Shadowsocks/V2Ray and building your own infra.
  • tonetegeatinst 2y ago
• cricketlover 2y ago
  Great post. I was asked this question in an interview which I completely bombed, where the interviewer wanted me to think of flaky networks while designing an image upload system. I spoke about things like chunking, but didn't cover timeouts, variable chunk size and also just sizing up the network conditions and then adjusting those parameters. Not to mention having a good UX and explaining to the customer what's going on, helping with session resumption. I regret it. Couldn't make it through :(
• teleforce 2y ago
  The kind of scenario screams a local-first applications and solutions, and it's the reason why the Internet was created in the first place [1][2]. People have been duped by the misleading no software of Salesforce's advert slogan that goes against the very foundation and the spirit of the Internet. For most of its life and duration starting back in 1969, the Mbps is the anomaly not the norm and the its first killer application of email messaging (arguably the still best Internet application) is a local-first [3]. Ironically the culprit application that the author was lamenting in the article is a messaging app. [1] Local-first software: You own your data, in spite of the cloud: https://www.inkandswitch.com/local-first/ https://www.inkandswitch.com/local-first/ [2] Local-first Software: https://localfirstweb.dev/ https://localfirstweb.dev/ [3] Leonard Kleinrock: Mr. Internet: https://www.latimes.com/opinion/la-oe-morrison-use24-2009oct24-story.html https://www.latimes.com/opinion/la-oe-morrison-use24-2009oct...
• walterbell 2y ago
  IETF draft proposal to extend HTTP for efficient state synchronization, which could improve UX on slow networks, https://news.ycombinator.com/item?id=40480016 https://news.ycombinator.com/item?id=40480016 The Braid Protocol allows multiple synchronization algorithms to interoperate over a common network protocol, which any synchronizer's network messages can be translated into.. The current Braid specification extends HTTP with two dimensions of synchronization: Level 0: Today's HTTP Level 1: Subscriptions with Push Updates Level 2: P2P Consistency (Patches, Versions, Merges) Even though today's synchronizers use different protocols, their network messages convey the same types of information: versions in time, locations in space, and patches to regions of space across spans of time. The composition of any set of patches forms a mathematical structure called a braid—the forks, mergers, and re-orderings of space over time. Hope springs eternal!
  • YoshiRulz 2y ago
    This sounds suspiciously like Matrix. Does is required buy-in from user agents or will it benefit existing browsers once implemented?
    • walterbell 2y ago
      From https://braid.org/ https://braid.org/ Braid is backwards-compatible with today's web, works in today's browsers, and is easy to add to existing web applications.. You can use Braid features in Chrome with the Braid-Chrome extension. Demo of Statebus+Braid sync on existing browsers: https://stateb.us/#demos https://stateb.us/#demos
  • klabb3 2y ago
    Grump take: More complex technology will not fix a business-social problem. In fact, you have to go out of your way to make things this shitty. It’s not hard to build things with few round trips and less bloat, it’s much easier. The bloat is there for completely different reasons. Sometimes the bloat is unnoticeable on juicy machines and fast internet close to the DC. You can simulate that easily, but it requires the company to care. Generally, ad-tech and friends cares very little about small cohorts of users. In fact, the only reason they care about end users at all is because they generate revenue for their actual customers, ie the advertisers.
• pech0rin 2y ago
  Great post. One thing though. Maybe the engineers were misguided but its possible they were trying to mitigate slow loris attacks. Which are annoying to deal with and hard to separate from users who are just sending data at a really slow pace. Having had to mitigate these attacks before, we usually do a global timeout on the backend. Maybe different but definitely a possibility.
• EdwardDiego 2y ago
  I gave a fellow who'd just come off the ice a ride while he was hitchhiking, he was saying that the blog author was somewhat resented by others because his blog posts, as amazing as they are, tended to hog what limited bandwidth they already had while the images uploaded, but he was given priority because the administration realised the PR value of it. Which I thought ties into the discussion about slow internet nicely.
  • Aachen 2y ago
    I was wondering about the practicalities indeed. Not everyone knows when their OS or applications decided it is now a great time to update. You'll have a phone in your pocket that is unnecessarily using all the bandwidth it can get its hands on, or maybe you're using the phone but just don't realise that watching a 720p video, while barely functional, also means the person trying to load it after you cannot watch even 480p anymore (you might not notice because you've got buffer and they'll give up before their buffer is filled enough to start playing). It seems as though there should be accounting so you at least know what % of traffic went to you in the last hour (and a reference value of bandwidth_available divided by connected_users so you know what % was your share if everyone had equal need of it), if not a system that deprioritises everyone unless you punched the button that says "yes I'm aware what bandwidth I'm using in the next [X≤24] hour(s) and actually need it, thank you" which'll set the QoS priority for your MAC/IP address to normal
• RedShift1 2y ago
  I would highly recommend not only testing on slow network connections, but also on slow computers, tablets and smartphones. At least in my case there was some low hanging fruit that immediately improved the experience on these slow devices which I would have never noticed had I not tested on slower machines.
• finack 2y ago
  [flagged]
  • The_Colonel 2y ago
    [flagged]
    • finack 2y ago
      [flagged]
  • kylehotchkiss 2y ago
    Step 4: audit whatever code ChatGPT wrote for you for performance impact
• HL33tibCe7 2y ago
  Why do writers like this feel so entitled to engineering effort from companies? Maybe companies don’t want to plow millions into microoptimising their sites so a handful of people in Antarctica can access them, when the vast majority of their clients can use their sites just fine.
  • theodote 2y ago
    The author puts a lot of effort into emphasizing that it's not just "a handful of people in Antarctica" facing such issues, but quite a noticeable percentage of global population with unstable or otherwise weird connectivity. The internet shouldn't be gatekept from people behind such limitations and reserved for the convenient "target audience" of companies, whoever that might be - especially when solutions to these problems are largely trivial (as presented in the article) and don't require that much "engineering effort" for companies of that scale, since they are already half-implemented, just not exposed to users. People should not be limited from employing already existing infrastructure to overcome their edge-case troubles just because that infrastructure is not exposed due to it being unnecessary to the "majority of the clients".
  • croes 2y ago
    You do realize it's not just Antarctica? There are lots of places in the world where the internet speed isn't great. I'm actually experiencing slow internet right now and I' m in Germany.
  • 1oooqooq 2y ago
    your comment is self defeating. for example, the audience of a site might be in southern Africa, with same bad connectivity, but the EU/UN site developers are in the north so they don't care and the consequences are in program adoption that fails to blame that. Or you might be doing business with a coffee producer and now your spiffy ERP is missing data because it's too much effort for them to update the orders, so your procurement team have to hire an intern to do it over the phone costing you an extra 2k a month. Or which is more likely for the crowd here, yall losing clients left and right because your lazy sysadmin blocked entire country ip ranges because once they saw a single DoS wave from there.
  • 2y ago
• 1oooqooq 2y ago
  one American have to go to Antarctica for this site to note the obvious for everyone in most of Africa and south America.
• yason 2y ago
  I cringe whenever I think how blazing fast things could be today if only we hadn't bloated the web by 1000x. In the dial-up era things used to be unworldly fast merely by getting access to something like 10M ethernet. Now mobile connections are way, way faster than physical connections in the 90's but web pages aren't few KB, they are few MB at minimum. It takes four seconds and 2.5 MB to load my local meteorological institute's weather page which changes not more often than maybe once an hour and could be cached and served as a static base page in a few dozen milliseconds (i.e. instantly). A modern connection that's plenty capable to support all my remote work and development over a VPN and interactive shells without any lag can't help me get modern web pages load any faster because of the amount of data and the required processing/execution of a million lines of javascript that's imported from a bunch of number of sources, with the appropriate handshake delays of new connections implied, for each page load. A weather page from 2004 served exactly the same information as a weather page from 2024, and that information is everything required to get a sufficient glimpse of today's weather. One web page could be fixed but there are billions of URIs that load poorly. The overall user experience hasn't improved much, if at all. Yes, you can stream 4K video without any problems which reveals how fast things actually are today but you won't see it when browsing common pages -- I'd actually like to say web pages have only gone slower despite the improvements in bandwidth and processing power. When many pages still had mobile versions it was occasionally a very welcome alternative. Either the mobile version was so crappy you wanted to use the desktop version on your phone, or it was so good you wanted to predominantly load the mobile version even on desktop. I'd love to see an information internet where things like weather data, news articles, forum posts, etc. would be downloadable as snippets of plaintext, presumably intended to be machine readable, and "web" would actually be a www site that builds a presentation and UI for loading and viewing these snippets. You could use whichever "web" you want but you would still ultimately see the same information. This would disconnect information sources from the presentation which I think is the reason web sites started considering "browser" a programmable platform, thus taking away user control and each site bloating their pages each individually, leaving no choice for the user but maybe some 3rd party monkeyscripts or forced CSS rules. If the end user could always choose the presentation, the user would be greatly empowered in comparison to the current state of affairs where web users are currently being tamed down to be mere receivers or consumers of information, much not unlike passive TV viewers.
• xenodium 2y ago
  Not yet officially launched, but I’m working on a no-bloat, no-tracking, no-JS… blogging platform, powered by a drag/drop markdown file: https://lmno.lol https://lmno.lol Blogs can be read from just about any device (or your favourite terminal). My blog, as an example: https://lmno.lol/alvaro https://lmno.lol/alvaro Shared more details at https://indieweb.social/@xenodium/112265481282475542 https://indieweb.social/@xenodium/112265481282475542 ps. If keen to join as an early adopter, email help at lmno.lol
• nicbou 2y ago
  I travel a lot. Slow internet is pretty common. Also, right now my mobile data ran out and I'm capped at 8 kbps. Websites that are Just Text On A Page should load fast, but many don't. Hacker News is blazing fast, but Google's API docs never load. The worst problem is that most UIs fail to account for slow requests. Buttons feel broken. Things that really shouldn't need megabytes of data to load still take minutes to load or just fail. Google Maps' entire UI is broken. I wish that developers spent more time designing and testing for slow internet. Instead we get data hungry websites that only work great on fast company laptops with fast internet. --- On a related note, I run a website for a living, and moving to a static site generators was one of the best productivity moves I've made. Instead of the latency of a CMS permeating everything I do, I edit text files at blazing speed, even when fully offline. I just push changes once I'm back online. It's a game changer.
  • habosa 2y ago
    > Google's API docs never load I used to work on the team that served those docs. Due to some unfortunate technical decisions made in the name of making docs dynamic/interactive they are almost entirely uncached. Basically every request you send hits an AppEngine app which runs Python code to send you back the HTML. So even though it looks like it should be fast, it’s not.
  • Tade0 2y ago
    One additional benefit of static sites, which I learned the hard way, is that you're mostly immune to attacks. I have a domain that's currently marked as "dangerous" because I didn't use the latest version of Wordpress.
    • qingcharles 2y ago
      I had a client that I set up with a static site generator. Sadly the client changed their FTP password to something insecure and someone FTP'd in and added a tiny piece of code to every HTML file!
  • daveoc64 2y ago
    >Websites that are Just Text On A Page should load fast, but many don't. Hacker News is blazing fast, but Google's API docs never load. Things aren't always that simple. I'm in the UK, and my ping time to news.ycombinator.com is 147ms - presumably because it's not using a CDN and is hosted in the USA. cloud.google.com on the other hand has an 8ms ping time. So yes, Hacker News is a simple, low-JS page - but there can be other factors that make it feel slow for users in some places. This is despite me being in a privileged situation, having an XGS-PON fibre connection providing symmetric 8Gbps speeds.
• lawgimenez 2y ago
  From where I'm from (Southeast Asia), slow internet is common in provincial and remote areas. It's like the OP's experience in South Pole but slower. That's why I always cringe at these fancy-looking UI cross-platform apps since I know they will never work in a remote environment. Also, that is why offline support is very important. I only use Apple Notes and Things 3, both work tremendously in such remote settings. Imagine your notes or to-do list (ehem Basecamp) not loading since it needs internet connection
  • bombcar 2y ago
    What's sad is that the app-style setup on phones SHOULD be perfect for this - you download the app when you DO have a good connection, and then when you're out on the slow/intermittent connection the ONLY thing the app is sending is the new data needed. Instead almost all apps are just a bad web browser that goes to one webpage.
• 29athrowaway 2y ago
  A must read https://www.gamedeveloper.com/programming/1500-archers-on-a-28-8-network-programming-in-age-of-empires-and-beyond https://www.gamedeveloper.com/programming/1500-archers-on-a-...
• onenukecourse 2y ago
  List didn't include the one we're all most likely to encounter - crappy hotel connections.
• anthk 2y ago
  Mosh and NNCP will help a lot, but you need some good sysadmin to set NNCP as the mail MUA/MTA backend to spool everything efficiently. NNCP it's an expert level skill, but your data will be sent over very unreliable channels: https://nncp.mirrors.quux.org/Use-cases.html https://nncp.mirrors.quux.org/Use-cases.html Also, relying on propietary OS'es is not recommended. Apple and iOS are disasters to work on remote, isolated places. No wonder no one uses Apple in Europe for any serious work except for iOS development. Most science and engineering setups will use anything else as a backend. Most offline distros, like Ubuntu or Trisquel have methods of downloading the software packages for offline installs. On chats, Slack and Zoom are disasters. Any SIP or Jabber client with VOIP support will be far more reliable as it can use several different protocols which can use far less bandwidth (OPUS for audio) without having to download tons of JS to use it, even if you cached your web app it will still download tons of crap in the background. And, again, distros like Debian have a full offline DVD/BR pack which ironically can be better if you got that by mail. Or you can just use the downloaded/stored ISO files with apt-cdrom add -m /path/to/your/file.iso This way everything from Debian could be installed without even having an Internet connection.
  • greenish_shores 2y ago
    Well, statistically average end-user internet connection in Europe is much faster than in the US. Maybe outside some places like most of western Germany, but these are an exception. Europe has really good bandwidth speeds, overall. I absolutely agree with the rest, though, including the part saying any "serious" software will have such features (and better support in general), and I second the examples you gave.
    • anthk 2y ago
      USA it's huge and sparse, but Spain it's the same, it's like USA.rar. Crowded coasts, lots of mountains with a rough interior almost as empty as Lapland modulo Madrid. So, yes, you can have the issues on phone signal issues.
      • greenish_shores 2y ago
• ajsnigrutin 2y ago
  Slow internet and also slow devices! If you're targeting the general population (so, chat service, banking app, utility app,...), you should be targeting the majority of users, not just the new-flagship ones, so all the testing should be done on a cheapest smartphone you could buy in a supermarket two years ago (because well.. that's what "grandmas" use). Then downgrade the connection to 3g, or maybe even edge speeds (can be smulated on network devices), and the app/service should still work. Somehow it seems that devs get the best new flagships only, optimize the software for that, and forget about the rest... and I understand that for a 3d shooter game or something, but an eg. banking app, should work on older devices too!
• Karrot_Kream 2y ago
  So I've hacked a lot on networking things over the years and have spent time getting my own "slow internet" cases working. Nothing as interesting as McMurdo by far but I've chatted and watched YouTube videos on international flights, trains through the middle of nowhere, crappy rural hotels, and through tunnels. If you have access/the power (since these tend to be power hungry) to a general-purpose computing device and are willing to roll your own my suggestion is to use NNCP [1]. NNCP can can take data, chunk it, then send it. It also comes with a sync protocol that uses noise (though I can't remember if this enables 0RTT) over TCP (no TLS needed so only 1.5 RTT time spent establishing connection) and sends chunks, retrying failed chunks along the way. NNCP supports feeding data as stdin to a remote program. I wrote a YouTube downloader, a Slack bot, a Telegram bot, and a Discord bot that reads incoming data and interacts with the appropriate services. On the local machine I have a local Matrix (Dendrite) server and bot running which sends data to the appropriate remote service via NNCP. You'll still want to hope (or experiment such) that MTU/MSS along your path is as low as possible to support frequent TCP level retries, but this setup has never really failed me wherever I go and let's me consume media and chat. The most annoying thing on an international flight is that the NNCP endpoint isn't geographically distributed and depending on the route your packets end up taking to the endpoint, this could add a lot of latency and jitter. I try to locate my NNCP endpoint near my destination but based on the flight's WiFi the actual path may be terrible. NNCP now has Yggdrasil support which may ameliorate this (and help control MTU issues) but I've never tried Ygg under these conditions. [1]: http://www.nncpgo.org/ http://www.nncpgo.org/
  • alanpearce 2y ago
    This sounds fascinating. Do you have some articles describing your setup?
    • Karrot_Kream 2y ago
      Hah no, but maybe I should. The reason I haven't is that most of my work is just glue code. I use yt-dlp to do Youtube downloads, make use of the Discord, Slack and Telegram APIs to access those services. I run NNCP and the bots in systemd units, though at this point I should probably bake all of these into a VM and just bring it up on whichever cloud instance I want to act as ingress. Cloud IPs stay static as long as the box itself stays up so you don't need to deal with DNS either. John Goerzen has a bunch of articles about using NNCP [1] that I do recommend interested folks look into but given the popularity of my post maybe I should write an article on my setup. FWIW I think it's fine that major services do not work under these conditions, though I wish messaging apps did. Both WhatsApp and Telegram IME are well tuned for poor network conditions and do take a lot of these issues into account (a former WA engineer comments in this thread and you can see their attention to detail.) Complaining about these things a lot is sort of like eating out at restaurants and complaining at how much sodium and fat goes into the dishes: restaurants have to turn a profit and catering to niche dietary needs just isn't enough for them to survive. You can always cook at home and get the macros you want. But for you to "cook" your own software you need access to APIs and I'm glad Telegram, Slack, and Discord make this fairly easy. Youtube yt-dlp does the heavy lifting but I wish it were easier, at least for Premium subscribers, to access Youtube via API. I find Slack to be the absolute worst offender networking-wise. I have no idea how, now that Slack is owned by Salesforce, the app experience can continue to be so crappy on network usage. It's obvious that management there does not prioritize the experience under non-ideal conditions in any way possible. Their app's usage of networks is almost shameful in how bad it is. [1]: https://www.complete.org/nncp/ https://www.complete.org/nncp/
• langsoul-com 2y ago
  Too bad only US has starlink active. Other nations, like Australia, has nothing...
  • daveoc64 2y ago
    Starlink is available in something like 40 countries, including Australia. Unless you're specifically talking about bases in Antarctica?
    • langsoul-com 2y ago
      Yeah I'm talking about Antarctica. I was really thinking about going there, just to see how it is. But the lack thereof proper net puts me off. No different from regional Australia, which ALWAYS has some crippling addiction, whether it's alcohol, domestic violence or something else. Not the best environment.
• hooby 2y ago
  All of that sounds like, that torrent-based updaters/downloaders should be the absolute killer-app for environments like that. Infinitely resume-able, never looses progress, remains completely unfazed by timeouts, connection loss, etc. - and the ability to share received update-data between multiple devices peer-to-peer.
• AllegedAlec 2y ago
  I currently develop applications which are used on machines with very spotty connections and speeds which are still calculated in Baud. We have to hand-write compression protocols to optimize for our use. Any updates/installs over network are out of the question. It's a great lesson in the importance of edge computing, but it also provides some harsh trusth about the current way we produce software. We cannot afford to deliver a spotty product. To get new updates out to all parties takes a prohibitively long time. This is hard for new people or outsiders giving courses to grok, and makes most modern devops practices useless to us.
• miyuru 2y ago
  Wonder if the author tried BitTorrent. Despite its usage issues, its built with reliability in mind and its a great way to transfer large files.
  • palata 2y ago
    How do you use WhatsApp over BitTorrent? And how do you update your macOS over BitTorrent? The author clearly says that downloaders elaborate enough to deal with slow connections (e.g. "download in a browser") were fine. The problem is that modern apps don't let you download the file the way you want, they just expect you to have a fast internet connection.
    • miyuru 2y ago
      No, I just meant if he tried to get a large file(Linux ISO) with BitTorrent. which should be reliable in theory. Bittorent has webseeds support, which can use apples direct CDN urls to create a torrent file to download. archive.org still uses this technique and AWS S3 used to do this when they had torrent support. There a website that do just that, it creates a torrent file from any direct weburl. https://www.urlhash.com/ https://www.urlhash.com/
      • Aachen 2y ago
        I didn't know that was possible! Thanks so much, I've got a torrent client but with so few things using bittorrent these days, it feels like innovation went backwards and it's now one-shot http downloads or bust. This will be helpful :)
      • palata 2y ago
        I can't believe I had never heard about webseeds [1] before! Do I understand correctly that one can make a torrent from any HTTP-available file (without even owning the file) and start downloading and sharing with BitTorrent? It's incredible! https://en.wikipedia.org/wiki/BitTorrent#Web_seeding https://en.wikipedia.org/wiki/BitTorrent#Web_seeding
• dano 2y ago
  Back in 1999 while at MP3.com it was common for people outside of engineering to complain about speed when they were at home. In the office we had 1G symmetric (that was lot 25 yrs ago!) between the office and primary DC. I tied to explain that the large graphics wanted by some, and heavy videos didn't work great for dial up or with snow cable modem connections. Surely the servers are misconfigured!
• xacky 2y ago
  Try using the internet on an "exchange only line". Technically it's broadband but its speeds are still dialup tier. I know several streets in my city that still have these connections.
  • xacky 2y ago
    And also ever since the 3G shutdown in the UK phones often fall back to GPRS and EDGE connections (2G), as 2G is not scheduled to shut down in the UK until 2033. I know several apps that are too slow to work in such conditions, as they are developed by people who use the latest 5G links in urban locations instead of testing it in rural and suburban areas with large amounts of trees.
• ape4 2y ago
  Perhaps they could benefit from Usenet - with its store-and-forward attributes
• spintin 2y ago
  [dead]
• amelius 2y ago
  I have a question about this. I have multiple instances of curl running in different terminals, all using a single slow internet connection. How can I give priority to one of these instances?
  • meindnoch 2y ago
    Create multiple virtual interfaces, apply different traffic shaping to them, and then use the --interface option of cURL.
    • amelius 2y ago
      Thanks, but I don't have root access to my machine.
      • Aachen 2y ago
        That's not your machine But to add something useful as well, you could just ctrl+z the ones you want to pause to let the priority one finish?
• nashashmi 2y ago
  It takes a “special” skill Level to develop web applications in JS for low bandwidth connections. it takes time because frameworks and libraries are not built for this. there are very few libraries and frameworks in JS that are optimized for Low bandwidth connections. this requires having to program applications from scratch. I went through such a process. Took me two weeks versus two hours using JS query
  • Aachen 2y ago
    What's odd is that I have the exact opposite: I find it uncomfortable to use megabytes of libraries with hundreds of dependencies when I can spend 30 seconds and discover the html5 native way of doing something. But my applications look butt ugly. They're fast and privacy preserving, but also it requires a special skill, from my point of view, to make them look good (be it design skills or knowing how to use these frameworks)
    • filleduchaos 2y ago
      The "html5 native way of doing things" is not necessarily inherently fast (or reliable) without scripting tbh.
      • Aachen 2y ago
        I'm thinking of UI elements that are often half-implemented in JavaScript but also exist natively nowadays, such as sliders. What kind of thing are you thinking of?
        • filleduchaos 2y ago
          If by slider you mean range slider, they only really work out-of-the-box for integer values. If by slider you mean carousel, HTML/CSS-only carousels lack a full range of interactivity without...scripting And both those things don't really have anything to do with application frameworks.
• austin-cheney 2y ago
  70%+ of the web is putting text on screen and responding to user interactions, 25%+ is spyware and advertising, and the last 5% are cool applications. How complicated should that really be? This is a good example of why I gave up a career as a JavaScript developer after 15 years. I got tired of fighting stupid, but even stupid woefully unqualified people need to make 6 figures spinning their wheels to justify their existence.
• password4321 2y ago
  It doesn't take much to slow down RDP over TCP (especially when port forwarding through SSH). I did find mention of increasing the cache¹ and lowering the refresh rate to 4 fps² (avoiding unnecessary animations), but I still feel the need for a server-side QUIC proxy that is less pushy based on network conditions. There is a red team project that has the protocol parsed out in Python³ instead of all the ActiveX control clients. ¹ https://superuser.com/questions/13487/how-to-increase-performance-of-remote-desktop https://superuser.com/questions/13487/how-to-increase-perfor... ² https://learn.microsoft.com/en-us/troubleshoot/windows-server/remote/frame-rate-limited-to-30-fps https://learn.microsoft.com/en-us/troubleshoot/windows-serve... ³ https://github.com/GoSecure/pyrdp https://github.com/GoSecure/pyrdp
• matlin 2y ago
  It's funny how similar the problems that affect a workstation in Antartica are to designing a robust mobile app. I personally think all apps benefit from being less reliant on a stable internet connection and that's why there's a growing local-first movement and why I'm working on Triplit[1]. [1] www.triplit.dev
• luuurker 2y ago
  Those of us working on apps, websites, etc, need to remember that there are lots of people out there that are not connected to the fast Wi-Fi or fibre connections we have. Here in the UK, some networks started shutting down 3G. Some have 2G as a low energy fall back, but we're supposed to use 4G/5G now. The problem is that 4G is not available everywhere yet, some areas until recently only had good 3G signal. So I've been dropping to 2G/EDGE more often than I'd like and a lot of stuff just stops working. A lot of apps are just not tested on slow, high latency, high package loss scenarios.
  • jimmaswell 2y ago
    Shutting down 3G was a mistake. Besides turning so many devices into e-waste, it was a good backup when 4g was congested.
    • HPsquared 2y ago
      The lower-bandwidth connections get completely saturated by modern phones with modern data allowances. Back in the day I had 500MB a month on 3G, for instance. I can use that in a few minutes these days.
      • kjkjadksj 2y ago
        Thats been true since the iphone 3g with unlimited data plans though
        • HPsquared 2y ago
          Modern phones and apps use a lot more though. YouTube 1080p60 or even 4K, for example.
          • deleted 2y ago
            [deleted]
    • luuurker 2y ago
• the__alchemist 2y ago
  Thank you! I am in the East coast US, and consistently find web sites and internet-connected applications are too slow. If I am at home, they are probably fine, but on mobile internet? Coffee shops etc? Traveling? No! No excuses! It is easier than ever to build fast, interactive websites, now that modern, native Javascript includes so many niceties. Using JS dependencies is a minefield. At work, where I give less of a fuck, a dev recently brought in Material UI and Plotly. My god.
• gmuslera 2y ago
  It is not just bandwidth or latency, and is not just for Antarctica. Not in all places of the world you have the best connectivity. Even with not so bad connectivity, you may have environmental interference, shared use or be just far from the wifi router. You may have a browser running in a not so powerful CPU, doing more things chewing processor, or the available memory, so heavy JS sites may suffer or not work at all there. You don't know what is in the other side, putting high requirements there may turn your solution unfit for a lot of situations. Things should be improving (sometimes fast, sometimes slowly) in that direction, but still is not something guaranteed everywhere, or at least in every place that your application is intended or needed to run. And there may be even setbacks in that road.
  • kjkjadksj 2y ago
    I run into this daily on my phone. Where I live, its hilly, network is usually saturated, my speeds are crap usually and some sites more complicated than hn cannot even load at all without timing out sometimes.
• cletus 2y ago
  So I have some experience with this because I wrote the non-Fleash Speedtest for Google Fiber. I also have experience with this by virtue of being from Australia. Let me explain. So Google Fiber needed a pure JS Speedtest for installers to verify connections. Installers were issued with Chromebooks, which don't support Flash and the Ookla Speedtest at the time used Flash. There's actually good reasons for this. It turns out figuring out the maximum capacity of a network link is a nontrivial problem. You can crash the browser with too much traffic (or just slow down your reported result). You can easily under-report speed by not sending enough traffic. You have to weigh packet sizes with throughput. You need to stop browsers trying to be helpful by caching things (by ignoring caching headers). There's a long list. So I did get a pure JS Speedtest that could actually run up to about ~8.5Gbps on a 10GbE link to a Macbook (external 10GbE controller over TB3). You learn just how super-sensitive throughput is to latency due to TCP throttling. This is a known and longstanding problem, which is why Google invested in newer congestion control schemes like BRR [1]. Anyway, adding 100ms of latency to a 1GbE connection would drop the measured throughput from ~920-930Mbps to a fraction of that. It's been a few years so I don't remember the exact numbers but even with adjustments I recall the drop off being like 50-90%. The author here talks about satellite Internet to Antarctica that isn't always available. That is indeed a cool application but you don't need to go this extreme. You have this throughput problem even in Australia because pure distance pretty much gives you 100ms minimum latency in some parts and there's literaly nothing you can do about it. It's actually amazing how much breaks or just plain sucks on that kind of latency. Networked applications are clearly not designed for this and have never been tested on it. This is a general problem with apps: some have never been tested in non-perfect Internet conditions. Just th eother day I was using one of the Citi-bike apps and it could hang trying to do some TCP query and you'd every now and again get "Connection timed out" pop ups to the user. That should never happen. This is the lazy dev's way of just giving up, of catching an exception and fatalling. I wish more people would actually test their experience when there was 100ms latency or if there was just random 2% packet loss. Standard TCP congestion control simply doesn't handle packet loss in a way that's desirable or appropriate to modern network conditions. [1]: https://cloud.google.com/blog/products/networking/tcp-bbr-congestion-control-comes-to-gcp-your-internet-just-got-faster https://cloud.google.com/blog/products/networking/tcp-bbr-co...
• juangacovas 2y ago
  Come to Spain. We haven't everything, but our optic fibers and internet plans are top notch ;P
• phaedrus 2y ago
  One of my first professional software projects, as an intern, was I wrote a tool for simulating this type of latency. I modeled it as a set of pipe objects that you could chain together with command line arguments. There was one that would do a fixed delay, another that would introduce random dropped packets, a tee component in case you wanted to send traffic to another port as well, etc.
• infinet 2y ago
  I had similar problem on a ship with many users share a 2M VSAT Internet. Few tricks made Internet less painful: - block windows update by returning DNS query for microsoft update endpoints as NXDOMAIN. - use a captive portal to limit user session duration, so that unattended devices won't consume bandwidth. - with freebsd dummynet, pfSense can share bandwidth equally among users. It can also share bandwidth by weight among groups. It helps. - inside Arctic circle, the geosynchronous satellites are very low on the horizon and were blocked frequently when ship turns. I was able to read the ship's gyro and available satellites from VSAT controller and generate a plot to show the satellite blockage. It was so popular that everyone is using it to forecast next satellite online.
• gbalduzzi 2y ago
  I agree with the overall take by OP, but I find this point quite problematic: > If you have the ability to measure whether bytes are flowing, and they are, leave them alone, no matter how slow. Perhaps show some UI indicating what is happening. Allowing this means easy DDOS attack. An attacker can simply keep thousand of connections open
  • klabb3 2y ago
    Close after 10-60s of complete inactivity, don’t use JS bloatware and allow for range/etag requests should go a long way though. The issue is people setting fixed timeouts per request which isn’t meant for large transfers.
• jrhey 2y ago
  Engineering for slow CPUs next. No matter how fast our machines get these days, it's just never enough for the memory/CPU/battery hungry essential apps and operating systems we use nowadays.
• mayormcmatt 2y ago
  This topic resonates with me, because I'm currently building a horrible marketing static page with images and videos that top 150MB, prior to optimization. It causes me psychic pain to think about pushing that over the wire to people that might have data caps. Not my call, though...
• deleted 2y ago
  [deleted]
• beeandapenguin 2y ago
  This is why we need more incremental rendering[1] (or "streaming"). This pattern become somewhat of a lost art in the era of SPAs — it's been possible since HTTP/1.1 via chunked transfer encoding, allowing servers to start sending a response without knowing the total length. With this technique, the server can break down a page load into smaller chunks of UI, and progressively stream smaller parts of the UI to the client as they become available. No more waiting for the entire page to load in, especially in poor network conditions as the author experienced from Anartica. [1]: https://www.patterns.dev/react/streaming-ssr https://www.patterns.dev/react/streaming-ssr
  • jasoncartwright 2y ago
    I remember doing this with ASPv3 pages back in the day on a content site. It made it easy to dump what HTML has already been completed out before continuing to generate the heavier, but much less important, comments section below.
• nirav72 2y ago
  > From my berthing room at the South Pole, it was about 750 milliseconds I’m currently on a moving cruise ship in the Mediterranean with a starlink connection. A latency of 300-500 ms seems to be normal. Although bandwidth is tolerable at 2-4 mbps during the day with hundreds of passengers using it. At night it gets better. But latency can still be frustrating.
• deleted 2y ago
  [deleted]
• palata 2y ago
  One problem is that developers have the best hardware and Internet because "it's their job", so they are completely biased. A bit like rich people tend to not understand what it means to be poor. The other problem is that nobody in the software industry gives a damn. Everyone wants to make shiny apps with the last shiny tech. Try to mention optimizing for slow hardware/Internet and look at the face of your colleagues, behind their brand new M3. I worked in a company with some remote colleagues in Africa. There were projects that they could literally not build, because it would require downloading tens of GB of docker crap multiple times a week for no apparent reason. The solution was to not have those colleagues work on those projects. Nobody even considered that maybe there was something to fix somewhere.
• y-c-o-m-b 2y ago
  Some of these web apps are from very profitable or big companies and that drives me insane because they have more than enough funding to do things right. Take Home Depot for example. Loading their website in a mobile browser is soooooooooo slow. The rendering is atrocious, with elements jumping all over the place. You click on one thing and it ends up activating a completely different element, then you have to wait for whatever you just clicked to load and jump all over the place again. Very frustrating! Inside their stores is even worse! I asked for help locating an item from one of their workers one day and they pull up their in-store app. That too was slower than molasses and janky, so we ended up standing there for several minutes just chatting waiting for it to load.
• deleted 2y ago
  [deleted]
• AdamH12113 2y ago
  It's interesting that these are exactly the sort of conditions that the internet protocols were designed for. A typical suite of internet software from the 90s would handle easily. One key difference is that client software was installed locally, which (in modern terms) decouples UI from content. As the article points out, the actual data you're dealing with is often measured in bytes. An email reader or AOL Instant Messenger would only have to deal with that data (plus headers and basic login) instead of having to download an entire web app. And since the protocols didn't change often, there was no need to update software every few weeks (or months, or even years). Another key difference, which is less relevant today, is that more data came from servers on the local network. Email and Usenet were both designed to do bulk data transfer between servers and then let users download their individual data off of their local server. As I recall, email servers can spend several days making delivery attempts before giving up.
• _feynon 2y ago
  For the last three years, Rekka Bellum and Devine Lu Linvega have sailed the Pacific Ocean on Pino, a sailboat turned mobile studio, making videogames, art, and music with their own homegrown software. Off-grid for long stretcheS they’ve built a custom OS for their needs: https://100r.co/site/uxn.html https://100r.co/site/uxn.html Talk: https://youtu.be/BW32yUEymvU?si=YGsc3A3oAzowKbRU https://youtu.be/BW32yUEymvU?si=YGsc3A3oAzowKbRU