3 ms·

30MB binaries can be a bit of a problem for some people. I started developing a system from home and when deploying to a VPS for testing and production. Transf

by shirro 11y ago

30MB binaries can be a bit of a problem for some people. I started developing a system from home and when deploying to a VPS for testing and production. Transferring Go binaries was like waiting for a floppy disk to copy due to poor Internet infrastructure. Compression didn't help much. It turned out to be significantly faster to push to a remote git and have a hook do the build. I think before you look to compress a go binary consider if there aren't other options.

dietrichepp 11y ago
I feel your pain. My home internet connection is 24 Mbit/s down, but only 1 Mbit/s up. I asked my ISP for more, but it's basically impossible to get. Thank goodness for multipart uploads, at least.
- michaelcampbell 11y ago
  I've ameliorated this somewhat (I have 18/1), albeit at the expense of space, by uploading things to a common place, then copying or (sym)linking them to the final spot on the target computer. This gives me the ability to `rsync` them up rather than copy, using its ability to only transfer the pieces of the files that have changed. We have a ~20MB .war file that I have done this with over airport wifi, and it ended up only transfer a few 10's of KB (maybe 100) between compiles.
  - dietrichepp 11y ago
    I do something similar. I have a database of all my files on S3 and their SHA-256 hashes. This not only means that I can skip uploading files which haven't changed, but I can do S3->S3 copies for files which are the same. But it's still too often that I have 10s or 100s of MB of data to upload. At 1 Mbit/s, it takes a half hour for every 100 MB!
    - michaelcampbell 11y ago
      Indeed, sometimes it just sucks and you have to bite the bullet.
- shirro 11y ago
  I am in the same situation though my actual speed is more like 0.7 to 0.8Mbps up. Uploading a 30MB Go binary would take nearly 6 minutes. I guess I should be thankful I mainly work with text and not images, video or large proprietary file formats otherwise I think I would be out of the game. My phone is now about 5 times faster than my home Internet for both uploads and downloads but also significantly more variable and expensive. The fibre internet upgrade that was scheduled for last year had the technology downgraded and the rollout delayed due to politics.
  - gcb0 11y ago
    shop for dsl. if you can still find out one. they don't advertise much these days, buy there are still some around.
    - voltagex_ 11y ago
      Shirro's probably in Australia, with terrible copper infrastructure. The fastest upstream ADSL connection is 1 to 2 megabits and SHDSL is rare.
      - gcb0 11y ago
        Cooper being rare makes it more likely that they upgrade the end points to get 3mbps or more. I'd guess that hold true for down under as it does for the other places i know.
riobard 11y ago
SSH compression is your friend.
voltagex_ 11y ago
30MB is on the small side. I've seen Electron apps go above 100 megabytes, and Android Studio is about 1.1 gigabytes. I feel sorry for anyone developing on less than 10 megabit down/up. I've been throwing around an idea for a site called "Church of the Byte Savers" or some other silly name based around saving hard drive space and bandwidth. e.g. Dropbox is good because it has LAN sync, whereas OneDrive will send your data around the world just to get it to a second PC on your network.
- hvm 11y ago
  > "Church of the Byte Savers" I like that. I see people preaching how we've become accustomed to large drive space and fast internet and we're just wasting it all out of laziness. Kind of like the post Muad'dib Fremen that got soft on water discipline.
self 11y ago
Have you tried rsync? If most of your changes are small ones, it should help a lot. rsync --inplace -vaz sourcefile user@remote:/targetdir/
s_kilk 11y ago
A few years back I thought it would be fun to do a Scala project (which ended up being http://nightchamber.com http://nightchamber.com) while staying with my in-laws over christmas. It got much less fun when each deploy to the prod server took an hour to complete.

by shirro 11y ago

dietrichepp 11y ago

I feel your pain. My home internet connection is 24 Mbit/s down, but only 1 Mbit/s up. I asked my ISP for more, but it's basically impossible to get. Thank goodness for multipart uploads, at least.

michaelcampbell 11y ago
I've ameliorated this somewhat (I have 18/1), albeit at the expense of space, by uploading things to a common place, then copying or (sym)linking them to the final spot on the target computer. This gives me the ability to `rsync` them up rather than copy, using its ability to only transfer the pieces of the files that have changed. We have a ~20MB .war file that I have done this with over airport wifi, and it ended up only transfer a few 10's of KB (maybe 100) between compiles.
- dietrichepp 11y ago
  I do something similar. I have a database of all my files on S3 and their SHA-256 hashes. This not only means that I can skip uploading files which haven't changed, but I can do S3->S3 copies for files which are the same. But it's still too often that I have 10s or 100s of MB of data to upload. At 1 Mbit/s, it takes a half hour for every 100 MB!
  - michaelcampbell 11y ago
    Indeed, sometimes it just sucks and you have to bite the bullet.
shirro 11y ago
I am in the same situation though my actual speed is more like 0.7 to 0.8Mbps up. Uploading a 30MB Go binary would take nearly 6 minutes. I guess I should be thankful I mainly work with text and not images, video or large proprietary file formats otherwise I think I would be out of the game. My phone is now about 5 times faster than my home Internet for both uploads and downloads but also significantly more variable and expensive. The fibre internet upgrade that was scheduled for last year had the technology downgraded and the rollout delayed due to politics.
- gcb0 11y ago
  shop for dsl. if you can still find out one. they don't advertise much these days, buy there are still some around.
  - voltagex_ 11y ago
    Shirro's probably in Australia, with terrible copper infrastructure. The fastest upstream ADSL connection is 1 to 2 megabits and SHDSL is rare.
    - gcb0 11y ago
      Cooper being rare makes it more likely that they upgrade the end points to get 3mbps or more. I'd guess that hold true for down under as it does for the other places i know.

riobard 11y ago

SSH compression is your friend.

voltagex_ 11y ago

30MB is on the small side. I've seen Electron apps go above 100 megabytes, and Android Studio is about 1.1 gigabytes. I feel sorry for anyone developing on less than 10 megabit down/up. I've been throwing around an idea for a site called "Church of the Byte Savers" or some other silly name based around saving hard drive space and bandwidth. e.g. Dropbox is good because it has LAN sync, whereas OneDrive will send your data around the world just to get it to a second PC on your network.

hvm 11y ago
> "Church of the Byte Savers" I like that. I see people preaching how we've become accustomed to large drive space and fast internet and we're just wasting it all out of laziness. Kind of like the post Muad'dib Fremen that got soft on water discipline.

self 11y ago

Have you tried rsync? If most of your changes are small ones, it should help a lot. rsync --inplace -vaz sourcefile user@remote:/targetdir/

s_kilk 11y ago

A few years back I thought it would be fun to do a Scala project (which ended up being http://nightchamber.com http://nightchamber.com) while staying with my in-laws over christmas. It got much less fun when each deploy to the prod server took an hour to complete.