4 ms·

Genuine question: why the obsession with gigabit? What applications demand it?

by massysett 13y ago

teddyh 13y ago
What application would demand more than 640k of RAM? Or, to quote: “One day sir, you may tax it.”.
- smoyer 13y ago
  I had an Apple IIe (with 64KB of RAM) when the PC came out (with 640KB of RAM) and I also once stated "what is anyone going to do with that much RAM?" - It sounds silly now, but it taught me a lesson about automatically rejecting new technologies and 5 years later I was part of the company who was first to send CATV signals over fiber optics.
wmf 13y ago
What I see is an obsession with getting it for no marginal cost. If you're already paying $70/month for 20 Mbps, of course you wouldn't turn down gigabit for the same price (even if Web pages or Netflix won't load any faster).
alecco 13y ago
Vídeos, videoconferencing, uploading data, backups, peer to peer legal/other. Usually the proportions are 8mbit downlink and 100k uplink. Sharing data from nodes is hard.
foobarqux 13y ago
Cloud sync for your entire hard drive. Thin clients. Faster spin up of environments that are sourced from the Internet (virtual machine configuration, bundle install, docker images). High(er) definition video. Multi-party video conferencing.
wmeredith 13y ago
Streaming 3D in 4k would use it. Removing bottlenecks makes things possible.
Amadou 13y ago
TOR Bridge Nodes (relays).
Zigurd 13y ago
4k video
MaulingMonkey 13y ago
Game development. Setting up a new workspace generally is a "go out to lunch and it'll hopefully be mostly done with the perforce checkout by the time you're back" affair on our internal LAN, nevermind if you wanted to work from home. Downloading the built assets through our internal tools takes a similar length of time, so you hopefully can wait until the end of the day to kick that off as you head out. Various SDKs are delivered by download - my last download was 15GB, compressed. These often require frequent updates, which we save to local shares as a simple matter of expediency when shared widely. Delivering complete, self contained milestones in specific formats causing 12-50GB .zips to be uploaded via FTP for hours... with the semi-frequent resort to sneakernet due to quota limitations, service issues, or just plain not having hours between when development for a build was stopped and when the build was needed someplace for a demo or whatever else. I have an archive of some of those deliveries on my local machine. Even with a good 30%+ of my archives missing due to lack of drive space, the archives for the single project I've archives for total over a terrabyte. They're useful for reproducing bugs exactly on occasion. Between all the other projects, other employees, and monstrous shared resources like complete Perforce history, build machines and other servers... complete backup would probably be somewhere around the 0.1 petabytes mark... or close to a full petabit of data. And we're considered only a "small" business, under 50 employees! Fortunately for us backups are fairly static, and the nightly rsync to offsite storage is usually done bringing all our services to a grinding halt by the time work starts the next day. But it's sufficiently slow that those are our secondary backups, with primaries and restoration being done through sneakernet as the rule, not the exception, despite the "beefy" pipe. A second internet backup would push backup times into our workday, and if we're ever forced to do a major restoration through the internet, it's going to be extremely painful... days of waiting with employees unable to do their jobs. Expensive!
- chubot 13y ago
  Wow I'm surprised game development is still like that. I haven't done game development in almost 9 years, but the "Perforce saturating the LAN" problem sounds very familiar! So I'm actually working on a (binary) software distribution system, and I think it would work very well for this use case. This problem can be solved with software -- it doesn't require an upgrade to fiber. The (straightforward) idea is to use content-addressed storage and differential compression, pretty much like Git does. Then you can actually sync data from your neighbor's machine, in a BitTorrent-ish fashion (possibly getting tiny amounts of metadata from a central server for consistency). Perforce definitely has problems with spurious lock contention, and just a few team members syncing from the same machine can easily clog its pipes if you're not careful. I'm not up to date, but I'm pretty sure you can do better than Perforce's delta compression. You can probably do it with a single generic algorithm, but one way to really improve would be to use file-type specific compression, e.g. bsdiff or Chrome's Courgette for executables, and use other heuristics for game data, raw audio, video, etc. You actually want to avoid single-file compression in the repository so you can take advantage of the differential compression. I can understand that the dynamics of game development teams means that this will never get written in house (although perhaps in the years since I've left, people started placing more emphasis on tools). But for all the entrepreneurs out there, I wouldn't dismiss the market of selling tools to game developers. Perforce is a tiny company that I know made an absolute killing in that market, and I'm somewhat surprised that after a decade they're still the state of the art.

Genuine question: why the obsession with gigabit? What applications demand it?

teddyh 13y ago

What application would demand more than 640k of RAM? Or, to quote: “One day sir, you may tax it.”.

smoyer 13y ago
I had an Apple IIe (with 64KB of RAM) when the PC came out (with 640KB of RAM) and I also once stated "what is anyone going to do with that much RAM?" - It sounds silly now, but it taught me a lesson about automatically rejecting new technologies and 5 years later I was part of the company who was first to send CATV signals over fiber optics.

wmf 13y ago

What I see is an obsession with getting it for no marginal cost. If you're already paying $70/month for 20 Mbps, of course you wouldn't turn down gigabit for the same price (even if Web pages or Netflix won't load any faster).

alecco 13y ago

Vídeos, videoconferencing, uploading data, backups, peer to peer legal/other. Usually the proportions are 8mbit downlink and 100k uplink. Sharing data from nodes is hard.

foobarqux 13y ago

Cloud sync for your entire hard drive. Thin clients. Faster spin up of environments that are sourced from the Internet (virtual machine configuration, bundle install, docker images). High(er) definition video. Multi-party video conferencing.

wmeredith 13y ago

Streaming 3D in 4k would use it. Removing bottlenecks makes things possible.

Amadou 13y ago

TOR Bridge Nodes (relays).

Zigurd 13y ago

4k video

MaulingMonkey 13y ago

Game development. Setting up a new workspace generally is a "go out to lunch and it'll hopefully be mostly done with the perforce checkout by the time you're back" affair on our internal LAN, nevermind if you wanted to work from home. Downloading the built assets through our internal tools takes a similar length of time, so you hopefully can wait until the end of the day to kick that off as you head out. Various SDKs are delivered by download - my last download was 15GB, compressed. These often require frequent updates, which we save to local shares as a simple matter of expediency when shared widely. Delivering complete, self contained milestones in specific formats causing 12-50GB .zips to be uploaded via FTP for hours... with the semi-frequent resort to sneakernet due to quota limitations, service issues, or just plain not having hours between when development for a build was stopped and when the build was needed someplace for a demo or whatever else. I have an archive of some of those deliveries on my local machine. Even with a good 30%+ of my archives missing due to lack of drive space, the archives for the single project I've archives for total over a terrabyte. They're useful for reproducing bugs exactly on occasion. Between all the other projects, other employees, and monstrous shared resources like complete Perforce history, build machines and other servers... complete backup would probably be somewhere around the 0.1 petabytes mark... or close to a full petabit of data. And we're considered only a "small" business, under 50 employees! Fortunately for us backups are fairly static, and the nightly rsync to offsite storage is usually done bringing all our services to a grinding halt by the time work starts the next day. But it's sufficiently slow that those are our secondary backups, with primaries and restoration being done through sneakernet as the rule, not the exception, despite the "beefy" pipe. A second internet backup would push backup times into our workday, and if we're ever forced to do a major restoration through the internet, it's going to be extremely painful... days of waiting with employees unable to do their jobs. Expensive!

chubot 13y ago
Wow I'm surprised game development is still like that. I haven't done game development in almost 9 years, but the "Perforce saturating the LAN" problem sounds very familiar! So I'm actually working on a (binary) software distribution system, and I think it would work very well for this use case. This problem can be solved with software -- it doesn't require an upgrade to fiber. The (straightforward) idea is to use content-addressed storage and differential compression, pretty much like Git does. Then you can actually sync data from your neighbor's machine, in a BitTorrent-ish fashion (possibly getting tiny amounts of metadata from a central server for consistency). Perforce definitely has problems with spurious lock contention, and just a few team members syncing from the same machine can easily clog its pipes if you're not careful. I'm not up to date, but I'm pretty sure you can do better than Perforce's delta compression. You can probably do it with a single generic algorithm, but one way to really improve would be to use file-type specific compression, e.g. bsdiff or Chrome's Courgette for executables, and use other heuristics for game data, raw audio, video, etc. You actually want to avoid single-file compression in the repository so you can take advantage of the differential compression. I can understand that the dynamics of game development teams means that this will never get written in house (although perhaps in the years since I've left, people started placing more emphasis on tools). But for all the entrepreneurs out there, I wouldn't dismiss the market of selling tools to game developers. Perforce is a tiny company that I know made an absolute killing in that market, and I'm somewhat surprised that after a decade they're still the state of the art.