3 ms·

That rumor was reportedly a rhetorical question asked by Perlman during the last company meeting. Anybody know why they didn't use a flexible virtual server sol

by fjarlq 14y ago

That rumor was reportedly a rhetorical question asked by Perlman during the last company meeting. Anybody know why they didn't use a flexible virtual server solution like Amazon EC2? "There's no way to exactly estimate how many servers we'd need. So we literally bought thousands of them, and all the equipment and networks to go with it," Perlman told employees. Those servers, he said, came with lengthy contracts -- contracts that tied OnLive's capital up in maintaining servers that few (if any) users were actually using. "If you've got 8,000 servers and 1,600 users, how could we ever get to cash flow positive, right?" -- http://www.joystiq.com/2012/08/18/documenting-the-death-of-onlive-notes-from-the-companys-final/ http://www.joystiq.com/2012/08/18/documenting-the-death-of-o...

phillmv 14y ago
There are EC2 instances with beefy consumer graphics cards? (How do the GPU clusters even work? And, at $2 it seems way more expensive.)
- mbreese 14y ago
  Kinda... there are GPU instances, but they are targetted towards GPU cluster computing for HPC applications. I suspect that GPU clusters work largely the same way that generic/CPU clusters work. You have a cluster manager that dishes jobs out to the worker nodes. The common HPC job scheduler that we use (Torque/PBS) includes support for scheduling jobs that require GPU access. (I assume SGE does as well). Cluster GPU Quadruple Extra Large Instance 22 GB of memory 33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core “Nehalem” architecture) 2 x NVIDIA Tesla “Fermi” M2050 GPUs 1690 GB of instance storage 64-bit platform I/O Performance: Very High (10 Gigabit Ethernet) EBS-Optimized Available: No* API name: cg1.4xlarge
wmf 14y ago
I suspect EC2 is not good for games, since the only GPU they offer is pretty high-end. Also, I think OnLive has more data centers than EC2 to reduce latency.
newhouseb 14y ago
As the other guys here have mentioned, they basically need CDN-level latency virtually everywhere for OnLive to properly work and while Amazon does have a CDN (Cloud Front), their normal datacenters only have a few regions within the United States (which aren't close to any major metropolitan hubs (except maybe DC)). The other big thing is that GPUs have very limited scheduling (i.e. no pre-emptive multitasking), so it's virtually impossible to virtualize into multiple VMs efficiently. So when you get a GPU machine you end up with isolated hardware which doesn't share the same economics as other Amazon solutions. As GPGPU become more popular, this should, however, improve.
- tmurray 14y ago
  You don't need preemptive multitasking for a known set of games, though; context switching can occur at frame boundaries (or even finer-grained than that, the GPU just has to go idle in some sense). It's necessary for GPGPU because you may have one kernel that runs for 10 minutes, but that's not the case with graphics apps (especially not a known set). I also question the need for one VM per client. There's a very small set of inputs (input devices from the client) and a lot of static output (rendered frame). Why not share the same VM for a set of games that share the same GPU?
  - newhouseb 14y ago
    I have no inside knowledge into OnLive, but you're correct, you could certainly context switch for games. I would assume that the limiting factor would likely be texture bandwidth if you were switching between too many different scenes (on the same or multiple VMs). It might be that some of OnLive's IP is smartly managing resources in n->1 VM->GPU configurations (and maybe why games IIRC require some adaptation).
lusr 14y ago
1. Seems a bit strange asking that question AFTER you spend the money. Surely you should ask it before you spend the money. 2. Worst case, at the very least you sell your excess capacity as cloud computing resources so your servers aren't sitting there doing absolutely nothing.

by fjarlq 14y ago

phillmv 14y ago

There are EC2 instances with beefy consumer graphics cards? (How do the GPU clusters even work? And, at $2 it seems way more expensive.)

mbreese 14y ago
Kinda... there are GPU instances, but they are targetted towards GPU cluster computing for HPC applications. I suspect that GPU clusters work largely the same way that generic/CPU clusters work. You have a cluster manager that dishes jobs out to the worker nodes. The common HPC job scheduler that we use (Torque/PBS) includes support for scheduling jobs that require GPU access. (I assume SGE does as well). Cluster GPU Quadruple Extra Large Instance 22 GB of memory 33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core “Nehalem” architecture) 2 x NVIDIA Tesla “Fermi” M2050 GPUs 1690 GB of instance storage 64-bit platform I/O Performance: Very High (10 Gigabit Ethernet) EBS-Optimized Available: No* API name: cg1.4xlarge

wmf 14y ago

I suspect EC2 is not good for games, since the only GPU they offer is pretty high-end. Also, I think OnLive has more data centers than EC2 to reduce latency.

newhouseb 14y ago

As the other guys here have mentioned, they basically need CDN-level latency virtually everywhere for OnLive to properly work and while Amazon does have a CDN (Cloud Front), their normal datacenters only have a few regions within the United States (which aren't close to any major metropolitan hubs (except maybe DC)). The other big thing is that GPUs have very limited scheduling (i.e. no pre-emptive multitasking), so it's virtually impossible to virtualize into multiple VMs efficiently. So when you get a GPU machine you end up with isolated hardware which doesn't share the same economics as other Amazon solutions. As GPGPU become more popular, this should, however, improve.

tmurray 14y ago
You don't need preemptive multitasking for a known set of games, though; context switching can occur at frame boundaries (or even finer-grained than that, the GPU just has to go idle in some sense). It's necessary for GPGPU because you may have one kernel that runs for 10 minutes, but that's not the case with graphics apps (especially not a known set). I also question the need for one VM per client. There's a very small set of inputs (input devices from the client) and a lot of static output (rendered frame). Why not share the same VM for a set of games that share the same GPU?
- newhouseb 14y ago
  I have no inside knowledge into OnLive, but you're correct, you could certainly context switch for games. I would assume that the limiting factor would likely be texture bandwidth if you were switching between too many different scenes (on the same or multiple VMs). It might be that some of OnLive's IP is smartly managing resources in n->1 VM->GPU configurations (and maybe why games IIRC require some adaptation).

lusr 14y ago

1. Seems a bit strange asking that question AFTER you spend the money. Surely you should ask it before you spend the money. 2. Worst case, at the very least you sell your excess capacity as cloud computing resources so your servers aren't sitting there doing absolutely nothing.