6 ms·
Google Compute Engine vs. Amazon EC2: Video Transcoding
- sp332 14y agoHow could transfers from Google to Amazon be so much faster than from Google to Google?
- jbeda 14y agoI'm the TL for Google Compute Engine. This isn't the performance we expect here and we will be looking in to see what is going on.
- sp332 14y agoOh I'm sure it's not normal. Let us know if you find out what happened :)
- rdl 14y agoHow do people get beta access? I'd love to look at moving workloads over to GCE.
- jbeda 14y agoSign up and let us know what you are looking to do. We are trickling out invites but being careful that we can deliver a great experience (support wise and otherwise) to everyone.
- Bjartr 14y agoA google product with a great support experience? This I gotta see.
- mythz 14y agoLike most things in life, you get better support when you pay for it.
- rdl 14y agoHonestly I've always gotten good support for Google products, either from friends at Google or through forums, since they are popular enough to have lots of users. Of course, I remember when Google was basically the only hiring destination for top people in the valley (2001-2004 or so), and I only narrowly avoided going to google myself. I suspect they will do a decent job of supporting GCE, either through a premium offering themselves, or through third party developers -- same thing AWS has done. I had some meetings with AWS application/security platform consultants recently, and they do a really good job of it, you just have to pay for it. This isn't a good argument for consumer products (if a random grandmother gets locked out of her gmail, and doesn't know anyone, she may be doomed), but I suspect any developer building products on GCE either knows someone or can pay for support, or can bitch in a high visibility forum and get help from third parties or by Google.
- boomzilla 14y agobummer :) don't you guys run perf. test with your biggest competitor before launch?
- tunnuz 14y agoThey are in closed beta.
- wavephorm 14y agoGoogle's infrastructure wasn't build to be used 3rd parties, and it shows. It is lipstick on a pig, and Google is absolutely looking like a dinosaur in the IaaS cloud computing world.
- jcdavis 14y agoIts not a fair comparison, but shouldn't they also be trying/using EC2 GPU instances? I would think a large transcoding service such as zencoder has at least looked into it at some point
- sp332 14y agoThey've looked into it, but they said GPU encoders don't compress as well, and don't look as good. https://www.quora.com/Does-Zencoder-use-regular-EC2-or-GPU-EC2-instances-from-Amazon https://www.quora.com/Does-Zencoder-use-regular-EC2-or-GPU-E...
- salem 14y agoMuch of the SFX film industry is not using GPUs, at least not for final renders because differences floating point precision.
- dpe82 14y agoFloating point precision isn't the problem - when your output is 8 or 10 bit resolution, floating point differences aren't a big deal. The biggest blocking factor is amount of code that needs to be ported to make it work. High end SFX companies spend virtually all their man hours implementing new stuff. They don't have the time to go back and reimplement everything, and their customers aren't so cost sensitive that they demand it.
- salem 14y agoWell, I know of at least one very large SFX shop that skipped GPUs for now because the results were not consistent with results from the host CPUs. But you're correct, man-hours to rewrite are more expensive than the CPU time in that application.
- wmf 14y agoDoes 8 cores mean 8 cores or 4 cores/8 threads? The documentation is confusing.
- ChuckMcM 14y agoNot to worry it is confusing for everyone. My experience when drilling down on these things are that generally in marketing advertising literature a 'core' is what most people would call a 'thread.' Engineering documentation is usually more precise but when its perceived value, perception take priority over precision.
- jbeda 14y agoI'd love to hear what people think the standard nomenclature here is. I've been arguing with our PM as to call these things virtual cores or virtual CPUs. Our API uses guestCpus. This stuff is very confusing and I'd love to find the right way to communicate it clearly.
- ChuckMcM 14y agoPart of the challenge is that Intel and AMD don't do you any favors. In the ideal world for a set of cores C if I run a piece of independent, memory bound, code on all of them I get O(C) scale. Generally that doesn't hold in shared L2/L3 cores or where HyperThreads are counted as cores. We can define a scaling factor Cf such that 0 < Cf < 1.0 independent code scales at O(Cf). In one school you could price out at cores such that 1/Cf physical cores were assigned per Core purchased (rounded up to the nearest unit of computation) but that doesn't begin to get into the question of shared memory vs shared network vs shared disk. I think the best you can do for now is call it a core if Cf is > .95.
- ezran 14y agoIn this case it means 8 cores/16 threads. Logging into a cc2.8xlarge instance shows 32 threads.
- wmf 14y ago
- dpe82 14y agoGoogle cache: http://webcache.googleusercontent.com/search?q=cache:http://blog.zencoder.com/2012/07/23/first-look-at-google-compute-engine-for-video-transcoding/&hl=en&prmd=imvns&strip=1 http://webcache.googleusercontent.com/search?q=cache:http://...
- Heff 14y agoThe actual page should be performing better now.
- dmv 14y agoThis is nice, simple experiment design. However, I'm really surprised they only ran this once - or indicate trial counts and indicate range/distribution. I'd be curious if video transcoding is so consistent that a single measurement is enough to draw a conclusion; certainly network/storage transfer is not. Sure, time and bandwidth are not free...
- jon_dahl 14y agoHi DMV - we ran the transcoding tests a few times, but transcoding performance is pretty steady across multiple runs (here and elsewhere). Network obviously isn't; the numbers here include about a dozen test runs. We should make that more clear. Even a dozen isn't enough to be a scientific test, so hopefully we (or someone else) will do more benchmarking in the future.
- ww520 14y agoThe comparison on cost is again on the EC2 on-demand pricing. Reserved pricing or spot pricing would cut that down substantially.
- deleted 14y ago[deleted]
- radarsat1 14y agobah, measurements without standard deviations are next to useless...