4 ms·

Sure, it could be a scam. But for what it's worth, it's not - we absolutely didn't cheat. Cheating in a benchmark might work some places, but it certainly doesn

Sure, it could be a scam. But for what it's worth, it's not - we absolutely didn't cheat. Cheating in a benchmark might work some places, but it certainly doesn't work for a company that sells to hackers. If you want to verify the results yourself, we're going to post the benchmarking source code to Github this afternoon. You can review the methodology that CloudHarmony used and run the tests yourself, with your own account, without us having any knowledge that you're testing us. That way, we have no way of cheating. You can even change up the files and use different encoding profiles to be sure that we didn't pick specific files or settings that happen to be faster at Zencoder.

sologoub 15y ago
Would be interesting if someone independent could corroborate the results. The sponsorship disclosure is appreciated, but I agree with the original comment - raises red flags.
- brandonarbini 15y ago
  Here's the source code that anyone can use to independently test and corroborate the results: https://github.com/zencoder/cloudharmony-benchmark https://github.com/zencoder/cloudharmony-benchmark. We chose Cloud Harmony to run the benchmark on the basis of their independent reputation. However, we'd happily sponsor (pay for encoding fees) another third-party to run the tests as well. Feel free to suggest an additional, neutral person/company that could do this. Though the results will be the same, no matter who runs it. :)
timc3 15y ago
And how did you measure the quality of the outputted file? This matters a huge amount, it doesn't matter if you are 4 times as quick if the output is not up to the standards needed by the client.
- jon_dahl 15y ago
  Definitely - this is important as a control. Every test was done using the same core settings (codecs, profiles, resolution, frame rate, bitrates, two-pass encoding). Beyond that, CloudHarmony used the default settings at each provider. We'll try to publish each video for visual inspection, but suffice it to say that they all look pretty close, especially three of the four, which use the same core encoder (x264) and the same settings. There may be microversion differences and minor variations in settings, but not much. The fourth uses MainConcept, which isn't a bad encoder itself. You'll probably see a bigger spread in audio quality - Encoding.com and Panda rely on faac, I think, which is an abandoned AAC encoder that isn't very good.
zimbatm 15y ago
Thanks for taking the time to do these tests (@panda here). I checked your benchmarks and I can confirm that they are legit on our side. To be fair our customers are not the same type. We mainly have people who trans-code a lot of videos (like DVD libraries) and in that case we are a lot cheaper than zencoder. Our presets also use a much more recent version of ffmpeg since we are not stuck with command-line compatibility issues

sologoub 15y ago

Would be interesting if someone independent could corroborate the results. The sponsorship disclosure is appreciated, but I agree with the original comment - raises red flags.

brandonarbini 15y ago
Here's the source code that anyone can use to independently test and corroborate the results: https://github.com/zencoder/cloudharmony-benchmark https://github.com/zencoder/cloudharmony-benchmark. We chose Cloud Harmony to run the benchmark on the basis of their independent reputation. However, we'd happily sponsor (pay for encoding fees) another third-party to run the tests as well. Feel free to suggest an additional, neutral person/company that could do this. Though the results will be the same, no matter who runs it. :)

timc3 15y ago

And how did you measure the quality of the outputted file? This matters a huge amount, it doesn't matter if you are 4 times as quick if the output is not up to the standards needed by the client.

jon_dahl 15y ago
Definitely - this is important as a control. Every test was done using the same core settings (codecs, profiles, resolution, frame rate, bitrates, two-pass encoding). Beyond that, CloudHarmony used the default settings at each provider. We'll try to publish each video for visual inspection, but suffice it to say that they all look pretty close, especially three of the four, which use the same core encoder (x264) and the same settings. There may be microversion differences and minor variations in settings, but not much. The fourth uses MainConcept, which isn't a bad encoder itself. You'll probably see a bigger spread in audio quality - Encoding.com and Panda rely on faac, I think, which is an abandoned AAC encoder that isn't very good.

zimbatm 15y ago

Thanks for taking the time to do these tests (@panda here). I checked your benchmarks and I can confirm that they are legit on our side. To be fair our customers are not the same type. We mainly have people who trans-code a lot of videos (like DVD libraries) and in that case we are a lot cheaper than zencoder. Our presets also use a much more recent version of ffmpeg since we are not stuck with command-line compatibility issues