5 ms·
As you can probably tell, we are super excited about this, feel free to ask me anything : ).
by felixge 16y ago
As you can probably tell, we are super excited about this, feel free to ask me anything : ).
- zdw 16y agoThis is really neat, but since you said "anything"... I have a legal question that I haven't been able to find the answer for: How do you deal with the licensing issues regarding open source encoding software? Do you pay the MPEG-LA a fee directly for use of the software? Is it per file/per minute/flat fee? Just wondering about the mechanics of this, mainly for an in-house streaming application.
- felixge 16y agoNo, we don't pay the MPEG-LA (yet). The information they provide has given us reason to believe that we don't need a license at this point.
- jon_dahl 16y agoIANAL, but generally, you need a license if you're using open-source encoders or decoders and you're the one who actually compiles them. (Again, not a lawyer, but that changes ffmpeg from "a source code description of an encoder" to "an encoder.") MPEG video stuff is generally pretty cheap or free for low volume. Audio codecs are fairly expensive, on the other hand. AAC + MP3 + AMR is >$20K in minimum fees.
- dantheman 16y agoLooks great. Does this do encoding of streaming video or only files?
- felixge 16y agoIt only encodes uploaded files.
- maxklein 16y agoHow many ffmpeg instances are you holding in memory? You have a single ffmpeg process for each upload that is happening, do you? Additionally, if I am uploading files with B-frames and GOPs of 300 (which I assume will lead to a lot of frames being kept in memory, not sure about this), is your RAM requirement not going to be massive if you have several uploads happening at the same time? And considering the length of time a single video takes, it's reasonably likely you'll have a lot running in parallel, is it not?
- felixge 16y agoWe have one ffmpeg process for each upload, yes. From our tests with the kind of videos we mostly deal with (user generated content), this seem to work quite well while only requiring reasonable memory allocations. So sure, there could be problems we don't know about yet, but that's the price for doing something nobody has done before. We will deal with these problems as they come up. In the worst case that might mean falling back to normal encoding if a ffmpeg child process starts to misbehave.
- maxklein 16y agoSo you're using plain command line ffmpeg or are you feeding the frames yourself? How do you throttle the speed of ffmpeg?
- felixge 16y agoWe are feeding raw bytes into a command line ffmpeg process via standard input. Ffmpeg only reads as much from stdin as it can process. Node.js gives us a 'drain' event when this happend, which means we can pump more data into it. So we are no throttling the speed, just the flow of data.
- app 16y agoIs this limited to 1-pass encoding only?
- felixge 16y agoyes, 2-pass encoding will not be doable in realtime.
- Kaizyn 16y agoHow will this cool feature help your startup make money? How much cost in time and effort did it take to add the feature?
- felixge 16y ago1. We are infinitely [1] faster at converting video uploads than our competitors now. Speed matters. This is a strong USP. 2. Publicity. There are a bunch of features that would certainly be equally useful to our customers. However, none of them would get much interest from hacker news. We got a lot of signups and page views from this. 3. It took me probably 4 full days [2] to implement this, including the screencast and blog post. That's reasonable for the value added by this feature. --fg [1] We are really infinitely faster if the upload speed is slower than our encoding speed. This is probably the case for 99% of all consumer internet connections at this point. In any case we reduced the time required for an encoding from SUM(Upload, encoding) to MAX(Upload, Encoding). [2] We already made a lot of technology decisions that lowered the cost of implementing this. A competitor would certainly need much more resources to pull this off.
- JonnieCache 16y agoWhat have you done about the security implications of pushing user-provided data into ffmpeg straight off the wire? How worried about this are you? Obviously ffmpeg was not built with untrusted input in mind. I have only paranoia as a justification, but I bet I could make it crash pretty hard, and with a bit more work I bet I could design an input that caused ffmpeg to just fill your disk forever. How much work have you had to do around these issues? Did they prove to be bigger or smaller than you anticipated?
- nitrogen 16y agoI'd bet that ulimit, quota, and chroot would go a long way to solving those issues.
- badmash69 16y agoAs a curious outsider, I have to ask why are you giving away the recipe to your secret sauce ? I mean, there are several of your competitors such as zencoder , etc. that could easily copy this idea and implement it . So why would you blog about implementation know-how that made your product cooler and enabled you to command premium pricing ?
- siculars 16y agoI don't really think it matters for a number of reasons. Chief among them being motivation. If you are motivated enough to copy this idea then you would reverse engineer it as soon as you heard what they are doing without knowing exactly how, which is what I did knowing that they were based on nodejs and I know how nodejs handles buffers. Ultimately, if you are a provider of services, these or any other, you will add functionality or run the risk of being eclipsed.
- felixge 16y agoWell, anybody in the video encoding business will know how we did it. It's kind of painfully obvious. So it's not so much spilling the secret sauce. Also, we don't believe individual features themselves will give us a long term advantage. Consistently being a leader of innovation however, will.