6 ms·

Show HN: Kickflip – Open Source Live Mobile Video Streaming SDKs

39 points by Mizza 12y ago

Mizza 12y ago
Hey guys! Rich from Kickflip here. We've been working really hard on this for a long time and are super happy to get to announce it today. Kickflip is some really cool tech - we've lowered the cost of mobile video broadcasting by orders of magnitude, and we've opened it up to everybody! If you've got any questions or feedback, let me know and I'll be happy to answer them. Thanks very much!
- srean 12y ago
  Glad that someone is doing this. The last time I looked at it, it was hard to do hardware encoded video (out)streaming well on the iphone. The open API only allowed writing the compressed stream to a file. Pipelining or FIFO tricks did not work well because the file was not in a consistent state untill it had been closed. It appeared that the OS was not flushing key metadata until then. So the options seemed to be either have a quick cycle of starting and stopping recording on a ring buffer of files, this would drop frames; or parse and fix (with some second guessing) the compressed stream in the application. The first one was easier to implement but quality wasn't great, although software interpolation of the dropped frames might help some. The second seemed too much work. Has the scenario on the open API changed ? I would guess not. Congrats for taking care of this.
  - chrisballinger 12y ago
    Unfortunately Apple still doesn't expose the APIs that we need and are forced to use tricks to get the encoded frames. We based our hardware encoder wrapper on the technique described by Geraint Davies [1], but have modified and improved upon his method. The main trick is to encode only video frames in your temporary .mp4 file, and to make an additional 1 frame .mp4 file for extracting the avcC record. 1. http://www.gdcl.co.uk/2013/02/20/iOS-Video-Encoding.html http://www.gdcl.co.uk/2013/02/20/iOS-Video-Encoding.html
elwell 12y ago
That's actually a double 720-flip, not a kickflip: https://kickflip.io/ https://kickflip.io/
- Mizza 12y ago
  Well, that's because we're the only live video broadcasting SDK which can do 720p! (Ok, it's actually just because it's an awesome trick. The 720p thing is just a coincidence.)
elwell 12y ago
"python client source code" link is dead: https://github.com/kickflip/kickflip-docs#api https://github.com/kickflip/kickflip-docs#api
- Mizza 12y ago
  Nice catch! :) The python library and client is still work in progress. Still, I've open sourced the repo so those who are curious can take a peek, although there isn't much worth showing there yet. We'll make a larger announcement when that happens.. there's a lot of stuff we've got planned for the desktop yet. Here's the fixed link: https://github.com/Kickflip/python-kickflip https://github.com/Kickflip/python-kickflip
elwell 12y ago
Besides using FFmpeg, are any novel optimizations/compression being offered?
- Mizza 12y ago
  We actually don't use FFMpeg to do the compression, we use the hardware encoders, so it uses much less CPU. That's how we're able to broadcast at a much better resolution. It's also bandwidth adaptive streaming, backed by a global CDN, and generates both Live and VOD HLS streams. But, more importantly, we've designed it in such a way that as a developer, you don't even have to worry about that! The library just exposes a really simple API that does all of the video magic for you.
  - yarri 12y ago
    Nice work! What are the min hardware requirements for iOS & Android devices?
    - Mizza 12y ago
      Thanks! Kickflip currently requires iOS 7 and works on any iOS device which supports iOS 7. The Android requirements are a bit steeper, unfortunately: Android needs at least 4.3, as that's when the hardware encoder access API which we use was introduced.
    - deleted 12y ago
      [deleted]
jgh 12y ago
Very cool! I've recently open sourced a good portion of my game broadcasting SDK, which now has a few people using it for purely making camera apps to stream to RTMP services. Mine is not nearly as refined as this one, nor do I have any sort of CDN heh. Congrats guys, looks awesome.
- Mizza 12y ago
  Thanks man! I really appreciate it. All of our streams are HLS rather than RTMP, although we do have RTMP support working in the SDKs, we don't plan on making that a priority product focus right now - HLS and DASH are the future. Kickflip will likely officially support RTMP in the 1.1 versions of the SDKs. Thanks again for your support!
- srean 12y ago
  Hey https://github.com/jamesghurley/VideoCore https://github.com/jamesghurley/VideoCore looks great, wish I knew about it earlier.
  - jgh 12y ago
    Thanks! It's a side-project but I try to fix any bugs as quickly as I can and I've got some new features planned/in development for the future.
- chrisballinger 12y ago
  Wow, if only I knew about VideoCore earlier! I used a modified version of Geraint Davies temp .mp4 file hack, but it looks like you developed a great alternative.
  - jgh 12y ago
    I've never actually seen this code before. I find it interesting that he found that the frames were arriving 500ms after capture, but I've used a similar technique (to how I did it in VideoCore) for video conferencing that achieved much, much lower latency than that. I wonder if that's something that was true in 2010 but is no longer true. I also haven't really encountered much of an issue with frame reordering, but as it is I mostly don't care about that. I have noticed, however, that as of iOS 7.0.4 or so the encoder can output multiple slices per frame which it previously didn't do (or at least I never caught it doing it anyway)
lucasgonze 12y ago
Transcoding on the client instead of in the cloud increases the amount of CPU and upstream bandwidth needed, and both are in short supply on phones. To put this in perspective, three live encodes/ streams (360P, 540P, 720P) using Wirecast on a high end Macbook Pro pegs the CPU around 60% and requires at least 5MBPS upstream. I understand the financial reasons for this architecture - cloud-based transcoders are crazy expensive - but from a customer satisfaction POV it's not an easy project.
- jgh 12y ago
  Encoding is done in hardware on the client side and doesn't increase CPU load by very much. I'm using the same trick in a library I made to do game broadcasting from iOS and honestly network traffic increases CPU load more than encoding an 1136x640 video does.
  - lucasgonze 12y ago
    Can you be more specific by "in hardware"? I'll buy that there's some trick analogous to shaders on the desktop. But I'd want to know the specifics to be able to evaluate that. How many encodes have you been able to do, anyway? Encoding and upstreaming a single 640p video does seem plausible. But for each additional encode you need HLS and RTMP, and both CPU and upstream bandwidth grow linearly with each encode. A single bitrate means HLS and RTMP - two encodes, two upstreams. (Only HLS means only Apple software - 20% of the mobile market, 5% of the desktop market).
    - jgh 12y ago
      Why should you have to encode twice for HLS and RTMP? The only reason to encode twice or more would be if you wish to have different movie properties (i.e. bitrate, fps, resolution, etc) For example, for h.264, once you get the NAL units from the encoder (which in the case of iOS is a chip provided by PowerVR) you can encapsulate them in whatever format you need...mpeg-ts, MP4, RTMP, RTP, it doesn't matter... I only support one simultaneous encode in my SDK, and I'm not actually sure how many the hardware would support as an upper limit.
seshakiran 12y ago
Very cool. This could be the holygrail for live streaming. Unfortunately Qik was retired. Good job.
ldarcyftw 12y ago
Can you share your experience of the typical delay for CDNed live stream? 5 sec? 10 sec?
- Mizza 12y ago
  We found that Bambuser had a delay of around ~60s - ours is 10-20s, depending on the buffer size. It can be brought down to 5-10 by shortening the chunk size, but that increases the overhead somewhat. However, we're experimenting with this all the time, and optimizing based on the needs of our users, so hit us up via email if you have any specific requirements. It is worth nothing that this is definitely a broadcast-first technology (at least for now), meaning that it's designed for delivering a high quality stream which will be seen by a large audience, rather than a low quality stream to an indivudual audience with a short amount of latency.
oDot 12y ago
Can you share the costs of streaming to 100Ks clients?
- chrisballinger 12y ago
  What do you mean by "share the costs"? Our pricing plans are based on usage tiers: https://kickflip.io/pricing https://kickflip.io/pricing
Akkuma 12y ago
Is the SDK going to allow a way to bypass all of the Kickflip infrastructure to allow the core streaming library to be used on your own maintained infrastructure? On the product I work on, we already have our own house built media server and rtmp restreamer. As to not supporting RTMP, in our experience RTMP had much lower latency, which resulted in us switching away from HLS wherever possible.
- chrisballinger 12y ago
  RTMP support is on our roadmap (and actually exists inside the SDK in a disabled state), and our live transcoding backend will have the ability to restream to any arbitrary RTMP endpoint in addition to the other outputs.
  - Akkuma 12y ago
    That's good to hear. Is the plumbing the "cost" of using the open source sdk, which in turn technically requires you pay for the service, or is there a way to remove all of the kickflip infrastructure plumbing and rely on our own? I don't actually work on the mobile streaming side of things, I work on the web side, particularly the video player.
angryasian 12y ago
is this some sort of hosted service and if so where is the pricing ?
- chrisballinger 12y ago
  Yep, we manage all of the infrastructure so you don't have to. We should make our pricing more up-front but here is a direct link: https://kickflip.io/pricing https://kickflip.io/pricing
  - angryasian 12y ago
    requires me to sign in before viewing the pricing ??
spleeder 12y ago
I created an account but when trying to create an app it redirects to a blank page.

39 points by Mizza 12y ago

Mizza 12y ago

Hey guys! Rich from Kickflip here. We've been working really hard on this for a long time and are super happy to get to announce it today. Kickflip is some really cool tech - we've lowered the cost of mobile video broadcasting by orders of magnitude, and we've opened it up to everybody! If you've got any questions or feedback, let me know and I'll be happy to answer them. Thanks very much!

srean 12y ago
Glad that someone is doing this. The last time I looked at it, it was hard to do hardware encoded video (out)streaming well on the iphone. The open API only allowed writing the compressed stream to a file. Pipelining or FIFO tricks did not work well because the file was not in a consistent state untill it had been closed. It appeared that the OS was not flushing key metadata until then. So the options seemed to be either have a quick cycle of starting and stopping recording on a ring buffer of files, this would drop frames; or parse and fix (with some second guessing) the compressed stream in the application. The first one was easier to implement but quality wasn't great, although software interpolation of the dropped frames might help some. The second seemed too much work. Has the scenario on the open API changed ? I would guess not. Congrats for taking care of this.
- chrisballinger 12y ago
  Unfortunately Apple still doesn't expose the APIs that we need and are forced to use tricks to get the encoded frames. We based our hardware encoder wrapper on the technique described by Geraint Davies [1], but have modified and improved upon his method. The main trick is to encode only video frames in your temporary .mp4 file, and to make an additional 1 frame .mp4 file for extracting the avcC record. 1. http://www.gdcl.co.uk/2013/02/20/iOS-Video-Encoding.html http://www.gdcl.co.uk/2013/02/20/iOS-Video-Encoding.html

elwell 12y ago

That's actually a double 720-flip, not a kickflip: https://kickflip.io/ https://kickflip.io/

Mizza 12y ago
Well, that's because we're the only live video broadcasting SDK which can do 720p! (Ok, it's actually just because it's an awesome trick. The 720p thing is just a coincidence.)

elwell 12y ago

"python client source code" link is dead: https://github.com/kickflip/kickflip-docs#api https://github.com/kickflip/kickflip-docs#api

Mizza 12y ago
Nice catch! :) The python library and client is still work in progress. Still, I've open sourced the repo so those who are curious can take a peek, although there isn't much worth showing there yet. We'll make a larger announcement when that happens.. there's a lot of stuff we've got planned for the desktop yet. Here's the fixed link: https://github.com/Kickflip/python-kickflip https://github.com/Kickflip/python-kickflip

elwell 12y ago

Besides using FFmpeg, are any novel optimizations/compression being offered?

Mizza 12y ago
We actually don't use FFMpeg to do the compression, we use the hardware encoders, so it uses much less CPU. That's how we're able to broadcast at a much better resolution. It's also bandwidth adaptive streaming, backed by a global CDN, and generates both Live and VOD HLS streams. But, more importantly, we've designed it in such a way that as a developer, you don't even have to worry about that! The library just exposes a really simple API that does all of the video magic for you.
- yarri 12y ago
  Nice work! What are the min hardware requirements for iOS & Android devices?
  - Mizza 12y ago
    Thanks! Kickflip currently requires iOS 7 and works on any iOS device which supports iOS 7. The Android requirements are a bit steeper, unfortunately: Android needs at least 4.3, as that's when the hardware encoder access API which we use was introduced.
  - deleted 12y ago
    [deleted]

jgh 12y ago

Very cool! I've recently open sourced a good portion of my game broadcasting SDK, which now has a few people using it for purely making camera apps to stream to RTMP services. Mine is not nearly as refined as this one, nor do I have any sort of CDN heh. Congrats guys, looks awesome.

Mizza 12y ago
Thanks man! I really appreciate it. All of our streams are HLS rather than RTMP, although we do have RTMP support working in the SDKs, we don't plan on making that a priority product focus right now - HLS and DASH are the future. Kickflip will likely officially support RTMP in the 1.1 versions of the SDKs. Thanks again for your support!
srean 12y ago
Hey https://github.com/jamesghurley/VideoCore https://github.com/jamesghurley/VideoCore looks great, wish I knew about it earlier.
- jgh 12y ago
  Thanks! It's a side-project but I try to fix any bugs as quickly as I can and I've got some new features planned/in development for the future.
chrisballinger 12y ago
Wow, if only I knew about VideoCore earlier! I used a modified version of Geraint Davies temp .mp4 file hack, but it looks like you developed a great alternative.
- jgh 12y ago
  I've never actually seen this code before. I find it interesting that he found that the frames were arriving 500ms after capture, but I've used a similar technique (to how I did it in VideoCore) for video conferencing that achieved much, much lower latency than that. I wonder if that's something that was true in 2010 but is no longer true. I also haven't really encountered much of an issue with frame reordering, but as it is I mostly don't care about that. I have noticed, however, that as of iOS 7.0.4 or so the encoder can output multiple slices per frame which it previously didn't do (or at least I never caught it doing it anyway)

lucasgonze 12y ago

Transcoding on the client instead of in the cloud increases the amount of CPU and upstream bandwidth needed, and both are in short supply on phones. To put this in perspective, three live encodes/ streams (360P, 540P, 720P) using Wirecast on a high end Macbook Pro pegs the CPU around 60% and requires at least 5MBPS upstream. I understand the financial reasons for this architecture - cloud-based transcoders are crazy expensive - but from a customer satisfaction POV it's not an easy project.

jgh 12y ago
Encoding is done in hardware on the client side and doesn't increase CPU load by very much. I'm using the same trick in a library I made to do game broadcasting from iOS and honestly network traffic increases CPU load more than encoding an 1136x640 video does.
- lucasgonze 12y ago
  Can you be more specific by "in hardware"? I'll buy that there's some trick analogous to shaders on the desktop. But I'd want to know the specifics to be able to evaluate that. How many encodes have you been able to do, anyway? Encoding and upstreaming a single 640p video does seem plausible. But for each additional encode you need HLS and RTMP, and both CPU and upstream bandwidth grow linearly with each encode. A single bitrate means HLS and RTMP - two encodes, two upstreams. (Only HLS means only Apple software - 20% of the mobile market, 5% of the desktop market).
  - jgh 12y ago
    Why should you have to encode twice for HLS and RTMP? The only reason to encode twice or more would be if you wish to have different movie properties (i.e. bitrate, fps, resolution, etc) For example, for h.264, once you get the NAL units from the encoder (which in the case of iOS is a chip provided by PowerVR) you can encapsulate them in whatever format you need...mpeg-ts, MP4, RTMP, RTP, it doesn't matter... I only support one simultaneous encode in my SDK, and I'm not actually sure how many the hardware would support as an upper limit.

seshakiran 12y ago

Very cool. This could be the holygrail for live streaming. Unfortunately Qik was retired. Good job.

ldarcyftw 12y ago

Can you share your experience of the typical delay for CDNed live stream? 5 sec? 10 sec?

Mizza 12y ago
We found that Bambuser had a delay of around ~60s - ours is 10-20s, depending on the buffer size. It can be brought down to 5-10 by shortening the chunk size, but that increases the overhead somewhat. However, we're experimenting with this all the time, and optimizing based on the needs of our users, so hit us up via email if you have any specific requirements. It is worth nothing that this is definitely a broadcast-first technology (at least for now), meaning that it's designed for delivering a high quality stream which will be seen by a large audience, rather than a low quality stream to an indivudual audience with a short amount of latency.

oDot 12y ago

Can you share the costs of streaming to 100Ks clients?

chrisballinger 12y ago
What do you mean by "share the costs"? Our pricing plans are based on usage tiers: https://kickflip.io/pricing https://kickflip.io/pricing

Akkuma 12y ago

Is the SDK going to allow a way to bypass all of the Kickflip infrastructure to allow the core streaming library to be used on your own maintained infrastructure? On the product I work on, we already have our own house built media server and rtmp restreamer. As to not supporting RTMP, in our experience RTMP had much lower latency, which resulted in us switching away from HLS wherever possible.

chrisballinger 12y ago
RTMP support is on our roadmap (and actually exists inside the SDK in a disabled state), and our live transcoding backend will have the ability to restream to any arbitrary RTMP endpoint in addition to the other outputs.
- Akkuma 12y ago
  That's good to hear. Is the plumbing the "cost" of using the open source sdk, which in turn technically requires you pay for the service, or is there a way to remove all of the kickflip infrastructure plumbing and rely on our own? I don't actually work on the mobile streaming side of things, I work on the web side, particularly the video player.

angryasian 12y ago

is this some sort of hosted service and if so where is the pricing ?

chrisballinger 12y ago
Yep, we manage all of the infrastructure so you don't have to. We should make our pricing more up-front but here is a direct link: https://kickflip.io/pricing https://kickflip.io/pricing
- angryasian 12y ago
  requires me to sign in before viewing the pricing ??

spleeder 12y ago

I created an account but when trying to create an app it redirects to a blank page.