6 ms·

Nuclio: Serverless for Real-Time and Data-Driven Applications

71 points by dominotw 9y ago

fest 9y ago
Slightly OT, but I find it interesting how the definition of real-time depends on the industry: * Embedded systems: nanoseconds to milliseconds (depends on the system, but at the very least needs to be predictable within some bounds); * Application software (including web): tens to hundreds of milliseconds; * TV broadcasting: seconds to minutes; I'd love to hear more examples from different industries!
- anfilt 9y ago
  If we want to talk about the minutia. None of these are "real time". Heck even the universe puts a limit on how quickly events can happen. We can look at the light from galaxies so far away that we see the galaxy billions of years into it's past. So nothing can be instantaneous. However, I would say real time generally means acceptable latency or acceptable time reach to consistency. Although, I generally think of real time more along the lines of embedded systems. Moreover, the word server-less really irks me. It's not server-less. You just don't have to manage the server or really interact with the server software that much. It's still a server.
  - RHSman2 9y ago
    server-lite?
- deleted 9y ago
  [deleted]
- jacques_chester 9y ago
  Traditionally, "realtime" means you can tell, in terms of actual wall clock time, how long different operations will take. Tends to show up in embedded systems because they tend to control devices in which fixed deadlines matter. Somehow it morphed into a shorthand for "fast".
  - vvanders 9y ago
    Yup, realtime was always predictable(ex. 16.666ms for 60fps). Time unit largely doesn't matter. Generally precludes GC(yes there are edge cases where you can shoehorn GC in, I don't care about them since malloc is usually out of bounds as well).
- sturadnidge 9y ago
  I always liked the way Bryan Cantrill defined it in one of his wonderful talks, which was basically that the difference between 'hard' real-time systems (eg missile guidance) and 'soft' real-time systems (eg whatsapp) boils down to the consequences of being late.
mrtesthah 9y ago
What is the median latency in milliseconds?
- pwaai 9y ago
  this is what I'm curious about, serverless automatically makes me think of cold startup time.
  - jacques_chester 9y ago
    Some poking around suggests that currently they scale to a single copy, so for sporadic workloads latency will be good (relative to normal latency for warm code). Based on the roadmap in the repo[0] (under "In Design"). Disclosure: I also work on a FaaS project, Project Riff. [0] https://github.com/nuclio/nuclio/blob/master/ROADMAP.md https://github.com/nuclio/nuclio/blob/master/ROADMAP.md
    - yaronhaviv 9y ago
      nuclio auto-scale today (use k8s HPA), the roadmap item is to scale-down to zero (today holds one instance when idle and will soon be able to shut it down when idle). nuclio latency is as low as 30 microsec in Py & Go vs 10s of milisec in Lambda
      - jacques_chester 9y ago
        Warm vs cold start is not a very meaningful comparison, though. Lambda typically scales from zero. Any warm system is going to greatly outperform a cold one. Java on Lambda can vary from nanoseconds to seconds, depending on whether you hit a warm copy or a cold start. Disclosure: I work on a different FaaS, Project Riff.
        yaronhaviv 9y ago
        I benchmarked all the frameworks in cold & warm, Warm is also very slow, no concurrency etc'. how many events/sec can u do with a warm Riff instance e.g. in Go & Py? whats the latency?
keyle 9y ago
I had to look up "Serverless", especially after it says "here is how to start our docker app". I might be old by now, but serverless feels like renting a sublet place per hour. I guess it was only going to happen, after we went from 1 server to many instances to unmanageable mess: "aaeaeh let it be someone else's mess! Take my money!"
- ralusek 9y ago
  "Let it be someone else's mess, take my money," is the basis of most businesses. That's why I don't grow my own food, and I often don't make it.
rburhum 9y ago
> nuclio is an open source serverless platform which is faster than bare-metal code WTF? Can somebody explain to me what they are trying to convey here?
- reilly3000 9y ago
  Like AWS lambda but in a cluster you own.
  - weego 9y ago
    So you take pretty much the single benefit of serverless ie low cost of ownership at low volume and then ditch that and have to run your own cluster? I guess if it was a zero redevelopment migration path from lambda it makes sense.
- SteveNuts 9y ago
  Yeah that's a terrible term to describe what's going on, "bare metal" usually means running directly on an OS that's installed directly to hardware (not going through a hypervisor).
- yaronhaviv 9y ago
  nuclio is using a real-time arch (no context switch, zero copy, no Ser/Des, parallelism, .. ) Go function serves 400K events/sec/proc, Py does 70K events/sec/proc, this is faster than running your own function with standard HTTP serving on a bare-metal server. for comparison most serverless frameworks do 1-5k events/sec and up to 100x higher latency. disclosure: i work on nuclio
  - rburhum 9y ago
    My point is that - as described - it is a dumb comparison. > open source serverless platform which is faster than bare-metal code does't make sense. It eventually has to run on top of bare metal (via hypervisor or directly), so claiming that something that eventually runs on top of bare metal is "faster than bare metal" doesn't make sense at all.
chx 9y ago
Clearly this needs blockchain in the title, it's not meaningless buzzword dense enough otherwise.
jmspring 9y ago
Python and "real time"? Tensorflow as an example in the image/example? Needs more buzzwords, for sure. Developers need to spend more time understanding "real-time" as well.
IanCal 9y ago
I think I'm following along with this and it looks pretty cool. Is there management for handling things with a slow startup time? I've got a few things that have startup times in the region of 30s+ but then each thing to process takes very little time. I found this, which I think is relevant: https://github.com/nuclio/nuclio/blob/master/hack/examples/python/tensorflow/tensor.py https://github.com/nuclio/nuclio/blob/master/hack/examples/p... Are there some pitfalls about this to be wary about / known bad edge cases?
RHSman2 9y ago
Well I think this is frickin awesome. Very Well Done!

71 points by dominotw 9y ago

fest 9y ago

Slightly OT, but I find it interesting how the definition of real-time depends on the industry: * Embedded systems: nanoseconds to milliseconds (depends on the system, but at the very least needs to be predictable within some bounds); * Application software (including web): tens to hundreds of milliseconds; * TV broadcasting: seconds to minutes; I'd love to hear more examples from different industries!

anfilt 9y ago
If we want to talk about the minutia. None of these are "real time". Heck even the universe puts a limit on how quickly events can happen. We can look at the light from galaxies so far away that we see the galaxy billions of years into it's past. So nothing can be instantaneous. However, I would say real time generally means acceptable latency or acceptable time reach to consistency. Although, I generally think of real time more along the lines of embedded systems. Moreover, the word server-less really irks me. It's not server-less. You just don't have to manage the server or really interact with the server software that much. It's still a server.
- RHSman2 9y ago
  server-lite?
deleted 9y ago
[deleted]
jacques_chester 9y ago
Traditionally, "realtime" means you can tell, in terms of actual wall clock time, how long different operations will take. Tends to show up in embedded systems because they tend to control devices in which fixed deadlines matter. Somehow it morphed into a shorthand for "fast".
- vvanders 9y ago
  Yup, realtime was always predictable(ex. 16.666ms for 60fps). Time unit largely doesn't matter. Generally precludes GC(yes there are edge cases where you can shoehorn GC in, I don't care about them since malloc is usually out of bounds as well).
sturadnidge 9y ago
I always liked the way Bryan Cantrill defined it in one of his wonderful talks, which was basically that the difference between 'hard' real-time systems (eg missile guidance) and 'soft' real-time systems (eg whatsapp) boils down to the consequences of being late.

mrtesthah 9y ago

What is the median latency in milliseconds?

pwaai 9y ago
this is what I'm curious about, serverless automatically makes me think of cold startup time.
- jacques_chester 9y ago
  Some poking around suggests that currently they scale to a single copy, so for sporadic workloads latency will be good (relative to normal latency for warm code). Based on the roadmap in the repo[0] (under "In Design"). Disclosure: I also work on a FaaS project, Project Riff. [0] https://github.com/nuclio/nuclio/blob/master/ROADMAP.md https://github.com/nuclio/nuclio/blob/master/ROADMAP.md
  - yaronhaviv 9y ago
    nuclio auto-scale today (use k8s HPA), the roadmap item is to scale-down to zero (today holds one instance when idle and will soon be able to shut it down when idle). nuclio latency is as low as 30 microsec in Py & Go vs 10s of milisec in Lambda
    - jacques_chester 9y ago
      Warm vs cold start is not a very meaningful comparison, though. Lambda typically scales from zero. Any warm system is going to greatly outperform a cold one. Java on Lambda can vary from nanoseconds to seconds, depending on whether you hit a warm copy or a cold start. Disclosure: I work on a different FaaS, Project Riff.
      - yaronhaviv 9y ago
        I benchmarked all the frameworks in cold & warm, Warm is also very slow, no concurrency etc'. how many events/sec can u do with a warm Riff instance e.g. in Go & Py? whats the latency?

keyle 9y ago

I had to look up "Serverless", especially after it says "here is how to start our docker app". I might be old by now, but serverless feels like renting a sublet place per hour. I guess it was only going to happen, after we went from 1 server to many instances to unmanageable mess: "aaeaeh let it be someone else's mess! Take my money!"

ralusek 9y ago
"Let it be someone else's mess, take my money," is the basis of most businesses. That's why I don't grow my own food, and I often don't make it.

rburhum 9y ago

> nuclio is an open source serverless platform which is faster than bare-metal code WTF? Can somebody explain to me what they are trying to convey here?

reilly3000 9y ago
Like AWS lambda but in a cluster you own.
- weego 9y ago
  So you take pretty much the single benefit of serverless ie low cost of ownership at low volume and then ditch that and have to run your own cluster? I guess if it was a zero redevelopment migration path from lambda it makes sense.
SteveNuts 9y ago
Yeah that's a terrible term to describe what's going on, "bare metal" usually means running directly on an OS that's installed directly to hardware (not going through a hypervisor).
yaronhaviv 9y ago
nuclio is using a real-time arch (no context switch, zero copy, no Ser/Des, parallelism, .. ) Go function serves 400K events/sec/proc, Py does 70K events/sec/proc, this is faster than running your own function with standard HTTP serving on a bare-metal server. for comparison most serverless frameworks do 1-5k events/sec and up to 100x higher latency. disclosure: i work on nuclio
- rburhum 9y ago
  My point is that - as described - it is a dumb comparison. > open source serverless platform which is faster than bare-metal code does't make sense. It eventually has to run on top of bare metal (via hypervisor or directly), so claiming that something that eventually runs on top of bare metal is "faster than bare metal" doesn't make sense at all.

chx 9y ago

Clearly this needs blockchain in the title, it's not meaningless buzzword dense enough otherwise.

jmspring 9y ago

Python and "real time"? Tensorflow as an example in the image/example? Needs more buzzwords, for sure. Developers need to spend more time understanding "real-time" as well.

IanCal 9y ago

I think I'm following along with this and it looks pretty cool. Is there management for handling things with a slow startup time? I've got a few things that have startup times in the region of 30s+ but then each thing to process takes very little time. I found this, which I think is relevant: https://github.com/nuclio/nuclio/blob/master/hack/examples/python/tensorflow/tensor.py https://github.com/nuclio/nuclio/blob/master/hack/examples/p... Are there some pitfalls about this to be wary about / known bad edge cases?

RHSman2 9y ago

Well I think this is frickin awesome. Very Well Done!