5 ms·

So are we talking about something like a compile-to-native-binary storm / kafka?

spooneybarger 9y ago
The C++ API that we currently aren't supporting and the Go language bindings that we are working on create native binaries; yes. The Python version embeds a Python 2.7 interpreter inside a running application. We have C code that acts as an interface between our Pony runtime and the developer's Python code, calling the required framework functions at the correct time. The event-by-event features in Wallaroo are somewhat analagous to Storm but we had management of in-memory state that allows you to grow the size of the cluster and have the application state redistributed across the cluster. This can be done while an applicaiton is running. In the case of Storm, this would be like: 1- keeping your application state in objects in your bolts (this helps with performance compared to an external store) 2- having a means to add more workers to the running storm topology and have that in bolt state redistributed across the new number of machines. The 15-minute "scale independence" video shows this in action: https://vimeo.com/234753585 https://vimeo.com/234753585 In many ways, Wallaroo looks a lot like the systems that Pat Helland describes in "Beyond Distributed Transactions": http://queue.acm.org/detail.cfm?id=3025012 http://queue.acm.org/detail.cfm?id=3025012 One of our engineers is giving a talk at Velocity conf NY next week. He's going to be discussing Pat's paper and how Wallaroo exhibits a number of characteristics he describes.
- makmanalp 9y ago
  Yes, that paper reference was exactly what I wanted. Thank you!
  - spooneybarger 9y ago
    If you are into papers and talks, we have a few more on the website that we love to turn people on to: http://www.wallaroolabs.com/community/talks http://www.wallaroolabs.com/community/talks
    - makmanalp 9y ago
      Brilliant! I've already read the fault injection one, excited to look at the others!
      - spooneybarger 9y ago
        I love Peter's work. I gave a talk on fault injection and how we use it to test Wallaroo: https://www.monkeysnatchbanana.com/2016/11/22/codemeshio-how-did-i-get-here/ https://www.monkeysnatchbanana.com/2016/11/22/codemeshio-how...

The C++ API that we currently aren't supporting and the Go language bindings that we are working on create native binaries; yes. The Python version embeds a Python 2.7 interpreter inside a running application. We have C code that acts as an interface between our Pony runtime and the developer's Python code, calling the required framework functions at the correct time. The event-by-event features in Wallaroo are somewhat analagous to Storm but we had management of in-memory state that allows you to grow the size of the cluster and have the application state redistributed across the cluster. This can be done while an applicaiton is running. In the case of Storm, this would be like: 1- keeping your application state in objects in your bolts (this helps with performance compared to an external store) 2- having a means to add more workers to the running storm topology and have that in bolt state redistributed across the new number of machines. The 15-minute "scale independence" video shows this in action: https://vimeo.com/234753585 https://vimeo.com/234753585 In many ways, Wallaroo looks a lot like the systems that Pat Helland describes in "Beyond Distributed Transactions": http://queue.acm.org/detail.cfm?id=3025012 http://queue.acm.org/detail.cfm?id=3025012 One of our engineers is giving a talk at Velocity conf NY next week. He's going to be discussing Pat's paper and how Wallaroo exhibits a number of characteristics he describes.

makmanalp 9y ago
Yes, that paper reference was exactly what I wanted. Thank you!
- spooneybarger 9y ago
  If you are into papers and talks, we have a few more on the website that we love to turn people on to: http://www.wallaroolabs.com/community/talks http://www.wallaroolabs.com/community/talks
  - makmanalp 9y ago
    Brilliant! I've already read the fault injection one, excited to look at the others!
    - spooneybarger 9y ago
      I love Peter's work. I gave a talk on fault injection and how we use it to test Wallaroo: https://www.monkeysnatchbanana.com/2016/11/22/codemeshio-how-did-i-get-here/ https://www.monkeysnatchbanana.com/2016/11/22/codemeshio-how...