5 ms·
Van Lindberg here from Rackspace. If you have any questions, I am around to answer.
by VanL 13y ago
Van Lindberg here from Rackspace. If you have any questions, I am around to answer.
- vishvananda 13y agoHey Van! How much did you guys pay for them? :p
- jmillikin 13y ago* The ZeroVM site makes a big deal about application execution being completely deterministic. How does this interact with applications that require random numbers, such as crypto? * Is ZeroVM capable of running unmodified Linux binaries? If not, what compiler toolchain is required to get it working? The main advantage of other lightweight virtualization solutions (OpenVZ, LXC) is that it's very easy to take regular binaries (e.g. postgresql) and drop them in a sandbox with minimal fuss.
- pkconstantine 13y ago* You will need to supply it with random seed. * No, you will need to recompile. We use modified gcc/glibc toolchain.
- VanL 13y ago- It is deterministic based on the inputs. You would need to pass in a seed or read from an external source of randomness to get different values out of a PNRG. - Binaries need to be recompiled. There are two toolchains, a GCC-based one and an LLVM-based one. We can also compile within the ZeroVM container itself. We expect that a lot of people will use existing language runtimes (Python, Lua, JS) To avoid compilation. Over the long term, though, a lot of the power comes from composability. Think Unix pipes, in parallel, across the cloud.
- codex 13y agoDoes the hypervisor use multiple communicating CPUs? If so, how do the races inherent in concurrency not destroy the determinism? Is this a single CPU/thread/fiber hypervisor?
- VanL 13y agoEach container has a single process. Each Individual part is deterministic, so the entire system is composable deterministically.
- codex 13y agoThat's not true if the processes communicate, or contain communicating threads. In one run, an input queue looks like {A, B}. In another, {B, A}. The source of the non-determinism is just entropy bubbling up from the hardware. EDIT: using completely synchronous I/O (mentioned below) is a very clever solution, but it requires a process to know its inputs ahead of time. This may also cause cluster scalability issues, as now each "round" of inputs is gated by the slowest of the source processes.
- pkconstantine 13y agoAll reads from other sessions are blocking. There is no input queue, zerovm processes read and write directly to each other. This way determinism can be preserved even for clusters.
- picomancer 13y agoAssuming A and B are produced from separate input processes, they have to be submitted to a "gather" process that takes both of them. If the "gather" process uses select() or non-blocking I/O -- says basically "read from A or B, whichever becomes available first" -- you'll get them gathered in a nondeterminstic order. OTOH if the "gather" process uses synchronous blocking I/O -- "Read one message from A, then read one message from B" -- then you always get (A, B) order. If the framework you're using requires all I/O to be synchronous, and there's no way for a program to tell time or tell when an action would cause a delay, then there's no way for nondeterminism to develop based on timing. I don't have any idea if ZeroVM is like this, and a framework that only allows synchronous I/O would have its own problems (basically you'd have to worry a lot about deadlock). EDIT: To expand on this, you might still be able to do a lot even if cyclic interprocess data flows are forbidden. This is particularly true of database style applications, which are where ZeroVM originated.
- mamcx 13y agoProbably the most obvious if will work only in rackspace or can be used everywhere (and the cost)
- pkconstantine 13y agoIt's open source. https://github.com/zerovm https://github.com/zerovm Can be used anywhere. Can be installed on top of Openstack installation.
- VanL 13y agoWe see this as bigger than just Rackspace. Anywhere your data is, we want ZeroVM to be there.
- retr0h 13y ago... and how much did you buy them for ? :)
- Keyframe 13y agoI've read the architecture page and I am not clear how multithreading is dealt with if application demands it. If you can expand a bit on that.
- pkconstantine 13y agoIf you have a legacy application that demands threads we can emulate threads by essentially using coroutine approach. http://en.wikipedia.org/wiki/Coroutine http://en.wikipedia.org/wiki/Coroutine
- ogrisel 13y agoWhat would be the impact of such a coroutine emulation if the threading is used to leverage multi-core hardware for high performance computing such as done by Atlas [1], OpenBLAS [2] or MKL [3]? These libraries are tuned to maximize CPU cache hits. It seems to me that executing each thread task sequentially using coroutines would probably break such optimizations. [1] http://math-atlas.sourceforge.net/ http://math-atlas.sourceforge.net/ [2] http://www.openblas.net/ http://www.openblas.net/ [3] http://software.intel.com/en-us/intel-mkl http://software.intel.com/en-us/intel-mkl
- vidarh 13y agoFrom the sounds of it, you'd want to split up your workload and run each subset in a separate zerovm
- pkconstantine 13y agoThat's correct. And that's how most of parallel processing frameworks do it anyway.
- IanCal 13y agoWith the potential of starting a ZeroVM in 5ms, and running it for a very short amount of time, do you see yourselves starting to charge in smaller time increments (ms)? I work a lot with smallish data, where queries/processing may take 20 minutes or a few hours. Often I can split this work up significantly, but when most places charge by the hour it's rarely worth it. PiCloud are excellent in this area, and Manta looks interesting (but possibly a bit more expensive) but I'm not aware of many others. I love the idea of being able to start, with little overhead, a large number of short lived jobs. Particularly if I can run them locally or my own cluster. I look forward to seeing more on this.
- pkconstantine 13y agoThe platform itself produces accounting data with 1 ms accuracy. How to do accounting in a real world public service is another question. But we aim for a very short-lived (10 seconds) but widely horizontally spread (1000 machines) workloads.
- IanCal 13y agoThanks for the reply, this sounds like just the kind of thing I'm interested in, and I'm happy to see more competition in the field of renting machines for short periods. I look forward to seeing this released and giving it a go.
- patrickg_zill 13y agoAre there interpreters already available for ZeroVM? Or at the moment, must everything be compiled?
- deleted 13y ago[deleted]
- pkconstantine 13y agoLua, Python and C/C++ are available right now. C/C++ is compiled in run-time with LLVM. Porting an interpreter requires a minimal effort, although porting some of the interpreter libraries can be not that easy. I have compiled PHP just for fun, and it was working "out of the box".
- samstave 13y agoHow much did you acquire them for?