6 ms·

Some time ago I wanted to use Cap'n Proto in the browser but then I found that the only existing implementation written in JavaScript hadn't been updated in two

by pagnol 9y ago

Some time ago I wanted to use Cap'n Proto in the browser but then I found that the only existing implementation written in JavaScript hadn't been updated in two years and the author himself recommended against its use somewhere in a thread on Stackoverflow. I would love to use Cap'n Proto but for me a robust JS implementation is a sine qua non. Does anyone here happen to know if there's been any progress in this regard or have I missed something?

kentonv 9y ago
There has been some work on a new Javascript implementation, as described in this thread: https://groups.google.com/d/msg/capnproto/lESKRE_pix8/jX59zEUXBgAJ https://groups.google.com/d/msg/capnproto/lESKRE_pix8/jX59zE... But it's been slow. On the other hand, now that 0.6 includes a JSON library, it's relatively easy to do browser<->server in JSON and then use Cap'n Proto on the back-end. But, obviously, it would be nicer to use Cap'n Proto through the whole stack. Contributions are welcome!
kybernetikos 9y ago
I've got about 70% of a capnproto implementation written in pure javascript that works in the browser. Part of the problem is that there is a bit of an ideological difference, where I prefer dynamic code to adding build steps but all the existing tools and code assume you want to generate source. I also found it quite irritating to bootstrap too, because the tooling itself uses capnproto. It sounds neat, but then it means that you have to be able to read capnproto in order to be able to read it. The documentation was also incredibly patchy at the time which made it quite hard to develop for, although I'm told that the kentonv & gang are very helpful. In the end, I was just starting to struggle with capnproto generics when my requirement went away as the server I was connecting to added a msgpack option.
- pagnol 9y ago
  Do you happen to have a repository for this on Github?
  - kybernetikos 9y ago
    It's still very much in a sketch/exploration state (looking at it again, 70% is pretty optimistic :-) so it's unlikely to be much use to anyone who hasn't spent a bunch of time trying to implement it themselves, but if you think there's a chance it might help you then send me your contact details (my email is in my profile) and I'll send it to you.
- kentonv 9y ago
  FWIW, the Python implementation of Cap'n Proto loads schemas dynamically -- there is no code generator. However, this is done through a C extension that calls into the C++ implementation. For browser-side Javascript, that's obviously problematic. (Emscripten isn't really the answer -- would be way too large a download.)
  - TazeTSchnitzel 9y ago
    > Emscripten isn't really the answer -- would be way too large a download. Could the emscripten result be slimmed-down? (I fear the answer is probably “in theory, yes, in practice, it's >100KB”)
artursapek 9y ago
Not to take away from this library's announcement, but on this note I have had a great experience with protobufjs (for vanilla Google protocol buffers)[1]. I use it to both read and write protobuf messages in the browser. Going to protobuf from JSON saved us about 50% on bandwidth for the high volume real time data service that we develop. Love it. [1] https://www.npmjs.com/package/protobufjs https://www.npmjs.com/package/protobufjs
bborud 9y ago
I think the target audience of Cap'n Proto and JSON are almost non-overlapping sets. If you use JS, anything other than JSON is a damn inconvenience. And if you cared about what people who are the target audience of Cap'n Proto care about, you won't be using JS :-)
- paulddraper 9y ago
  > if you cared about what people who are the target audience of Cap'n Proto care about, you won't be using JS What exactly would you be using? Flash? In all seriousness, it's nice to have a format that works for all your clients (browser, iOS, Android, desktop, server).
  - bborud 9y ago
    I don't understand your question.
rubber_duck 9y ago
Check out Google FlatBuffers [1], I haven't used in a while but it had a working JS implementation and better language support for other languages than Cap'n Proto last time I needed something and compared the two. It's just a serialization framework and doesn't bundle RPC/distributed objects like Cap'n Proto - no event loop library or the complexity that comes with this - if you just need a serialization I think it's more approachable. [1] https://github.com/google/flatbuffers https://github.com/google/flatbuffers

by pagnol 9y ago

kentonv 9y ago

There has been some work on a new Javascript implementation, as described in this thread: https://groups.google.com/d/msg/capnproto/lESKRE_pix8/jX59zEUXBgAJ https://groups.google.com/d/msg/capnproto/lESKRE_pix8/jX59zE... But it's been slow. On the other hand, now that 0.6 includes a JSON library, it's relatively easy to do browser<->server in JSON and then use Cap'n Proto on the back-end. But, obviously, it would be nicer to use Cap'n Proto through the whole stack. Contributions are welcome!

kybernetikos 9y ago

I've got about 70% of a capnproto implementation written in pure javascript that works in the browser. Part of the problem is that there is a bit of an ideological difference, where I prefer dynamic code to adding build steps but all the existing tools and code assume you want to generate source. I also found it quite irritating to bootstrap too, because the tooling itself uses capnproto. It sounds neat, but then it means that you have to be able to read capnproto in order to be able to read it. The documentation was also incredibly patchy at the time which made it quite hard to develop for, although I'm told that the kentonv & gang are very helpful. In the end, I was just starting to struggle with capnproto generics when my requirement went away as the server I was connecting to added a msgpack option.

pagnol 9y ago
Do you happen to have a repository for this on Github?
- kybernetikos 9y ago
  It's still very much in a sketch/exploration state (looking at it again, 70% is pretty optimistic :-) so it's unlikely to be much use to anyone who hasn't spent a bunch of time trying to implement it themselves, but if you think there's a chance it might help you then send me your contact details (my email is in my profile) and I'll send it to you.
kentonv 9y ago
FWIW, the Python implementation of Cap'n Proto loads schemas dynamically -- there is no code generator. However, this is done through a C extension that calls into the C++ implementation. For browser-side Javascript, that's obviously problematic. (Emscripten isn't really the answer -- would be way too large a download.)
- TazeTSchnitzel 9y ago
  > Emscripten isn't really the answer -- would be way too large a download. Could the emscripten result be slimmed-down? (I fear the answer is probably “in theory, yes, in practice, it's >100KB”)

artursapek 9y ago

Not to take away from this library's announcement, but on this note I have had a great experience with protobufjs (for vanilla Google protocol buffers)[1]. I use it to both read and write protobuf messages in the browser. Going to protobuf from JSON saved us about 50% on bandwidth for the high volume real time data service that we develop. Love it. [1] https://www.npmjs.com/package/protobufjs https://www.npmjs.com/package/protobufjs

bborud 9y ago

I think the target audience of Cap'n Proto and JSON are almost non-overlapping sets. If you use JS, anything other than JSON is a damn inconvenience. And if you cared about what people who are the target audience of Cap'n Proto care about, you won't be using JS :-)

paulddraper 9y ago
> if you cared about what people who are the target audience of Cap'n Proto care about, you won't be using JS What exactly would you be using? Flash? In all seriousness, it's nice to have a format that works for all your clients (browser, iOS, Android, desktop, server).
- bborud 9y ago
  I don't understand your question.

rubber_duck 9y ago

Check out Google FlatBuffers [1], I haven't used in a while but it had a working JS implementation and better language support for other languages than Cap'n Proto last time I needed something and compared the two. It's just a serialization framework and doesn't bundle RPC/distributed objects like Cap'n Proto - no event loop library or the complexity that comes with this - if you just need a serialization I think it's more approachable. [1] https://github.com/google/flatbuffers https://github.com/google/flatbuffers