3 ms·

Hmm, I thought that was what I was answering. Maybe I'm still misunderstanding. You're asking if Cap'n Proto's advantage over something like Protobufs will be l

by kentonv 12y ago

Hmm, I thought that was what I was answering. Maybe I'm still misunderstanding. You're asking if Cap'n Proto's advantage over something like Protobufs will be less pronounced in a dynamic language compared to C++? Yes, that is likely the case, due to one or both of the inlining issue and the the language's general slowness dwarfing any gains from the encoding library. Of course, in cases where Cap'n Proto has a more-than-constant speedup, such as reading a single field from a large message (O(1) in Cap'n Proto, O(n) in Protobufs), then the difference will still be huge regardless of language. If you're looking for specific benchmark numbers, I don't have any handy, sorry. (But benchmarks can be manipulated to show any result, so you shouldn't trust any author-provided numbers anyway.)

sandGorgon 12y ago
@kentonv - yes that is what I was asking and thank you for asking. I think one aspect of my question got lost in the noise and it is my fault. On Python, protobuf vs capnproto is not apples to apples, since the former is pure python ... while yours is python wrapper over C. I have read your justifications [1] and I agree with you. But do note that there are some large usecases for Python on desktop software. C-extensions turn to be blockers in those cases. In many ways, I was hoping that you would have a pure-python version as well (since you did build one at Google) which sacrifices speed for compatibility. I was thinking in that context. [1] http://kentonv.github.io/capnproto/news/2013-09-04-capnproto-0.3-python-tools-features.html http://kentonv.github.io/capnproto/news/2013-09-04-capnproto...
- kentonv 12y ago
  It would be great if someone were to contribute a pure-Python implementation, but it's unlikely the sandstorm.io team will work on this since it has no real use to us. I actually think it's likely that a pure-Python version of Cap'n Proto would be significantly faster than the pure-Python protobuf implementation. Parsing Protobufs in Python is really horrible performance-wise since you have to inspect and branch on almost every byte. The way to make Python fast is to delegate as much work as possible to the built-in libraries that are written in C. But, there's just nothing that can be delegated in the case of Protobufs. In contrast, a Cap'n Proto parser could pretty easily leverage the existing `struct` module. That said, if you enable Cap'n Proto's "packed" mode, then this advantage is lost, since that's another byte-by-byte algorithm that will perform poorly in pure Python.

by kentonv 12y ago

sandGorgon 12y ago

@kentonv - yes that is what I was asking and thank you for asking. I think one aspect of my question got lost in the noise and it is my fault. On Python, protobuf vs capnproto is not apples to apples, since the former is pure python ... while yours is python wrapper over C. I have read your justifications [1] and I agree with you. But do note that there are some large usecases for Python on desktop software. C-extensions turn to be blockers in those cases. In many ways, I was hoping that you would have a pure-python version as well (since you did build one at Google) which sacrifices speed for compatibility. I was thinking in that context. [1] http://kentonv.github.io/capnproto/news/2013-09-04-capnproto-0.3-python-tools-features.html http://kentonv.github.io/capnproto/news/2013-09-04-capnproto...

kentonv 12y ago
It would be great if someone were to contribute a pure-Python implementation, but it's unlikely the sandstorm.io team will work on this since it has no real use to us. I actually think it's likely that a pure-Python version of Cap'n Proto would be significantly faster than the pure-Python protobuf implementation. Parsing Protobufs in Python is really horrible performance-wise since you have to inspect and branch on almost every byte. The way to make Python fast is to delegate as much work as possible to the built-in libraries that are written in C. But, there's just nothing that can be delegated in the case of Protobufs. In contrast, a Cap'n Proto parser could pretty easily leverage the existing `struct` module. That said, if you enable Cap'n Proto's "packed" mode, then this advantage is lost, since that's another byte-by-byte algorithm that will perform poorly in pure Python.