3 ms·

Very cool, seems I missed it the last time it was on HN. From a quick glance over the docs, it seems by default the only parsing interfaces available are "fully

by _wmd 12y ago

Very cool, seems I missed it the last time it was on HN. From a quick glance over the docs, it seems by default the only parsing interfaces available are "fully realized", i.e. complete conversion to data structures in the target programming language, or "pull based" parsing a bit like expat. One of the more interesting aspects of a serialization library is how it copes with efficiently extracting small fields out of otherwise large structures, usually by only decoding the absolute minimum required to access the field. This is a common operation e.g. when running aggregate queries over data-at-rest. For example, Cap'n'proto is well suited to this. Does Bond support anything like it? I can imagine driving the parser APIs using custom code to achieve a similar effect, but that seems a lot of work when there are alternatives that don't require it.

sapek 12y ago
Bond has 3 ways of working with serialized payloads: 1) Simple deserialization where an object is fully materialized. This is the simplest (you write your business logic against idiomatic types in your target language) but, as you note, might not be best from performance perspective in some scenarios. 2) Lazy deserialization at the level of nested structs via bonded<T> [1]. In many cases this allows to optimize performance if you don't need to materialize the whole object but w/o loosing much in terms of simplicity of the programming model. 3) Transforms [2]. This is a very generic and very powerful abstraction in Bond. All higher level APIs like serialization and deserialization are internally implemented using this mechanism (even Python bindings [3] are fully implemented on top of Bond C++ meta-programming infrastructure driving Boost Python). In C++ it is built using template meta-programming and in C# using LINQ expressions. In Both cases it is a little like having a generic, strongly typed parser for richly typed data. [1] https://microsoft.github.io/bond/manual/bond_cpp.html#understanding-bondedt https://microsoft.github.io/bond/manual/bond_cpp.html#unders... [2] https://microsoft.github.io/bond/manual/bond_cpp.html#transforms https://microsoft.github.io/bond/manual/bond_cpp.html#transf... [3] https://microsoft.github.io/bond/manual/bond_py.html https://microsoft.github.io/bond/manual/bond_py.html

by _wmd 12y ago

sapek 12y ago

Bond has 3 ways of working with serialized payloads: 1) Simple deserialization where an object is fully materialized. This is the simplest (you write your business logic against idiomatic types in your target language) but, as you note, might not be best from performance perspective in some scenarios. 2) Lazy deserialization at the level of nested structs via bonded<T> [1]. In many cases this allows to optimize performance if you don't need to materialize the whole object but w/o loosing much in terms of simplicity of the programming model. 3) Transforms [2]. This is a very generic and very powerful abstraction in Bond. All higher level APIs like serialization and deserialization are internally implemented using this mechanism (even Python bindings [3] are fully implemented on top of Bond C++ meta-programming infrastructure driving Boost Python). In C++ it is built using template meta-programming and in C# using LINQ expressions. In Both cases it is a little like having a generic, strongly typed parser for richly typed data. [1] https://microsoft.github.io/bond/manual/bond_cpp.html#understanding-bondedt https://microsoft.github.io/bond/manual/bond_cpp.html#unders... [2] https://microsoft.github.io/bond/manual/bond_cpp.html#transforms https://microsoft.github.io/bond/manual/bond_cpp.html#transf... [3] https://microsoft.github.io/bond/manual/bond_py.html https://microsoft.github.io/bond/manual/bond_py.html