3 ms·
We’ve used Avro in production for a number of years now. We use it to define the schema for Kafka messages passing between parts of the system and it has worked
by sfev 7y ago
We’ve used Avro in production for a number of years now. We use it to define the schema for Kafka messages passing between parts of the system and it has worked well - but you want some downsides, so..
Compatibility: Avro deals with this well, but there are gotchas especially when you need/want to have “forward compatibility” as well as “backward compatibility” in your messages. Adding a new type to a union? Breaking change. Adding a new element to an enum? Breaking change. Want to add a new member to a type? Best give it a default, or.. breaking change. We use the Confluent schema registry to manage our schemas on kafka and we find ourselves having to do carefully managed rollouts (stop all producers, put schema-reg into compat mode, upgrade consumers, upgrade and start producers) every now and again
Performance: we frequently see Avro as the “hot” code in our apps - mainly I suspect because we’ve got a fairly large schema with lots of union types etc, and our messages are up to 2kb or so in size. This results in our applications topping out at low tens of thousands of messages per second, though it scales linearly with number of cores. The most recent java version has a “fast read” flag that promises to speed up our use-case but we haven’t tried it yet
- nly 7y ago> Adding a new type to a union? Breaking change. Adding a new element to an enum? Breaking change. This is incorrect. I've personally gone from type X to union {X,Y} in C++, you just need to use the resolving decoder that takes both schemas. Enums can also be resolved based on names. Check out the schema evolution rules here: See https://avro.apache.org/docs/current/spec.html#Schema+Resolution https://avro.apache.org/docs/current/spec.html#Schema+Resolu... Of all the binary serialization formats of this ilk (proto, thrift, flatbuffers etc), Avro definitely has the most schema flexibility.