3 ms·
I do not think its unfair at all. Schema registry needs to add a wrapper and UUID to an Avro payload for it to work, so at the very least Avro as-is is unsuitab
by dtech 3y ago
I do not think its unfair at all. Schema registry needs to add a wrapper and UUID to an Avro payload for it to work, so at the very least Avro as-is is unsuitable for a MQ like Kafka since you cannot use it efficiently without some out-of-band communication channel.
- insanitybit 3y agoEveryone knows you need an out of band channel for it, I don't know why you're putting this out there like it's a fault instead of how it's designed. Whether it's RPC where you can deploy your services or a schema registry, that is literally just how it works. Wrapping a message with its schema version so that you can look up that version is a really sensible way to go. A uuid is way more than what's needed since they could have just used a serial integer but whatever, that's on Kafka for building it that way, not Avro.
- morelisp 3y ago> a serial integer And now you can't trivially port your data between environments.
- insanitybit 3y agoCan you elaborate? I don't see any issue at all.
- morelisp 3y agoUnderstanding the data now depends not just on having a schema to be found in a registry, but your schema registry, with the schemata registered in the same specific order you registered them in. If you want to port some data from prod back to staging, you need to rewrite the IDs. If you merge with some other company using serial IDs and want to share data, you need to rewrite the IDs. Etc.
- insanitybit 3y agoI have no idea what you're talking about. If the standard were a 64bit integer none of what you're saying would be the case at all. There is no difference between a random 128bit integer vs a sequentual 64bit integer except that the 64bit integer is smaller.
- kentonv 3y agoThey're saying that for a sequential number, you must have a central authority managing that number. If two environments have different authorities, they have a different mapping of these sequential numbers, so now they can't share data.
- insanitybit 3y agoOh. Like, if you have two different schema registries that you simultaneously deploy new schemas to while also wanting to synchronize the schemas across them. Sounds weird as hell.