3 ms·
This pattern is really meaningful conceptually but the tricky thing is to not create a mess in the process. If it's too easy to branch people will do so and th
by ragulpr 2y ago
This pattern is really meaningful conceptually but the tricky thing is to not create a mess in the process.
If it's too easy to branch people will do so and the (knowledge) economics scale disappears (and we'll have a mess).
If the common data definition is too hard to branch from no experiments will happen (slow).
I think most tech for this seem to make it too easy, and in the process injecting a bunch of dependencies that makes it slow and harder to access. May have changed since I last looked.
I found that the simple pattern of versioned paths/table names as `s3://mybucket/mystage/version=42/` or `my_table_v42` puts a high enough evolutionary cost on branching (as consumers need to explicitly adapt) while it also doesn't have the costs associated with using special tech (legacy/lock in/dependencies).
It's also searchable on github/slack/etc if done right.