3 ms·
A similar strategy we [1] are pursuing is to use `reduce` JSON Schema annotations to make it easy to define generalized reductions over your JSON data types [2]
by jgraettinger1 5y ago
A similar strategy we [1] are pursuing is to use `reduce` JSON Schema annotations to make it easy to define generalized reductions over your JSON data types [2].
Taking the examples from the article, equivalent schemas might be:
type: object
properties:
mySum:
type: number
reduce: { strategy: sum }
Or even:
type: object
properties:
myDeeplyAggregatedMap:
type: object
reduce: { strategy: merge }
additionalProperties:
reduce: { strategy: sum }
Use of WASM is really interesting. We've been exploring it as a means for powering user-defined reduction strategies, in cases where the built-in strategies are insufficient.
[1] https://estuary.dev https://estuary.dev
[2] https://docs.estuary.dev/reference/catalog-reference/schemas-and-data-reductions https://docs.estuary.dev/reference/catalog-reference/schemas...