3 ms·
Fair enough, especially if explicit alerting is involved. Have you considered a hybrid solution, something that generates a contract from a large corpus of dat
by nerdponx 3y ago
Fair enough, especially if explicit alerting is involved.
Have you considered a hybrid solution, something that generates a contract from a large corpus of data, which can then be deployed statically?
I consider "responding to change" as a somewhat different scenario from "heterogeneous but not changing". So statically generating a contract from an existing corpus supports the latter.
I could also envision some kind of graceful degradation, where you have a static contract, but you have dynamic adjustments instead of outright failures if the data does not conform to that contract.
- vasa_ 3y agoI worked with dlt guys on exactly that. Using OpenAI functions to generate a schema for the data based on the raw data structure. You can check that work here: https://github.com/topoteretes/PromethAI-Memory https://github.com/topoteretes/PromethAI-Memory It's in the level 1 folder