4 ms·
Adding sources is pretty easy. We just need them to provide: * timestamped changes eg "record foo was deleted at time 42" * watermarks eg "I've now sent you a
by jamii 7y ago
Adding sources is pretty easy. We just need them to provide:
* timestamped changes eg "record foo was deleted at time 42"
* watermarks eg "I've now sent you all the changes up to time 42"
Here's the implementation of the kafka source - https://github.com/MaterializeInc/materialize/blob/master/src/dataflow/source/kafka.rs https://github.com/MaterializeInc/materialize/blob/master/sr.... (There's also a bit of code in the parser/planner to wire up the CREATE SOURCE syntax, but it's fairly trivial.)