5 ms·

> be careful to avoid implying that all events should be remembered Is that true in a post-AWS world with storage getting cheaper faster than you can consume i

> be careful to avoid implying that all events should be remembered Is that true in a post-AWS world with storage getting cheaper faster than you can consume it? Maybe don't put 4k video in the log. But, you're right, we can stream, shard and forget as necessary. > also avoid depending on anything in the future unless you want to wait for it Can you elaborate on this, my gut reaction is to ask why I need to avoid depending on git commits that haven't been written yet – it's kind of a weird question right? Time is explicit now, which means you have the right knobs you need to coordinate it, even across distributed nodes.

csande17 7y ago
> Is that true in a post-AWS world with storage getting cheaper faster than you can consume it? This might be true of "storage" as in disk space, but it definitely isn't true of "storage" as in RAM. If your phone kept an in-memory log of every single click event, you'd run out of RAM pretty fast.
skybrian 7y ago
If you use a value to do a calculation and it's from the future then often you end up blocking until it's available. This is how a Future works and is similar to what happens when you read from a TCP connection and the data has to be retransmitted because the network dropped the packet, so you block. If you want responsive output then you should avoid blocking or at least have a timeout and/or cancel button. The conceptual idea of a value that continuously changes is nice for animation or maybe video (though that's both discrete and lossy), but seems messier when your knowledge is limited to an unknown selection of discrete data points from that timeline, arriving with an unknown amount of lag? Maybe you could talk about that mathematically, but it seems like it's going to be rather abstract and messy.
convolvatron 7y ago
i think it falls into the slightly-too-clever category, but http://www.neilconway.org/docs/vldb2014_edelweiss.pdf http://www.neilconway.org/docs/vldb2014_edelweiss.pdf treats safe log truncation as a conservative optimization. its similar to tail-call in that the efficacy of your program depends on the compiler figuring out what you're trying to do - which is unsatisfying but its a great idea and a ray of hope here edit: this also implies that the set of reads against the history is fully known at compile time - which may make it irrelevant depending on the usage

csande17 7y ago

> Is that true in a post-AWS world with storage getting cheaper faster than you can consume it? This might be true of "storage" as in disk space, but it definitely isn't true of "storage" as in RAM. If your phone kept an in-memory log of every single click event, you'd run out of RAM pretty fast.

skybrian 7y ago

If you use a value to do a calculation and it's from the future then often you end up blocking until it's available. This is how a Future works and is similar to what happens when you read from a TCP connection and the data has to be retransmitted because the network dropped the packet, so you block. If you want responsive output then you should avoid blocking or at least have a timeout and/or cancel button. The conceptual idea of a value that continuously changes is nice for animation or maybe video (though that's both discrete and lossy), but seems messier when your knowledge is limited to an unknown selection of discrete data points from that timeline, arriving with an unknown amount of lag? Maybe you could talk about that mathematically, but it seems like it's going to be rather abstract and messy.

convolvatron 7y ago

i think it falls into the slightly-too-clever category, but http://www.neilconway.org/docs/vldb2014_edelweiss.pdf http://www.neilconway.org/docs/vldb2014_edelweiss.pdf treats safe log truncation as a conservative optimization. its similar to tail-call in that the efficacy of your program depends on the compiler figuring out what you're trying to do - which is unsatisfying but its a great idea and a ray of hope here edit: this also implies that the set of reads against the history is fully known at compile time - which may make it irrelevant depending on the usage