jamii

10 ms·

61.
by jamii 5y ago
It's getting there. I started 3.5 months ago and I'm currently making ~75% of minimum wage (which to be fair is pretty high in BC). I suspect it's also going to be fragile. The next time we hit a recession or, god forbid, ano
62.
by jamii 5y ago
This is why the high temporal locality part of the map is all EC - when everything is windowed you can just wait for the window to close. On the low temporal locality side, the only consistent system I've seen so far is differential da
63.
by jamii 5y ago
Spark structured streaming is in there under structured, high temporal locality. It didn't make it into https://scattered-thoughts.net/writing/internal-consistency-... because it has severe limitations for low tem
64.
by jamii 5y ago
> Do I have that right? Yes, I think so. If you want to be able to handle out-of-order data, I don't think there is a way to garbage collect old time periods and still produce correct results unless you stop accepting new data for t
65.
by jamii 5y ago
Yeah, I touch on this near the bottom (ctrl-f 'bitemporal'). I think having multiple watermarks would be a neat solution. Kinda like the way flink sends markers through to get consistent snapshots for fault tolerance.
66.
by jamii 5y ago
Ok, that makes sense.
67.
by jamii 5y ago
I wrote a really long reply to a comment that got deleted before I finished. In case other people have the same questions: > What is a valid subset to consider? For these purposes, any subset. > What is "correct output"? I a
68.
by jamii 5y ago
Oh, wait, this doesn't sound promising. > If the watchType is set to FileProcessingMode.PROCESS_CONTINUOUSLY, when a file is modified, its contents are re-processed entirely. This can break the “exactly-once” semantics, as appending
69.
by jamii 5y ago
You could instead put the global max seen id into every row, but then you would have to update all the rows on every transaction. Which is not great peformance-wise, but would also massively exacerbate the non-atomic sum problem downstream
70.
by jamii 5y ago
> And then try to join credits and debits together by updating_tx. You can't join on updating_tx because the credits and debits per account are disjoint sets of transactions - that join will never produce output. I did try something
71.
by jamii 5y ago
Oh, I have seen some of the lasp papers. And I guess https://dl.acm.org/doi/abs/10.1145/2578855.2535842 is related too. Some of the stuff coming out of the RISE lab too eg https://rise.cs.berkeley.
72.
by jamii 5y ago
> internal consistency as defined here seems to mean how "multiple reads" can lock into the same storage state Sort of. When you make a single read of a single key at the end of a streaming topology, that's equivalent to r
73.
by jamii 5y ago
> you can easily express any operations you want That's true in that the table api itself is built on top of the datastream api. But to run the example you'd have to implement your own joins and retraction-aware aggregates and
74.
by jamii 5y ago
Did basho have any internal discussion about how to ensure that operations reading and writing from multiple CRDTs were still confluent? The only work I've seen on this is http://www.neilconway.org/docs/socc2012_bl
75.
by jamii 5y ago
> converting transactions to datastream then back to SQL will introduce a materializing barrier It seems that this would still leave the problem where each transactions causes two deletes and two inserts to `balance` and then `total` sum
76.
by jamii 5y ago
> it should have all the necessary building blocks to do anything that can be done with the table API That's true in that the table api itself is built on top of the datastream api. But to run the example you'd have to implemen
77.
by jamii 6y ago
My bad, should have verified first.
78.
by jamii 6y ago
I mentioned it in the next section: > The standard library includes a set of allocators which don't reuse allocations, preventing use-after-free, and which catch double-free. I'm not clear yet on how high the runtime and memory
79.
by jamii 6y ago
I wasn't able to find many breakdowns of actual exploits by root cause. Do you have additional sources that I could add to the article?
80.
by jamii 6y ago
The author gives more detail in this thread - https://lobste.rs/s/v5y4jb/how_safe_is_zig#c_vddk9j
81.
by jamii 6y ago
> it's trivial to construct a non nullable pointer that is null or uninitialised This is checked at runtime in ReleaseSafe. jamie@machine:~$ cat test.zig pub fn main() void { var x: usize = 3; _ = @intToPtr
82.
by jamii 6y ago
That's also the recommended way to deal with it in c. It hasn't been effective in preventing vulnerabilities based on use-after-free. If the GPA is practical to use in production, that will be a different story. But it doesn'
83.
by jamii 6y ago
> I don't think I understand what is meant by "type confusion". Accessing a memory location with one type as if it's another. In C or C++ it's usually because you accessed a union without checking a tag somewhere
84.
by jamii 6y ago
Despite being an 'easy problem' it has led to code execution on android/ios: https://research.checkpoint.com/2019/select-code_execution-f... > We established that simply querying a database may not be
85.
by jamii 6y ago
Good idea. I'll figure out how to do that.
86.
by jamii 6y ago
You're already there - https://scattered-thoughts.net Also past HN conversation - https://news.ycombinator.com/from?site=scattered-thoughts.ne...
87.
by jamii 6y ago
Judging by https://news.ycombinator.com/from?site=scattered-thoughts.ne... this would have been submitted anyway. But I will be more patient next time. Nervous energy, you know.
88.
A new newsletter on database engines, streaming systems, query planning(scattered-thoughts.net)
47 points by jamii 6y ago|8 comments
89.
by jamii 6y ago
Express Entry also doesn't require an employer, although you do get an extra 100 points if you have a job offer from an employer with a Labor Market Impact Assessment. This was one of the big upsides for me - very few countries will gi
90.
by jamii 6y ago
What do you usually want to happen with late data? In DD you have the option to ignore it at the source but not to update already-emitted results. Is the latter important for you?

More ›