gunnarmorling

5 ms·

31.
by gunnarmorling 11mo ago
You could do that, but you'd run into exactly the same bottleneck the author describes with a remote Postgres instance. The workload exposes high contention on hot rows. If transactions are kept open for several milliseconds due to thi
32.
by gunnarmorling 11mo ago
The comment at [1] hints at a solution: in the response of the write request return the id of the transaction or its commit position in the TX log (LSN). When routing a subsequent read request to a replica, the system can either wait until
33.
by gunnarmorling 11mo ago
Great to see more love for using Lucene on Lambda via GraalVM native binaries! Explored that same idea few years back for my personal blog [1]. It worked pretty well, but I moved eventually away from Lambda as I wanted to close down my pers
34.
by gunnarmorling 11mo ago
Camunda (not affiliated btw.) crossed 100M ARR last year. So yes, this apparently is being used. Whether these users hang out here on HN or not, that I don't know.
35.
by gunnarmorling 11mo ago
It depends on what exactly you mean by "Atomic Transaction". If you are referring to database transactions, then DE is not that. But DE helps you implement potentially long-running multi-step business transactions, a.k.a. workflow
36.
by gunnarmorling 11mo ago
Sure, systems like Activiti or Camunda come to mind.
37.
by gunnarmorling 11mo ago
Ah, that's awesome. Definitely need to take a closer look at your implementation.
38.
by gunnarmorling 11mo ago
I'm still swinging back and forth which approach I ultimately prefer. As stated in the post, I like how the proxy approach largely avoids any API dependency. I'd also argue that Java developers actually are very familiar with this
39.
by gunnarmorling 11mo ago
Yupp, making that same point in the post :) > You could think of [Durable Execution] as a persistent implementation of the memoization pattern, or a persistent form of continuations.
40.
by gunnarmorling 11mo ago
> I never understood why AOT never took off for Java. GraalVM native images certainly are being adopted, the creation of native binaries via GraalVM is seamlessly integrated into stacks like Quarkus or Spring Boot. One small example woul
41.
by gunnarmorling 11mo ago
They can happen, yes, although this should be a rather rare event (the most common reason would be misconfiguration, such as a K8s pod with too low memory limits). That said, work towards exactly-once has been done [1], utilizing the suppor
42.
by gunnarmorling 11mo ago
For those not on 18 yet, there's also max_slot_wal_keep_size, which invalidates slots based on the amount of WAL they retain rather than based on a period of inactivity. It's available since PG 13.
43.
by gunnarmorling 11mo ago
Debezium generally produces each change event exactly once if there are no unclean connector shut-downs. If that's not the case, I'd consider this a bug which ought to be fixed. (Disclaimer: I used to lead the Debezium project)
44.
by gunnarmorling 11mo ago
Robustness is a key design goal of Debezium. It supports heart beating to address WAL growth issues (wrote about that issue at [1]). If Kafka crashes (or Debezium itself), it will resume consuming the replication slot from where it left off
45.
by gunnarmorling 11mo ago
> but it feels a bit contrived to me ('missing fields are ok'?) IIUC, the idea is that an object can be safely deserialized if its field set upon deserialization is a strict subset of its field set upon serialization. I think t
46.
by gunnarmorling 1y ago
To those who flagged this submission, can you share your reasoning? What guidelines do you see being violated by it? Thanks! Disclaimer: I'm the author of this post.
47.
by gunnarmorling 1y ago
Are you referring to using Debezium embedded as a library? If so, yes, it absolutely has its place; for instance, it's used by Flink CDC. There's pros and cons to either way of running Debezium. Seeing embedded Debezium a lot for
48.
by gunnarmorling 1y ago
> the answer is "depends"? Indeed that is the point I am trying to make in the article. Postgres oftentimes absolutely is the right tool to use, and oftentimes it's not. The thing I'm advocating to be wary of is &quot
49.
by gunnarmorling 1y ago
Thank you, appreciate it!
50.
by gunnarmorling 1y ago
> this blog post should really have a disclaimer somewhere that the author works for Confluent Good idea; this is stated in the bio on my web site, but I've just added the same info again to the end of the post.
51.
Why I'm not a fan of zero-copy Apache Kafka-Apache Iceberg(jack-vanlightly.com)
2 points by gunnarmorling 1y ago|0 comments
52.
Lower Java Tail Latencies with ZGC(morling.dev)
2 points by gunnarmorling 1y ago|0 comments
53.
by gunnarmorling 1y ago
I wouldn't expect migrating from 17 to 25 to be an awful lot of work. The hard bump was moving from Java 8 to 9+ (11, typically), due to the introduction of the module system, removal of APIs previously shipped with the JDK (e.g. JAXB)
54.
by gunnarmorling 1y ago
> PostgreSQL databases typically don't generate huge amounts of data The live data set may not be huge, but the entire trail of all changes of all current and all previously existing data may easily exceed the volume of data you can
55.
by gunnarmorling 1y ago
GC also has its downsides: - Marking and sweeping cause latency spikes which may be unacceptable if your program must have millisecond responsiveness. - GC happens intermittently, which means garbage accumulates until each collection,
56.
by gunnarmorling 1y ago
Ha, that's interesting :) Do you have any more details to that one?
57.
Mastering Postgres Replication Slots(morling.dev)
1 points by gunnarmorling 1y ago|0 comments
58.
by gunnarmorling 1y ago
Debezium handles logical decoding messages OOTB. There's also an SMT (single message transform) for decoding the binary payload: https://debezium.io/documentation/reference/stable/transform... .
59.
by gunnarmorling 1y ago
pg_logical_emit_message() is how I recommend users on Postgres to implement the outbox pattern [1]. No table overhead as you say, no need for housekeeping, etc. It has some other cool applications, e.g. providing application-specific metada
60.
by gunnarmorling 1y ago
The other day, I came across a blog post by someone I really value and it sounded very much like written by AI. So I decided to be very explicit and transparent about my own ways of using (and not using) AI for my blog: https://w

More ›