russellthehippo

5 ms·

31.
by russellthehippo 6mo ago
"a small proliferation" is a nice way to describe the cluster that is my side project habit. if you bump into any issues pls pull a PR or drop an issue on the repo!
32.
by russellthehippo 6mo ago
exactly. then you're building distributed locking and it's probably time for a different tool
33.
by russellthehippo 6mo ago
also this is designed for a single machine. 10k listeners on one machine seems like a lot!
34.
by russellthehippo 6mo ago
Interesting, thank you for the response and explanation. Honker workers/listerners are holding an open connection anyway. I do trust SQLite guarantees more than cross-platform sys behavior. I will explore the C API angle.
35.
by russellthehippo 6mo ago
Apologies for not being specific. The specific thing I'm talking about is this: write events don't fire until the file handle is closed. [1] I didn't validate this myself btw, but my original design was certainly trying to us
36.
by russellthehippo 6mo ago
It's good for pubsub but not for claim/ack workflow unless you do If-None-Match CAS semantics on a separate filesystem which, actually, yeah that's probably fine. Feels heavy on S3 ops. But! you do save on inter-AZ networking
37.
by russellthehippo 6mo ago
honk
38.
by russellthehippo 6mo ago
Actually need to test this. Will report back
39.
by russellthehippo 6mo ago
See comment below - Darwin silently drops same-process notifs. I could change the behavior depending on same vs cross process and platform but I wanted to”just one thing to worry about”. Potentially a good optimization later. Would help red
40.
by russellthehippo 6mo ago
Absolutely! That’s the durable pubsub angle for sure.
41.
by russellthehippo 6mo ago
Yep, definitely still in use. Do yall above have an opinion if the pragma is better than the syscall? What are the trade offs there? Another comment thread mentioned this as well and pointed to io uring. I was thinking that dism spam is wor
42.
by russellthehippo 6mo ago
Whoa I really hadn’t considered this. Do a litestream read replica, trigger across machines with S3 as the broker essentially. But you’re still stuck with the litestream sync interval. Maybe interesting for cross server notify?
43.
by russellthehippo 6mo ago
10k listeners is a lot. Thundering herd issue at stat(). SQLite may not be your best choice at this scale.
44.
by russellthehippo 6mo ago
Oh cool, so using io uring plus pragma data version would actually beat stat on Linux holistically speaking? The stat choice was all about cross platform consistency over inotify speed. But syscalls overwhelm can be real.
45.
by russellthehippo 6mo ago
I actually hadn’t thought about it this way. The killer app I was imagining was 1ms reactivity without SQL polling and messaging atomic with business commits, plus “one db” and no daemon. But this is actually a great main benefit as well.
46.
by russellthehippo 6mo ago
Nope! The extension just functions as a shortcut for raw SQL. Litestream edits the wal file but only like a normal checkpoint. So not too bad. Although I haven’t tested it directly. Probably need to
47.
by russellthehippo 6mo ago
The WAL file sticks around but gets truncated so that counts as an update. Though I don’t have tests for this. Good input, thanks I’ll make sure
48.
by russellthehippo 6mo ago
Breaks cross-platform, specifically Macs swallow silently. stat just works
49.
by russellthehippo 6mo ago
Great plug very relevant
50.
by russellthehippo 6mo ago
Best thing I've read on hacker news in so long. I relate so hard. Simon Willison's stuff on cognitive debt and "your job is to ship code you have proven to work" have led me to work on a project about Intent-Driven Devel
51.
by russellthehippo 6mo ago
Hey HN, I built this. Honker adds cross-process NOTIFY/LISTEN to SQLite. You get push-style event delivery with single-digit millisecond latency without a damon/broker, using your existing SQLite file. A lot of pretty high-traffic
52.
Show HN: Honker – Postgres NOTIFY/LISTEN Semantics for SQLite(github.com)
327 points by russellthehippo 6mo ago|84 comments
53.
by russellthehippo 6mo ago
Initial testing today - 4.7 excels at abstractions/implementations of abstractions in ways that often failed in 4.5/4.6. This is a great update, I've had to do a lot of manual spec to ensure consistency between design and imp
54.
by russellthehippo 7mo ago
yep came here to say this. great to hear it's in process.
55.
by russellthehippo 7mo ago
your comment doesn't read like llm. the other one does (other fruit entirelY?). i commented in the wrong place. i apologize. >>> AbanoubRodolf 2 days ago | root | parent | prev | next [–] [flagged]
56.
by russellthehippo 7mo ago
I have implemented a full eviction suite for turbolite in https://github.com/russellromney/turbolite/pull/6 . I includes connection-time and env-var-set eviction policies (size, count, etc.), user functions fo
57.
by russellthehippo 7mo ago
Also I'm curious how you would handle credentials - would you be proxying through your backend? To me turbolite is definitely a backend tool.
58.
by russellthehippo 7mo ago
Thanks! yeah it's definitely slow, but compared to other options it's quite fast: - Neon startup is 500ms+ - a Fly Machine with a SQLite database on a volume takes 150ms+ to start up at minimum - restoring a db from S3 with Litest
59.
by russellthehippo 7mo ago
Fantastic comment, thanks for jumping in. I have well-tuned replies here lol given we're working on exactly the same problems! First: your prefetch solution is precisely what I have in the roadmap for the next level of optimization: B-
60.
by russellthehippo 7mo ago
Not today, but the architecture isn't fundamentally incompatible. The page grouping and seekable compression would translate well to browser fetch + range GETs. It would need a new storage backend targeting OPFS/fetch instead of S

More ›