enduku

4 ms·

1.
by enduku 9d ago
I think tcmalloc gains on thread churn and oversubscription by going the cpu-cache route on Linux. on other platforms, I am not so sure but that can be offset by say a treiber-stack like setup for cross-thread frees/teardowns. So lesse
2.
by enduku 3mo ago
yeah the site's clearly vibecoded and isn't opensource, but i also think this is a genuinely interesting design space and more people should be building in it. APL ( https://www.dyalog.com/ ), BQN ( https:/&#x2
3.
by enduku 4mo ago
The `Cilk` angle is interesting. There’s still room for small runtimes focused just on fork/join recursion. I’ve been working on one for C: https://github.com/xtellect/cactus It’s narrower than Silk/SeaStar:
4.
by enduku 5mo ago
Fair points on both. Thanks for aasking. The 120 MiB/s encode ceiling is the cost of the mode competition. that's where the ratio comes from. At 800-1600 MiB/s off a digitizer, fc is the bottleneck no matter what transport si
5.
Vibe, A single-header lock-free networking library for Linux(github.com)
4 points by enduku 5mo ago|0 comments
6.
by enduku 5mo ago
Yeah, and also approximating a double (within range) to int32 :) https://x.com/Densebit/status/1839705674378613043?s=20
7.
by enduku 5mo ago
Thank you. Fuzz safety is definitely on my list. Current focus is to broaden the benchmarks , predictors and preprocessors and see what sticks
8.
by enduku 5mo ago
I need to add it to the benchmark. My expectation is that OpenZL should be strong when the enclosing format is known and SDDL can separate typed fields cleanly. Running both on the same f64 arrays will give some information
9.
by enduku 5mo ago
Yes it is. The mismatch is mainly representation and purpose: audio is usually int16/int24/float32 PCM, and audio codecs often exploit perceptual loss. fc is lossless and currently tuned for float64 streams.
10.
by enduku 5mo ago
Agreed; pcodec is probably one of the most relevant comparisons. I will add pcodec to teh benchmark
11.
by enduku 5mo ago
I have an XOR128-style mode and a byte-transpose/byte-split-like mode, but I should not claim that as a proper Chimp128 or Arrow Parquet byte-stream-split comparison yet. I willadd direct baselines for Chimp128 and Arrow/Parquet B
12.
by enduku 5mo ago
Agreed. will work on that :)
13.
by enduku 5mo ago
I’m regarding that term loosely here- in this case it is 'try several representations/codecs for a block and store the winner.' Similar ideas show up in columnar formats choosing encodings per column/page, OpenZL select
14.
by enduku 5mo ago
That’s a fair description. One mode does not dominate in my current harness; the winning mode varies quite a bit by dataset/block. If real workloads show one or two modes dominate, I’d rather simplify the portfolio :) For now the extra
15.
by enduku 5mo ago
Thanks, this looks super relevant. I think the transferable part is the per-block selectrover predictors, strides, deltas, exponent/mantissa-ish structure, byte transpose, fallback raw/LZ, etc.sddl2 looks like a natural place to
16.
by enduku 5mo ago
It is intended t obe mainly source agnostic (will try to add custom source predictors too). The idea is to treat input as an ordered stream of doubles and look for numeric structure like repeats, smooth deltas, fixed increments, or low-ent
17.
Fc, a lossless compressor for floating-point streams(github.com)
108 points by enduku 5mo ago|33 comments
18.
by enduku 5mo ago
I built "fc", a C library for compressing streams of 64-bit floating-point values without quantization. It is not trying to replace zstd or lz4. The idea is narrower: take blocks of doubles, try a set of float-specific predictors&
19.
Show HN: Vibe, a single-header C networking library for Linux(github.com)
4 points by enduku 5mo ago|0 comments
20.
by enduku 5mo ago
I think this is interesting too sqlite a as the coordination boundary: business state, queue state, stream offsets, retries, and acks all sharing one transactional substrate. The 1ms polling is getting a lot of weight in the thread though :
21.
by enduku 5mo ago
Yes: contention and locality. In Cactus the fast path is local. A worker pushes its own continuation onto its own deque, runs the child, and later tries to reclaim that continuation locally. Other workers only touch that deque when they bec
22.
Cactus, a work-stealing parallel recursion runtime for C(github.com)
14 points by enduku 5mo ago|2 comments
23.
by enduku 5mo ago
I ran into this rabbithole while writing an x86-64 asm rewriter. xor was the default zeroing idiom.I onkly did sub reg,reg when I actually want its flags result. Otherwise the main rule is: do not touch either form unless flags liveness mak
24.
by enduku 6mo ago
I feel like LLMs are just forcing me to realize what writing actually is. For me, writing is basically a mental cache clear. I write things down so I can process them fully and then safely forget them. If I let an LLM generate the text, tha
25.
by enduku 6mo ago
No AI was used. I see no problems with using AI to write code whatsoever, but this isn't that. The formatting is my screw-up. I ran clang-format with a bad config, then tried to hand-fix the result and made it worse. The parenthesizati
26.
by enduku 6mo ago
Fair points on both - the 5ns is the L2 hit case. I should have stated the range (30-60ns?) instead of the best case. And yes, fixing the tcmalloc case is on my list - thanks for pointing that out. And also to be clear, the goal was never t
27.
by enduku 6mo ago
I am aware of dlmallc/mspaces and GNU Obstacks. Both were in a way, original inspirations for spaces. Though I hadn't looked at mspaces source in years, I remember its inline boundary tags enabling zero overhad per allocation and
28.
Show HN: jsoon, a streaming JSON parser and query engine in C(github.com)
3 points by enduku 6mo ago|0 comments
29.
by enduku 6mo ago
Thanks for taking a look, really appreciate the thoughtful feedback! You're absolutely right about Fibonacci. It's a terrible performance example since the work per fork is basically zero :) I included it as an 8-line API sh
30.
Cactus, a work-stealing parallel recursion runtime for C(github.com)
4 points by enduku 6mo ago|3 comments

More ›