3 ms·

nice to see more go tooling. curious how you handled the lock-free part - was it mostly atomics or just clever channel usage? always found backpressure tricky

• sandeepk235 10mo ago
  It's mostly atomics, with a deliberate design choice to avoid traditional backpressure. 1. Lock-Free Parts (Atomics) The ring buffer relies on atomic.Uint64 for head/tail cursors. Also, hot-swapping the processor chain uses atomic.Pointer[T] to ensure the worker loop never blocks even during a config reload: // Worker reads config without locks currentChain := p.chain.Load() currentChain.Process(data) // Config updates swap the pointer atomically p.chain.Store(newChain) I intentionally avoided channels for the data path. Benchmarks showed that at >100k msgs/sec, the channel allocation/locking overhead was 2-3x costlier than the ring buffer approach. Handling Backpressure: I Don't (Intentionally) You're right that backpressure is tricky without locks. My approach was to eliminate the need for it: Drop-on-full: If saturated, new logs are dropped (preserves recent history, non-blocking) Fail-open circuit breaker: If buffer >80% full, bypass processing to drain faster Philosophy: For observability, dropping overflow is better than blocking ingestion The honest answer: This isn't "wait-free" in the academic sense—WaitGroup is used in output fanout. But the ingestion→buffering path has zero mutexes.