nihalwashere

3 ms·

1. 1.▲
   by nihalwashere 7mo ago

   We discuss token limits and context windows extensively, yet we rarely talk about the least complicated issue: Claude Code simply forgets everything between sessions. It does not recall the bug you solved last week. It is not aware of which
2. 2.▲
   Reader – web scraping that outputs clean Markdown for LLMs(github.com)

   3 points by nihalwashere 9mo ago|2 comments
3. 3.▲
   by nihalwashere 9mo ago

   I built an open-source web scraping engine for LLMs, self-hostable, Docker ready I kept rebuilding web scraping infrastructure for AI projects, so I open-sourced it. Two functions: - `scrape()` – any URL to markdown - `crawl()` – entire sit
4. 4.▲
   by nihalwashere 9mo ago

   I will add some reference examples on how to build a rate-limiting and K8s orchestration layer on top of Reader soon :)
5. 5.▲
   by nihalwashere 9mo ago

   Distributed rate-limiting is intentionally not in the core library, Reader focuses on the scraping primitives and stays unopinionated about orchestration. For multi-node rate limiting, you'd layer that on top: Redis + a simple limiter
6. 6.▲
   by nihalwashere 9mo ago

   Thanks! Honestly that was just pure luck with the domain :) There's a Docker deployment guide here: https://docs.reader.dev/documentation/guides/deployment For k8s, you can run multiple Reader instances behin
7. 7.▲
   Why I Built Reader: Open-source web scraping for LLMs(reader.dev)

   3 points by nihalwashere 9mo ago|7 comments
8. 8.▲
   Show HN: Superjson – Simple, beautiful JSON explorer(superjson.dev)

   2 points by nihalwashere 9mo ago|0 comments
9. 9.▲
   Show HN: Reader – open-source web scraping engine built for LLMs(github.com)

   1 points by nihalwashere 9mo ago|0 comments