3 ms·
I had tried using monolith [0] to feed webpages into Claude but all the html was too much token context. I ended up Print > Save as PDF-ing somewhat often and t
by MattSayar 2y ago
I had tried using monolith [0] to feed webpages into Claude but all the html was too much token context. I ended up Print > Save as PDF-ing somewhat often and that worked pretty well. But just giving a URL is ideal.
[0] https://github.com/Y2Z/monolith https://github.com/Y2Z/monolith
- knowaveragejoe 2y agoUsing the fetch MCP tool(which is very basic) works fairly well for me, though various sites will block it from time to time. https://github.com/modelcontextprotocol/servers/tree/main/src/fetch https://github.com/modelcontextprotocol/servers/tree/main/sr...
- MattSayar 2y ago> though various sites will block it from time to time. The page itself describes a --ignore-robots-txt and customizing the user agent. Guess we can just all copy OpenAI and continue to make SourceHut's life miserable /s This is a cool tool, thanks for sharing
- knowaveragejoe 2y agoFor what it's worth, it's fetch under the hood. More akin to curl than automated scraping.