3 ms·

Oh damn I will definitely look into open sourcing it and making it a sdk

• stormfather 2y ago
  Awesome! I write LLM powered scrapers and stuff all the time and one of the biggest pain points is HTML is full of so much crap that isn't meaningful and overwhelms the context. And being a data science guy idk how to solve this.
  • jawerty 2y ago
    awesome that's the same reason why I use it. It's basically a balance between the full html and having the markdown type scrapers that are better for just text. Do you mind if I reach out to you once I set up the Github?
    • stormfather 2y ago
      You're very welcome to! Please do. You can reach out to notpricedinyet@gmail.com