6 ms·

Show HN: (Sukhoi) Minimalist and Powerful Web Crawler in Python

• staticautomatic 9y ago
  Can it be used with lxml's parser, and does it have support for what Requests calls `Session()`?
• iogf 9y ago
  No built-in session request implemented so far however, it can be easily emulated. Handling sessions is going to be implemented soon in websnake's framework. https://github.com/iogf/websnake https://github.com/iogf/websnake Feel free to contact me on irc i hang at #vy on irc.freenode.org if you need help/willing to contribute somehow.
• iogf 9y ago
  and yeah, it could be used with lxml parser too as well as xpath(i'm planning to add that feature too).
• gaojiuli 9y ago
  What's the advantages of Sukhoi compared to the following. Scrapy: https://github.com/scrapy/scrapy https://github.com/scrapy/scrapy PySpider: https://github.com/binux/pyspider https://github.com/binux/pyspider Gain: https://github.com/gaojiuli/gain https://github.com/gaojiuli/gain
  • iogf 9y ago
    sukhoi is a little baby yet, it is built on a good concept in which it permits you to construct all kind of structures from your fetched data easily. It has a shorter learning curve due to its base concept too. The second example in sukhoi web page is extremely simple and it does something that if you are going to implement using other crawler it may get a bit tricky. Try conceiving the two examples in sukhoi using those web crawlers it should give you a feeling about how sukhoi makes things easier.
• deleted 9y ago
  [deleted]
• deleted 9y ago
  [deleted]