pablohoffman

7 ms·

1.
Ask HN: What are the best tools for web scraping in 2022?
313 points by pablohoffman 4y ago|151 comments
2.
by pablohoffman 11y ago
Hey folks, I'm the one interviewed, happy to answer any follow up question here.
3.
by pablohoffman 11y ago
Not surprised, I'm pretty low profile :)
4.
by pablohoffman 11y ago
Portia from Scrapinghub is 100% open source: http://scrapinghub.com/portia/
5.
Portia (open source visual scraper) introduces JavaScript support(blog.scrapinghub.com)
20 points by pablohoffman 11y ago|0 comments
6.
Using Git to manage staff vacations in a large distributed team(blog.scrapinghub.com)
6 points by pablohoffman 11y ago|0 comments
7.
by pablohoffman 12y ago
We released a similar open source tool for visual scraping, earlier this year, called Portia: https://github.com/scrapinghub/portia It's been getting quite a bit of traction and we're currently working on the
8.
by pablohoffman 13y ago
Not yet.
9.
by pablohoffman 13y ago
Yes, you just need to select a different field type ("text", instead of "html").
10.
Portia, an open-source visual web scraper(blog.scrapinghub.com)
367 points by pablohoffman 13y ago|67 comments
11.
Optimizing memory usage of scikit-learn models using succinct Tries(blog.scrapinghub.com)
23 points by pablohoffman 13y ago|0 comments
12.
by pablohoffman 13y ago
We are using CDH4.2 and have had a very positive experience so far. Cloudera has in fact been an inspiration for us to follow, you guys have really struck the right balance between open source and commercial support. We follow the same phil
13.
by pablohoffman 13y ago
We are not plainly complaining about MongoDB, nor saying it's useless. We are just explaining why it's a poor choice for a specific use case: storing scraped data. FWIW, we still use Mongo in other internal applications, it's just not the r
14.
by pablohoffman 13y ago
We went with HBase. Cassandra would have been suitable too, but we already use Hadoop for data processing so it was a natural choice within the infrastructure ecosystem. We will write a followup about that.
15.
The Twisted Way(glyph.twistedmatrix.com)
27 points by pablohoffman 14y ago|29 comments
16.
Finding similar items (web crawling)(blog.scrapinghub.com)
2 points by pablohoffman 14y ago|0 comments
17.
by pablohoffman 15y ago
The title of this post is misleading, it makes you think GitHub is actually broken, when the article is about complaining how their fork feature works.
18.
by pablohoffman 15y ago
I initially submitted this post, but then deleted it and resubmitted to the original post on Common Crawl blog: http://news.ycombinator.com/item?id=3208853 I now regret since this one got much more attention. I was under the impression th
19.
by pablohoffman 15y ago
I found strange that no one mentioned Privnote ( https://privnote.com ), a much older/popular service that does this but more securely since it generates URLs with keys in the fragment to encrypt the messages on client side (so that the pla
20.
Common Crawl releases free 5 Billion Page Web Index(commoncrawl.org)
6 points by pablohoffman 15y ago|0 comments
21.
by pablohoffman 15y ago
Did it work?
22.
by pablohoffman 15y ago
I've been using tiled window managers for 7 years now (currently on awesome, which I switched 3 years ago) and the interface has always remained pretty consistent. I wonder if Linus has tried tiled WMs?
23.
by pablohoffman 16y ago
After checking a few registrars, I found they only sell third-level domains (.com.ng, .net.ng), not second-level domains (like "amazi.ng") which is probably what most people here would be interested in. Example: http://www.webdomains.com.n
24.
by pablohoffman 16y ago
Someone already wrote a Delicious scraper in Scrapy (Python): http://news.ycombinator.com/item?id=2015680
25.
by pablohoffman 16y ago
Feeling sad would imply that you hold some respect to the judges that decided who won, but I think Time magazine just lost the respect of many readers. I'm kinda sad for them , not for Assange.
26.
by pablohoffman 16y ago
That reminded me of a recent Paul Graham tweet: You've found market price when buyers complain but still pay. http://twitter.com/#!/paulg/status/22576762202
27.
Privnote gets EuroPriSe certification(blog.insophia.com)
1 points by pablohoffman 16y ago|0 comments
28.
by pablohoffman 16y ago
Primary source thread: http://news.ycombinator.com/item?id=1849274
29.
Interop returns 16 million IPv4 addresses to ARIN(arin.net)
6 points by pablohoffman 16y ago|5 comments
30.
by pablohoffman 16y ago
Is it me, or Zed is starting to get annoyed at the Python community now?. I wonder when he'll write "Python is a Ghetto".

More ›