5 ms·
I apologize in advance if this isn't the correct place to ask, but how were you able to create a page crawler like this? What does your stack look like?
by ilovezombies 15y ago
I apologize in advance if this isn't the correct place to ask, but how were you able to create a page crawler like this? What does your stack look like?
- polyfractal 15y agoThemeForest was scraped using a simple PHP script that fed the data through a regular expression to pull out the data. I used Baobab[1] to create the hierarchical nested-set model for categories. Another PHP script provides the JSON API interface. It takes category ids and spits out JSON for the client-side script to consume. The script itself is predominantly jsTree (the navigation on the left) and jQuery Masonry (the elements in the middle). I have a 512 Linode acting as the origin server, with CloudFront providing most of the static assets (gzipped scripts, css, images, etc) [1] http://www.sideralis.org/baobab/ http://www.sideralis.org/baobab/