troels

28 ms·

1. 301.▲
   by troels 12y ago

   Is it using a US-english dictionary or does it have both UK-en and US-en words?
2. 302.▲
   by troels 12y ago

   Cool. Some of the words are very exotic, but then some are surprisingly common words. I wonder if there exists an index with a "commonality rating" that could be used to sort/weight the randomiser with, so that one could get
3. 303.▲
   by troels 12y ago

   What would be MS' particular beef with dx.com?
4. 304.▲
   by troels 12y ago

   It's not exactly a common operation either, so basing the choice of rdbms on it seems a bit arbitrary.
5. 305.▲
   by troels 12y ago

   Yeah. And since they use ActiveRecord, how would they even go about doing that in the first place? Over all the years I've used MySql, that has never been an issue for me. Sure it has its quirks, but show me a mature piece of software
6. 306.▲
   by troels 12y ago

   Yeah, this. Or phantom.js is nice too. Or just use a plugin for your browser, that does a screen dump of the full page. Also, I then do some afterprocessing in imagemagick.
7. 307.▲
   by troels 12y ago

   Really? Is that true?
8. 308.▲
   by troels 12y ago

   Yeah, the clue is in "Server: AkamaiGHost". The site is clearly behind some sort of anti-crawl protection which decides it doesn't like me. I think it would make sense to make an exception for exactly that file though.
9. 309.▲
   by troels 12y ago

   I don't think you actually want to do that. Blocking in robots.txt will prevent Google from crawling the URL - not from indexing it. You actually want them to crawl the URL and then respond with a 404 or 410. If there are inbound lin
10. 310.▲
    by troels 12y ago

    Hmm ... $ curl --dump-header - 'http://shop.nordstrom.com/robots.txt' HTTP/1.1 403 Forbidden Server: AkamaiGHost Mime-Version: 1.0 Content-Type: text/html Content-Length: 281
11. 311.▲
    by troels 12y ago

    I tend to use HTML to create my prototypes in. I find that it's a pretty good medium for the purpose and there are plenty of developer tools around that makes it easy to work with. Plus it's easy to script.
12. 312.▲
    by troels 12y ago

    Diversity within the police department would probably help.
13. 313.▲
    by troels 12y ago

    Yes, presumably the larger corpus would tend to contain more "rare" words than the smaller. I think you may want to weigh individual terms by their inverse frequency in the full corpus or a representative sample (Look up tf-idf if
14. 314.▲
    by troels 12y ago

    I don't believe that. They would have to have some sort of software that is able to detect that you are connecting to cpanel and then act on your behalf. That is significantly more involved and more malicious than "just" inte
15. 315.▲
    by troels 12y ago

    You can always ask. If you want to do the math your self, for most companies the cost is predominantly determined by salaries. If you can figure out the number and positions of employees, you should be able to estimate it fairly accurate. F
16. 316.▲
    by troels 12y ago

    I wonder - How else would you manage uncertain information?
17. 317.▲
    by troels 12y ago

    Sure. But that still doesn't tell me exactly which arguments they take. Or give me an opportunity to debug the code when it doesn't do as I expect. I realise that it just-works (tm). It's when it doesn't, it gets problem
18. 318.▲
    by troels 12y ago

    Ryan Bates' Railscasts are great. Unfortunately he stopped producing new ones, but they are still a great resource: http://railscasts.com/
19. 319.▲
    by troels 12y ago

    I have to agree - The path helpers are very opaque. It would probably do Rails well to generate a app/helpers/path_helper.rb file with the actual implementations in them.
20. 320.▲
    by troels 12y ago

    Offload heavy task to run in the background. Use caching. Make cpu intensive processes parallelisable. (That's not a word, is it?)
21. 321.▲
    by troels 12y ago

    Corollary - some times hash tables aren't the fastest data structure. You can often replace a hash with a trie to see a considerable improvement (Caveat emptor ...)
22. 322.▲
    by troels 12y ago

    Well, taking out insurance usually comes with a cost. All I'm saying is that I think I would do the same.
23. 323.▲
    by troels 12y ago

    Regretting getting insurance just because nothing happened is pointless. Sounds like a sane choice to me.
24. 324.▲
    by troels 12y ago

    It's a good start, but I feel the UX could do with some refinement. Why is there no default notebook upon signup? Why isn't there a big fat "Add a note" button on the top of the list of notes? Why do I have to explicitly
25. 325.▲
    by troels 12y ago

    A lot of those are candy.
26. 326.▲
    by troels 12y ago

    Thanks for the snippet - make sense.
27. 327.▲
    by troels 12y ago

    I'd still rather be the driver in that scenario (Which I believe was the parent's point)
28. 328.▲
    by troels 12y ago

    I too run a crawler that visits a lot of pages, although not at a particular high frequency. We visit hundreds of sites and each site then has a custom bot that essentially has two methods: find_links and extract. The first finds more links
29. 329.▲
    by troels 12y ago

    If I understand you right, you have a lot of different data types to scrape, so essentially you have a sub-program for each data type and when a page is downloaded, you let each of these have a go at the page and emit content if it finds an
30. 330.▲
    by troels 12y ago

    Well, if you give a canonical tag out, you're really taking on the responsibility of resolving different url's, so you should make sure you do it right.