6 ms·
The Wrong Way to Get Noticed by YC
- jrockway 18y agoHow did pg find your e-mail? IP address from web log -> user account -> profile?
- a-priori 18y agoMaybe he had his indexer log into the site?
- brianlash 18y ago"A few weeks ago I emailed Paul Graham asking whether I could create a searchable database of Hacker News." PG had the email address from that first message. How he made the link between the server load and Matt's index is the real question.
- matt1 18y agoI post under the same IP that I crawled on. Wouldn't have been too hard to connect the two.
- markbao 18y agoI started to submit a lot (a TON) when I was really addicted to HN a few months ago. Received a similar message but on the topic of submitting a lot of articles from PG. Reaction was also similar, something along the lines of shitshitshitshit there goes my chance.
- gaika 18y agoHave to confess, I've got "please stop" email too (for my comments). First thing that came to mind was "there goes my chance to get funded by YC"
- deleted 18y ago[deleted]
- deleted 18y ago[deleted]
- Tichy 18y agoI think YC have better humor than that.
- icey 18y agoWell, Robert Morris is part of YC. I'm sure if anyone understands accidentally overloading a system because of programming curiosity, it's RTM.
- mattmaroon 18y agoCouldn't you just create a new HN account?
- allenbrunson 18y agothat sounds ill-advised, at best. i'd lay odds that pg pays enough attention to be able to connect the new username with the old one, if he put any effort into it.
- maxklein 18y agoThe answer is called Sleep(3000);
- Readmore 18y agoHaha I had the same problem the first time I set my Ruby crawler loose. Within 5 minutes I had crashed a "large electronics retailer's site". I'm omitting the name to protect the guilty.
- tstegart 18y agoWouldn't the guilty be you? :)
- tlrobinson 18y agoPresumably a site that can be taken down by a single simple crawler is guilty of crappy scaling.
- pg 18y agoDon't worry, you didn't slow down the server. I throttled your IP address. The server can withstand a crawler or two. The reason I ask people not to crawl the site is simply that if I let one person do it, I have to let everyone, and considering the audience here, that might mean we'd have 100 of them.
- spiralhead 18y agothe real question here, I think, is do you really have any legal recourse to stop us (not just being provocative, I actually wonder)?
- kylec 18y agoWhy not create periodic dumps of the database and allow people to glean interesting statistical data from it? You could remove any non-public information (passwords, preferences, individual up/down voting, last 2 octets of the IP, etc). It would be interesting to see what hours and what days people are active, where the visitors are coming from, what the most common words used are, etc.
- boredguy8 18y agoLike http://news.ycombinator.com/item?id=172701 http://news.ycombinator.com/item?id=172701 http://news.ycombinator.com/item?id=182374 http://news.ycombinator.com/item?id=182374 http://news.ycombinator.com/item?id=197644 http://news.ycombinator.com/item?id=197644 http://news.ycombinator.com/item?id=212491 http://news.ycombinator.com/item?id=212491 http://news.ycombinator.com/item?id=213891 http://news.ycombinator.com/item?id=213891 ? http://news.ycombinator.com/item?id=218782 http://news.ycombinator.com/item?id=218782
- henning 18y ago"You could do all sorts of interesting analysis on it… top posts, top contributors, posting frequency, etc etc." How is that interesting?
- alaskamiller 18y agoYou would be surprised: http://top.searchyc.com/ http://top.searchyc.com/
- deleted 18y ago[deleted]
- staticshock 18y agoIf you must, why not just scrape the google cache instead of the live site? http://www.google.com/search?q=cache:news.ycombinator.com/item%3Fid%3D261985 http://www.google.com/search?q=cache:news.ycombinator.com/it... At this moment, that's the latest cached item they have. So, it'll lag by a few days, but that wouldn't matter much here.
- deleted 18y ago[deleted]
- matt1 18y agoOn second thought Google cache might not be the best idea: "We're sorry... ... but your query looks similar to automated requests from a computer virus or spyware application. To protect our users, we can't process your request right now. We'll restore your access as quickly as possible, so try again soon. In the meantime, if you suspect that your computer or network has been infected, you might want to run a virus checker or spyware remover to make sure that your systems are free of viruses and other spurious software. If you're continually receiving this error, you may be able to resolve the problem by deleting your Google cookie and revisiting Google. For browser-specific instructions, please consult your browser's online support center. If your entire network is affected, more information is available in the Google Web Search Help Center. We apologize for the inconvenience, and hope we'll see you again on Google. To continue searching, please type the characters you see below:"
- staticshock 18y agointeresting. at what point do you get that message?
- matt1 18y agovery quickly. when i check it in my browser it was fine, but as soon as i started scanning with the software google blocked it. they may have recognized that the requests werent coming from a standard browser, so they flagged my ip.
- 18y ago
- pedalpete 18y agoif you want to do this kind of stuff, why not go with Yahoo BOSS or something similar. You could theoretically do some of the sorting you were hoping to do.
- sh1mmer 18y agoI think it's pretty big of you to fess up and apologise. Honesty and sincerity count. Well done. :)
- hooande 18y agoPaul is really cool man, don't worry about it. He'll probably respect the fact that you tried to make something more than anything. It's actually kind of hard to upset him. Frankly, he has a lot of money. It's going to take more than one errant script to ruin his day.
- babul 18y agoDoes having a lot of money make people harder to upset?
- khafra 18y agoAll other variables held equal, the more my money reserves and influx grow in proportion to my obligations and outflow, the more comfortable I am--with diminishing returns, of course. The part that gives money the "doesn't make you happy" reputation is that obligations usually grow as income does, and often other variables vary.
- davi 18y agoI'm starting to envy people with even-keeled dispositions the way other people envy the rich.
- mrtron 18y agoI feel left out that I haven't felt the need to crawl YC...I am merely hiding in anonymity!
- lyime 18y agoAfter reading the post and thread. I have a sudden urge to write a crawler. Obviously not on YC ;)
- tel 18y agoOf course, the real question is how you managed to not check HN for almost all of Wednesday!
- geuis 18y ago@pg I've started checking out Gnip, http://www.gnipcentral.com http://www.gnipcentral.com. Their API makes it very easy for sites to publish their data to the gnip service. They are pretty hardcore about "hit us as much as you want" to get data. Twitter, Digg, Flickr, and a number of other services are publishing to them. Might be a good way for HN to avoid getting hammered by crawlers and still let the hacker-types slurp the data.
- ltbarcly 18y agoThis server is already painfully slow. Lets not completely shut it down, mkay?