Good news on most fronts.
Paul Graham said that my indexing didnt slow down the server, which was a big relief. The slowness I experienced was a result of him throttling my IP address, which means he set a limit on how much data I could download. The limits are in place to discourage people from crawling the site, which if done en masse, could have a significant impact on Hacker News.
The post received mostly positive feedback other than the occasional “lol you idiot”. I expected a lot more of these. It reflects really well on the Hacker News community that there were so few of them. On any other popular news aggregator, there probably would have been a lot more trollish commentary. So, thank you HN.
I hesitated at first about writing anything. Maybe it’d be better just to ignore it and move on. I decided to consult the authority on these things: my lovely wife. I explained to her what happened. She laughed, did this wave motion with her hands and said “Smooooth”. She thought it was hilarious and told me to post it. “But its Hacker News!” I told her. She rolled her eyes. Women.
Anyway, I started analyzing the data from the 73% of submissions that I was able to log before I got throttled (sounds terrible every time I write it). As expected, there is a lot of interesting information to be gleaned, which I’ll share shortly.
Thanks for reading –