DaveDevelopment::Dave Marshall

Well then, a few months down the road and I’ve hopefully made a few more improvements to DaveProxy.

First off is some modifications to the CGIProxy script, thanks to the guys over at Proxy Society. It basically allows me to control which type of files can be accessed through the proxy, based on the file extension.

I did have Squid running as a httpd accelerator for a while, but in the end I decided that the benefits were not really worth the hassle. On the other hand, I can see where this type of setup could prove very useful, particularly with other CGI scripts, just not proxy one’s.

I also started rotating the cgi-bin folder by adding a random number to the end, changing all the configuration scripts and the index page of the site every two hours. This certainly stopped a lot of hotlinking, but stopped people from adding favourites through the site and also any incoming traffic from Proxy.org. I also tried using apache to deny any requests without referers, this worked well for a short time…

The site had some major downtime recently and although I’m not entirely sure I think I figured out what happened. About three days ago, I had a massive influx of traffic, which ended with me serving thousands of 403 denied requests, which culminated in the proxy not being able to handle any real requests because apache and the proxy itself were firing that many 403’s back. I personally think these requests were mainly coming from China. Pages like this providing links to FreeBSD and not to mention all the forum pages hotlinking porn, are just abusing the site and it’s not fair on me or the genuine users. First I thought about geotargetting and deny requests, but that wouldn’t help anyway. A 403 might cost me 1kB, which doesn’t sound like much, but times that by thousands and my server would be crippled again. So why not drop it before it gets to Apache? A quick google search not only led me to sites listing chinese and korean ‘A’ class network ranges, but also to iptables scripts set to drop all traffic for them. Changed the port from SMTP to HTTP and I was in business. Even implemented this at work for our SMTP server, cut the spam in half!

Add to that some more links, a news page, a links page and a little bit of promotion here and there and that’s DaveProxy as of now.