DaveDevelopment::Dave Marshall

Here’s a couple of little perl scripts I use on a day to day basis, they simply beautify the output of command line CVS client program. I’m not a perl programmer so I’m sure there are much better ways of doing this, any suggestions are welcome. I also haven’t tested either of them on anything but cygwin/winXP, cvs 1.11.17 and perl 5.8.7.

file: cvs-update

This little script simply takes the output from cvs update and removes most of the useless information and colours the remaining output, leaving something like this.

U docs/README
P docs/INSTALLM
M docs/FAQ
C docs/TODO

file: cvs-editors

This script takes the output of cvs editors and formats the output neatly in an ascii table, like this.

+--------------+---------+--------------------------+---------+
| File         | User    | Date                     | Host    |
+--------------+---------+--------------------------+---------+
| docs/README  | davem   | Mon Aug 14 10:04:56 2006 | lando   |
+--------------+---------+--------------------------+---------+
| docs/FAQ     | davem   | Mon Jul 31 14:38:12 2006 | lando   |
+--------------+---------+--------------------------+---------+
| docs/TODO    | davem   | Mon Jul 31 14:34:56 2006 | lando   |
+--------------+---------+--------------------------+---------+
| docs/INSTALL | davem   | Mon Jul 31 14:04:31 2006 | hansolo |
+--------------+---------+--------------------------+---------+

Well then, a few months down the road and I’ve hopefully made a few more improvements to DaveProxy.

First off is some modifications to the CGIProxy script, thanks to the guys over at Proxy Society. It basically allows me to control which type of files can be accessed through the proxy, based on the file extension.

I did have Squid running as a httpd accelerator for a while, but in the end I decided that the benefits were not really worth the hassle. On the other hand, I can see where this type of setup could prove very useful, particularly with other CGI scripts, just not proxy one’s.

I also started rotating the cgi-bin folder by adding a random number to the end, changing all the configuration scripts and the index page of the site every two hours. This certainly stopped a lot of hotlinking, but stopped people from adding favourites through the site and also any incoming traffic from Proxy.org. I also tried using apache to deny any requests without referers, this worked well for a short time…

The site had some major downtime recently and although I’m not entirely sure I think I figured out what happened. About three days ago, I had a massive influx of traffic, which ended with me serving thousands of 403 denied requests, which culminated in the proxy not being able to handle any real requests because apache and the proxy itself were firing that many 403’s back. I personally think these requests were mainly coming from China. Pages like this providing links to FreeBSD and not to mention all the forum pages hotlinking porn, are just abusing the site and it’s not fair on me or the genuine users. First I thought about geotargetting and deny requests, but that wouldn’t help anyway. A 403 might cost me 1kB, which doesn’t sound like much, but times that by thousands and my server would be crippled again. So why not drop it before it gets to Apache? A quick google search not only led me to sites listing chinese and korean ‘A’ class network ranges, but also to iptables scripts set to drop all traffic for them. Changed the port from SMTP to HTTP and I was in business. Even implemented this at work for our SMTP server, cut the spam in half!

Add to that some more links, a news page, a links page and a little bit of promotion here and there and that’s DaveProxy as of now.

Got an email today from 99 bottles of beer.net regarding the acceptance of my entry!

I wrote it way back in September, I think because the other PHP5 examples didn’t show much more than what could’ve been done in PHP4.x and because I was bored.

Basically there website is collection of programs that produce the lyrics to the ‘99 bottles of beer’ song. Apparently there are currently over 904 variations. This one has to be my favourite of the one’s I’ve seen, despite being generated by a script.

Just spent an hour or so tweaking the server running DaveProxy, mainly the introduction of mod_perl and a couple of little tweaks to Apache. Both together it should help keep it up during the worst periods of traffic.

I decided I’d take this course of action rather than moving over to PHProxy, although CGIProxy is very heavy on resources, using mod_perl should cut out the perl interpreter instantiation for every request and also allows easier monitoring using Monit. While PHProxy is better on the resources side of things, CGIProxy handles more features.

Depending on how well it goes over the next week, I may introduce squid as a cache between Apache and clients.

DaveProxy

DaveProxy, my new online proxy site is up and running. It uses CGIProxy, a free Perl proxy script. The design took me not long and I used GUG Tutorials for the graphics. It’ll stay up while I’ve got the bandwidth available, unless it starts making me loads of money and then I’ll keep it up.