Surviving Digg

There are a number of existing posts about how to survive a digging or a slashdotting. Recently I had to deal with this for bleenks.com, an online video site that a friend and I have been working on. The onslaught started at about 5pm on November the 6th, and over the next 24 hours we received over a million hits and 30k visits. So, here is the short description of how we survived. The page that was dugg was our home page. Step 1, view the source of the home page and save it to a file called cached_front.html. Step 2, add the following rules to your .htaccess file RewriteEngine on RewriteRule ^$ /cached_front.html [L] RewriteRule ^/$ /cached_front.html [L] RewriteRule ^/index.php$ /cached_front.html [L] RewriteRule ^index.php$ /cached_front.html [L] That's it. Now everyone visiting your home page is getting a cached copy. This took our server load from over 100 to less than 10. When you need to update the home page, comment out the mod rewrite rules, and update the source of the cached copy.


Developing with the Netvibes API

I've been using netvibes for some time, but only recently did I switch from Google Personal. Netvibes has several advantages over google personal including a publishing system for tabs and feeds which has really expanded my daily reading. The move to Netvibes was easy enough, but it was missing a couple of modules that Google had and I wanted, namely the "Word of the Day" and "Quotes of the Day" modules. These aren't essential to my daily activities, but they were nice to have. So, I spent a few hours on Saturday writing these replacement modules for Netvibes, you can find these modules below: The netvibes module API essentially works as follows. You create a page that conforms with xhtml standards and include the appropriate style and javascript files in the header. That's it, you now have a module. This model makes it easy to create powerful modules, however the lack of external CSS or Javascript is a limitation that will make modules potentially very large. I've managed to slow my netvibes experience down considerably by using the wrong modules. Below are links to the current available netvibes API documentation. The API Specification isn't much of a specification but it gives you a general idea of API capabilities. It's cited as being a draft and that's pretty accurate. The document goes into Javascript and CSS support, support for form submission as well as proxies for ajax use and basic module structure. The API tutorials and the API templates were the most useful documentation available. The biggest issue I ran into was that in using the template for rich lists I couldn't figure out why my list wouldn't render properly. Apparently the styles used by rich lists aren't available by emulation, and the module didn't render properly until it was submitted. This bit of documentation would have saved me several hours of tinkering. While having to reference the above documentation it occurred to me that a Wiki would be a better way of maintaining and developing the documentation. That's a change I would like to see. I really like Netvibes. A couple of things I would like to see are; Ability to update a Tab you have published via some type of a versioning system, the ability to delete something you have submitted for publishing, a request system for modules and email confirmation and approval of submissions. The above supplied code comes with no warranty, and is not supported. You should however be able to use it as a basis for other modules.


Tools every programmer needs

I've been doing software development for a little over 10 years now, and have almost always been able to use open source tools to get the job done. Below are a list of tools that I have found essential for software development. Linux If you are going to be a software developer, you need a decent operating system. Over the years I have used Solaris, OpenBSD, FreeBSD, Windows and Linux and have found Linux to have good support for all the tools I require. WMI/WindowMaker You need a usable window manager. Sometimes I use WMI (I actually still use WMI-10 and not WMII), and sometimes I use WindowMaker. Both of them offer a minimal environment that's ideal for development. I recently moved to a dual head setup, and WMI doesn't work well with it. WMI has to be my WM of choice though, it's like VIM for GUI applications. VIM I like VIM, unless I'm doing Scheme/Lisp development in which case I use emacs (better support for s-expression navigation). If you spend the time, VIM can be a very powerful IDE. With CTAGS and some customization you get all the nice things you would expect from any IDE including syntax highlighting, function/method tab completion, documentation lookup, scope knowledge, build and test integration, etc. My .vimrc is about 200 lines, plus ~8k lines of supporting custom written plugins, dictionaries and compilers. CVS/Subversion I have been using CVS for a long time, but branching and tagging are two of the most useful and needed CVS features and they are the most difficult ones to use. Slowly I'm moving to subversion because in addition to atomic operations, it has good support for branching. Bash In a Unix/Linux environment, a good shell is essential for getting any work done. My current .bashrc is about 500 lines, with tons of custom functions, aliases, completions and environment variables. Bash is a complete scripting environment. Perl Perl is great for prototyping software solutions. You can generally whip up something very quickly with Perl, see how it runs, and then implement it in the required language. It's also quite useful for one liners like removing comments, search and replace (perl -pi -e is your friend), and just about any 'glue' components you might need. If you have some discipline, perl can be readable. Doxygen Every project seems to use slightly different documentation standards. Fortunately, doxygen can read most of them. Doxygen can read most standards, deal with most languages (more than 10) and output in many different formats (HTML, Latex, Man Pages, RTF, XML, PostScript, PDF). Bug Tracking I know people like Bugzilla, but I prefer to use "Request Tracker". With pretty minimal effort you can integrate either system into your version control system, IDE and build tools. A bug tracker is an essential tool for managing change requests. Build Tools Ant, PHING and Make are the most common build tools I encounter. Ant and PHING are functionally equivalent, and I prefer them to Make. However, I use Make for common shell based builds (building my resume from latex source, building simple applications written in C/C++). I had a brief fascination with the GNU build tools (autoconf, automake, autohell), but they changed too frequently to be reliable. Test Tools Nobody likes writing tests, however there are some tools that can make it a bit more manageable. Unit testing is really popular, so I've used JUnit, Simpletest and a bunch of other unit testing tools. All of these require development input and a framework to run and manage them. My company will be open sourcing a unit testing framework called 'SPANC' in the next month or so which handles managing unit tests, automating their runs, reporting on failures/successes, etc. Of course, everyone has written custom test tools as well. These usually suck, since no one outside of the group you are working in will understand them. Freenode When I need quick help figuring out why something isn't working, I usually hop on the freenode IRC network (assuming I can't find what I'm looking for on Google). Caffeine I drink large amounts of caffeine, usually in the form of mountain dew or "energy drinks". I'm quite fond of kha0s from "Monster", it has a citrus flavor. The above are the things I use to get through day to day life as a software engineer. I don't think the list of too uncommon from what other engineers are doing, but maybe a newbie will find it useful.


Quotes for the Day

"Ultimately, peer-to-peer is about overcoming the barriers to the formation of adhoc communities, whether of people, of programs, of devices, or of distributed resources. It's about decoupling people, data, and services from specific machines, using redundancy to replace reliability of connections as the key to consistency." - Tim O'Reilly "It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them." - Steve Jobs "What makes Napster different is that it's drop-dead simple to use. Its interface isn't pretty, but it achieves that magic resonance with user expectations that marks the most revolutionary software development." - Keven Werbach