Creating click-able hyper-links in CSV files programmatically

If you are programmatically creating CSV files intended for Microsoft Excel users, and cannot figure out how to make URL data in the CSV files behave like click-able hyper-links for your users, this tip is for you.

There is a special format in which URLs should be formatted, that is used by a number of spreadsheet applications to interpret URLs as click-able hyper-links. The format is this:

=HYPERLINK("URL")

The URL must be enclosed in quotes for the format to be interpreted properly (without which, at least, Microsoft Excel complains).

Resize images with Preview on Mac OS X

Thinking that there’s no way built into Mac OS X to resize images, I have been for a long time hunting down free third-party image resize applications. What I did not know and could not find until today is that the Preview application, which is used to preview images by default on Mac OS X, has a convenient resize feature built right into it. It is found under the Tools menu, aptly named “Adjust Size“.

One apparent gotcha that can bite you crazy about resizing using Preview is that after adjusting the size for an image, the preview of the image will shrink considerably to a size smaller than you would expect. Don’t let that mislead you. Once you save the image, Preview will re-display the image in the proper, expected size.

Short and useless commit messages

I have been guilty of writing short, pointless, and completely useless commit messages [before]. Stoked by a recent thread on StackOverflow about the worst commit message that one has ever authored, I looked into the commit log for a project I have been working on and subsequently maintaining for the last one year. I had to learn git as it had been decided that git would be used for version control for the project. Git indeed has been used since the onset of the project, but during the first three quarters working on the project, due to my unfamiliarity with working with distributed version control and shameful lack of knowledge of how git worked and a growing misperception that git is overly complicated for beginners to distributed version control, I used git sparingly (in the sense that I did not commit regularly, nor did I use any of its more than simple features) and ham-handedly. Nevertheless, skimming through the commit log from since the very initial commit, I found the following gems:

  • minor changes
  • some brief changes
  • assorted changes
  • lots and lots of changes
  • another big bag of changes
  • lots of changes after a lot of time
  • LOTS of changes. period

Of course, the ever notorious “bug fixes” is also to be found at a lot of places.

On a happier note, since then, I have come to understand and utilise the full potential of git. And I no longer blurt out when authoring commit messages.

Debugging Django applications!

When developing Django applications, there can be many times when you would want to roll up your sleeves and get your hands dirty. You may want to know, for example, why the control isn’t falling into a certain block of code, what values are being returned to the view from the browser or test client, why a certain form is bailing out on you, or any manner of possible problematic scenarios. For these and many more, there are a number of things you can do to help yourself and your code. I am going to describe a couple of those that I frequently employ.

1. Quick and dirty debugging

Yes! It is the quickest, dirtiest, and easiest form of debugging that has been around since who knows when. Like with many other programming languages, it takes the form of Python print statements for Django. It is really helpful when you are running your Django applications off of the Django development server, where the print output make their way to the console from which the development server is run.

2. Python Logging framework

The Logging framework for Python is as easy to use as they come. The use of the logging framework resembles to some extent quick and dirty debugging, but goes much further in terms of the flexibility as well as the levels of sophistication it provides. The simplest and quickest use is to import the logging module in the file carrying the code you want to debug, and use any of the available log message creation methods: debug(), info(), warning(), error(), etc. The output from these methods will make their way to the console where the development server is running.

With the default settings, however, the logged messages do not provide useful information beyond what you tell them to print. Also, you have no control over where the messages end up showing. So, say, if you are running your Django project over Apache with mod_python or mod_wsgi, you may be up a stump trying to locate where the messages go, or you may want to keep the messages aloof in a different file, but will find that the default settings for the logging framework won’t be able to lend you much room to breathe.

However, that is where the Logging framework really shines. It is configurable to a great extent. The docs for the framework give detailed information about the different nuts and bolts of the framework and the different ways in which it can be tuned. For the sake of this article, I will briefly brush over a slightly basic configuration that I use the logging framework in when debugging Django applications. I simply create a basic configuration setting for the logger, and move it into the settings.py file for the Django project. It looks like this:

import logging
logging.basicConfig(level=logging.DEBUG,
  format='%(asctime)s %(funcName)s %(levelname)-8s %(message)s',
  datefmt='%a, %d %b %Y %H:%M:%S',
  filename='/tmp/project.log', filemode='w')

Not only does this separate the log messages to the file /tmp/project.log, it also adds useful debugging information to the start of each logged message. In this particular case, the date and time, the name of the function from which the logging method was called, the logging level, and the actual message passed are displayed. All these and much more are thoroughly documented with elaborate examples in the documentation for the logging module.

3. The Python Debugger (pdb)

You may probably already have used the pdb module before for debugging Python scripts. If you have not figured out already, you can just as easily use pdb to debug your Django applications.

If you are like me, you may have got into the habit of writing unit tests prior to writing down Django views that the unit tests test. It is a wonderful habit, but it can get unnerving at times when you are writing your tests first. This is primarily because of the way the unit tests interact with your code: Once written, they run in their entirety without any form of interaction from the user. If any number of tests fail, the fact is made clear at the conclusion of the test runner. What I want to say is that there is no easy way for you to interact when the tests are being run.

With pdb, you can have a moment to sit back and take a sigh of relief. By importing the pdb module, and calling the pdb.set_trace() method right before the point where you want to start to debug your code, you can force the test runner to freeze itself and drop to the familiar, friendly pdb prompt. This helps immensely when you want to find out, for example, why a form that you are unit testing is not validating, what error messages it is receiving, what errors or outputs it should produce so that your tests can simulate those, etc. Once at the pdb prompt, you may use the usual lot of pdb commands to inspect and step through the code.

The use of pdb, however, it not restricted to unit testing. It may equally well be used when serving your Django applications over the development server. However, there is one little detail that needs to be accounted for. When you want to debug your Django applications over the development server with pdb, you must start the development server with these additional switches:

$ python -m pdb manage.py runserver --noreload

The -m pdb switch is documented in the documentation for the pdb module. Simply, it makes sure that if the program being debugged burps and crashes (either owning to an error, or when stimulated such as by calling the pdb.set_trace() method), the pdb module automatically falls flat on its face and activates the post-mortem debugger. This is very convenient, because what it means is that the friendly pdb prompt shows up, and you can dissect the code from that point onwards.

The --noreload switch to the runserver command, however, is crucial. The Django development server is designed to automatically reload the Python interpreter if there’s a crash or error of some sort, or to reread all the Django files if there has been a change in any one of those. One fallout of this default behaviour of the development server is that since the Python interpreter is reloaded, all previous context is lost, and therefore, there is no way for pdb to save face. The --noreload switch, therefore, forces the Django server to stray off of its default behaviour.

With the development server running with these switches, all you have to do is make sure you have placed calls to pdb.set_trace() method in your code where you want to break out. And that’s as easy as it gets.

I hope that what I have described finds its way into the useful bucket of my readers. For now, that’s all. I hope you do enjoy, if you did not before, debugging your Django applications after reading this article. Please, stay safe, and good bye!

Of digital cameras and photography

In our house, we had two polaroid cameras. Brother and I, being young, were never allowed to touch them, but we saw parents use them often. On every birthday party, every gathering, we’d have a lot of pictures taken and then developed — you know how it used to be and is with polariod cameras.

Then one miserable day when I was still in school, some miscreants broke into our house and stole the cameras. Ever since, we have been deprived of cameras.

We do have digital cameras in our cell phones now. Brother is obsessed with taking pictures of himself with his cell phone all the time, while dad enjoys snapping cats when they aren’t looking. And mine’s just not good enough (at slightly more than 0.3 Mpx (mega pixels), you can hardly dream about becoming a photographer). But, that’s the conundrum: these cameras are just not up to the task. Granted, I could probably invest in a sophisticated cell phone equipped with a better camera that might cost me an arm and a leg, but cell phones don’t appeal to me. Furthermore, if you are from around here, you already know the perils of keeping an expensive cell phone. And, it is not like you can separate out the camera from the cell phone without decapitating it completely.

All these years I have waited for occasions where friends would bring their cameras, to take snaps of myself. That may probably paint me as desperate, come to think of it. I have also been to some exotic places and wished dearly that I had had a cam.

Finally, last week I decided to reach out on a limb and buy a proper, standalone digital camera. A friend from Down Under, Grant, helped narrow down the choices of cameras, constraint by my self-imposed budget, to the Canon PowerShot A580, and the Nikon Coolpix L16. I nearly went for the A580, but on the day I was going to place an order for it, I had a reeking afternoon (which I may if I feel up to it blather about another time), and decided to settle on the Nikon L16, overkill features and therefore price being the reasons for turning down the A580.

I have been playing with the Nikon L16 since the last week. I am no seasoned camera owner or photographer to be able to expertly judge how well the camera fares, but I find it satisfactory. The product page is here, so you can check out the features of the camera without having to rely on my words. I have been playing with different shooting modes, exposure settings, flash settings, and whatnot. These are all novel to me, and I am enthralled by what something so small as this camera is capable of. The fruits of my toying with the camera are on display on my flickr account. The camera works seamlessly with the MacBook. The installation CD that came with the camera has a Nikon Transfer utility and a copy of Panorama Maker 4 for OS X.

As an aside, people who frequent my blog will have have noticed instantly that I have reverted back to the simple, clean theme for the blog. The previous theme was neat, if you can put it that way, and I have tried some other themes too, but I always find myself at home coming back to this theme.

Stay safe, folks!

A guide to configuring two different Django versions for development

Django 0.9x and Django 1.x branches are so far apart that a project built on top of the former, not least when it has been rolled off into production, that porting it to the 1.x branch of Django can easily escalate into a nightmare. If you have to maintain a legacy Django project — I choose to call projects built on anything before 1.x legacy –, you will find that it is easier if you maintain two different versions of Django on your development environment. How do you do that, is the quagmire this post revolves around.

I prefer Apache with mod_python to Django’s simplistic development web-server for Django projects. Initially, the task of setting up mod_python and Apache to your tastes may seem daunting. However, once you get around its not so steep curve, the difficult task becomes second nature.

I am working with Python 2.5 on a project with Django 1.0.2. Django is deployed on the system in the site-wide default directory for third-party Python modules and applications. I also have to support development for a legacy application built on Django 0.97 (a trunk snapshot from a time long ago). My requirement is simple: I want to be able to run both projects simultaneously without having to muck around tweaking configurations besides the one time when I set the projects up.

A garden-variety solution is to change the default Django directory in the Python path to point to the specific Django that is the need of the moment. This approach puts upfront a limitation of not being able to run both projects side by side. It also is annoying because it requires tap-dancing around Django directories when switching between the two versions. It is not a considerable blow to productivity, but if it can be avoided for a better, more efficient solution, there is no reason not to.

The solution I am proposing involves having the recent Django set up as the default, with the legacy Django installed in a different directory—after all, the legacy Django is the odd item of the group. I develop on an OS X environment, where I have the two Django versions set up thus:

/Library/Python/2.5/site-packages/django
/Library/Python/2.5/site-packages/django-0.97/django

You can probably tell which is which.

I have both projects deployed as virtual hosts on Apache, each running off of the root of the web-server without stepping on the feet of the other. They can easily instead be set up to, instead of the root of the web-server, serve from under a unique sub-URL path. That is mostly a matter of preference.

For each virtual host, I have assigned a domain name based off of the name of the project. I should emphatically point out that these domains are set up to resolve to localhost on my system. They may likely have real world twins that are routable over the Internet, but for all intents and purposes, they are local to my development environment. I have done it so, partly out of convenience, and partly because one of the projects hinges on subdomain-based user-to-account mapping (what this means, simply, is that registered users on the project are assigned different accounts that directly relate to subdomains, and can log in to the project only via their respective subdomains) for proper functioning. For the latter, the domain based approach was inevitable.

With that in mind, here are the virtual host settings for the two projects

<VirtualHost *:80>
   ServerAdmin root@localhost
   ServerName projectone.com
   <Location "/">
      SetHandler python-program
      PythonHandler django.core.handlers.modpython
      SetEnv DJANGO_SETTINGS_MODULE projectone.settings
      PythonPath "['/Users/ayaz/Work/',
            '/Users/ayaz/Work/projectone/'] + sys.path"
      PythonDebug On
   </Location>
</VirtualHost>

<VirtualHost *:80>
   ServerAdmin root@localhost
   ServerName legacyproject.com
   <Location "/">
      SetHandler python-program
      PythonHandler django.core.handlers.modpython
      SetEnv DJANGO_SETTINGS_MODULE legacyproject.settings
      PythonPath "['/Library/Python/2.5/site-packages/django-0.97',
            '/Users/ayaz/Work/', '/Users/ayaz/Work/legacyproject/'] + sys.path"
      PythonDebug On
        </Location>
</VirtualHost>

While each line of the above is important, the following is the highlight of this post:

   PythonPath "['/Library/Python/2.5/site-packages/django-0.97',
            '/Users/ayaz/Work/', '/Users/ayaz/Work/legacyproject/'] + sys.path"

I have effectively injected the path to Django 0.97 into Python path. What this helps achieve is that when mod_python loads the Python interpreter to serve an incoming request for the project, Python analyses its path, looking for a module named django. The first successful match is under the /django-0.97 directory, which, if I have not already lulled you into sleep, is where Django 0.97 lives.

For the curious, all possible mod_python directives are documented here.

All this is a one-time headache: you pop in a pill, and forget about it. I can now type in projectone.com or legacyproject.com in the browser to get to either project.

I should mention, still and all, that I have of late come to know of virtualenv, a Virtual Python Environment builder. It may be a more proper solution than what I have proposed, but I have not yet used it myself to say more.

I hope I was able to clearly explain what I had intended to.

Of bubbles and bitter experiences!

When it comes to procrastination (and therefore exaggeration), I stand unparalleled among the many circles I am known to be a part of. It has been a little over a year since my writing about the procedure to follow to disconnect PTCL DSL service and my undeniably firm resolve to sever off the connection, and I have only very recently finally managed to get around to getting my act together.

The exchange building, where I found the DSL office, was anything but a pleasant sight. Dilapidated, the inordinate building worn down through constant neglect over the years recalled similar sights of government offices that I had had the misfortune of being an audience to. Every wall, every floor, every desk and chair, and every roof mounted fan that appeared to be dysfunctional, I could lay my gaze on was layered with dirt and gunge, but what startled me the most was the sight of the two staffers in the small DSL room sitting on chairs that could fall apart any minute and working on two dust-covered computers lying on an old, worn-out desk. My heart sank. I was sweating from the already sizzling weather outside. The room felt hellish. A dusty, half torn portable standing fan was as close as it got to having any hope of relief in the heat. Throughout the ten minutes I had to sit in that room, except for the few moments I spent answering what was asked of me, I kept looking about, reflecting thoroughly.

It is easier if one can phase out and shield oneself from unsavoury, unpleasant circumstances, by maintaining a bubble around oneself. It is an uphill struggle keeping that bubble intact alone, for there are many places and many moments where the frailness of it becomes perceptible.

I digress. The steps I described in the post before remain the same for applying for discontinuation of service, with the exception that I did not have to surrender the equipment as it had been well over a year. Folks from their call centre called me twice the next day to enquire into my reasons for cancelling the connection.

Red Alert 2 multiplayer on Windows Vista

Finally, there’s a working patch available for Red Alert 2 that transforms the game to utilise the UDP protocol on LAN instead of IPX (which is no longer supported on Microsoft Windows Vista). The patch requires the modified file, wsock32.dll, to be moved into the root folder of the game. I can hazard a guess that the author of the patch ingeniously hacked the DLL file and overwrote the network routines to use UDP, and since the DLL is available in the same folder as the game binary, it is loaded instead of the original DLL.

I use the patch on Microsoft Windows Vista Home Basic to play Red Alert 2 with a friend on Microsoft Windows XP. He, too, has the patch deployed. Give or take a few not too bothersome glitches sometimes, the game works flawlessly. In our configuration, both the players have the patch deployed. I cannot vouch for whether a mix of UPD and IPX game clients will work.

The patch can be found here.

To toss a wrapper out the window

No matter how depressing a picture the newspaper paints every passing day without fail, despite to what alarming heights corruption in the country is reaching, notwithstanding the growing number of people who continue to die like cockroaches every day and fall pray to countless forms of acts of crimes and injustices, there is always a small fire of patriotism glowing somewhere deep within an individual. I know, and I admit, amidst all that has been happening, you can only do so much to make a difference.

It is pitifully sad to find that we live in times where the motivation to do a good deed comes from what if any personal gains can be achieved as a result in the end. This puts the phrase, the means to an end, in a shady light: if the means is ignored for a bit, what constitutes the end? The personal gains which ultimately drive the means to achieve the end, or the ideal end for which the pristine means were acted out?

Patriotism, even if a wee, is too much to ask for in the times we live. I give you that. But being responsible is not something you can opt out of. I staunchly believe that every individual should do their part in being responsible humans. But, then, I am repeatedly told that the ends do not justify the means?

Need they?

Change can only begin from within you. If you can bring yourself to justify a good deed when there is nothing in there for you in the end, that is where you start to become responsible. That is where you start to embrace change. Keeping that wrapper or empty pack of juice in your pocket and disposing off in a waste bin later on, instead of pulling down your window and throwing it out on the road, may not get you anything in return. But what you would be doing is playing your part, as a responsible citizen and good human. And playing your part is after all your moral responsibility.

I’ve long given up on talking this into people’s minds. It is almost completely hopeless. Even if I can get in, I can’t drive the point anywhere except out across the other side. However, I do have found little success in leading by example. By playing my part, and by hinting ever so slightly, I can get people to notice and to think about it. The end is marginally different from before, but even if it isn’t, I could not care less. I merely try to play my part alone, in my capacity and when and where ever I can. If only more people would think alike.

URL rewriting and WordPress

You may have noticed that WordPress by default creates and uses “Search Engine Optimized or Friendly” URLs. The raw URLs, which refer to the file on disk followed up by a train of grotesque-looking keywords and equals-signs and ampersands and question marks, are hidden from view. For WordPress, the magic comes from what is popularly known as URL rewriting. By sketching out simple or complex yet powerful rules to define what and how to rewrite, you can force the web server to completely transform URLs into clean, beautiful, search engine friendly forms.

Apache, a web server that can commonly be found to be the choice for deployment of WordPress, delegates the entire responsibility of URL rewriting to a module named mod_rewrite. URL rewriting rules can be defined globally, or on a per directory basis, which are interpreted and acted upon by mod_rewrite. In order for rules to be interpreted on a per directory basis, for which these are defined in a special configuration file that can exist within any directory, the AllowOverride setting must be enabled within Apache globally. If it isn’t, rules defined per directory will quietly be ignored.

In order for WordPress to weave its magic, both mod_rewrite and the AllowOverride setting must be enabled within Apache. This realisation dawned upon me when Asim mentioned that the two need to be enabled on Apache. On a server on which I had recently deployed WordPress for a friend, I noticed that created pages beyond the home page would give a mysterious 404. The two, as Asim gratuitously told me, were not enabled on Apache on that server which in turn were causing the 404 to pop out. I am surprised I did not stumble upon these two gotchas documented within WordPress—I may have overlooked, I am not sure.

I hope this serves to help a lost soul fumbling along a similar path.