Month: July 2007

Uploading files via newforms in Django made easy.

Posted by

7

I have already described creating custom fields with Django and shown how easy it can get. Today, I am going to show how to manage file uploading through newforms. I expended a good part of my morning and afternoon today reading up on Django and asking for help in #django on irc.freenode.net. Many thanks to the folks in #django for helping me out.

I will take the aid of code snippets as much as I can to describe the process. That way, you can not only grasp it quickly and place it all in a useful context, but can also use my code and apply it in applications.

First of all, I have Django configured on the development webserver that ships with it. Therefore, in order for Django to serve static contents, such as images, a couple of settings need be taken care of.

I have a directory, ‘static/’, in my root Django project folder in which I keep all the static content to be served. Within settings.py file, it is important to correctly define the MEDIA_ROOT and MEDIA_URL variables to point to that static/ directory. I have the two variables defined thusly:

ayaz@laptop:~$ grep 'MEDIA_' settings.py
MEDIA_ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "static/")
MEDIA_URL = 'http://192.168.1.2:8000/static/'

The structure of the static/ directory looks like the following:

ayaz@laptop:~$ ls -l static/
total 20
drwxr-xr-x 3 ayaz ayaz 4096 2007-06-28 15:34 css
drwxr-xr-x 3 ayaz ayaz 4096 2007-07-06 21:51 icons
drwxr-xr-x 3 ayaz ayaz 4096 2007-07-06 21:51 images
drwxr-xr-x 3 ayaz ayaz 4096 2007-07-21 15:57 license
drwxr-xr-x 3 ayaz ayaz 4096 2007-07-21 11:41 pictures

For Django to serve static content, it must be told via pattern(s) in URLconf at which URL path the static content is available. The following are relevant lines from urls.py (note that MEDIA_PATH is defined to point to the same directory as does MEDIA_ROOT in settings.py):

ayaz@laptop:~$ head -5 urls.py
MEDIA_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "static/")
urlpatterns = patterns('',
  (r'^static/(?P.*)$', 'django.views.static.serve',
      {'document_root': MEDIA_PATH, 'show_indexes': True}),

That is all you need to do to enable Django to serve static content. Next up is the Model that I am using here as an example. I have the model DefaulProfile, which has two fields, the last of which is of interest here. The ‘picture’ field, as you can see below, is of type models.ImageField. Note the ‘upload_to’ argument (Consult the Model API for documentation on the various Model Fields and their supported arguments). It points to ‘pictures/’ which will, when Django processes it, be appended to the end of the path saved in MEDIA_ROOT. Also note that the ‘picture’ field is mandatory.

ayaz@laptop:~$ cat models.py
from django.db import models
from django.contrib.admin.models import User

class DefaultProfile(models.Model):
  user = models.ForeignKey(User)
  picture = models.ImageField("Personal Picture", upload_to="pictures/")

Now, let’s flock on to the views. I have already described the PictureField, so I will skip it.

ayaz@laptop:~$ cat views.py
from django import newforms as forms
from django.contrib.admin.models import User
from myapp.mymodel import DefaultProfile

class PictureField(forms.Field):
  def clean(self, value):
    if not value:
      return
    file_exts = ('.png', '.jpg', '.jpeg', '.bmp',)
    from os.path import splitext
    if splitext(value)[1].lower() not in file_exts:
      raise forms.ValidationError("Only following Picture types accepted: %s"
        % ", ".join([f.strip('.') for f in file_exts]))
    return value

‘UserForm’ is a small form model class, which describes the only field I am going to use, ‘picture’. Note the widget argument.

class UserForm(forms.Form):
  picture = PictureField(label="Personal Picture", widget=forms.FileInput)

From now starts the meat of it all. Note request.FILES. When you are dealing with multipart/form-data in forms (such as in this case where I am trying to accept an image file from the client), the multipart data is stored in the request.FILES dictionary and not in the request.POST dictionary. Note that the picture field data is stored within request.FILES[‘picture’].

The save_picture() function simply retrieves the ‘filename’, checks to see if the multipart data has the right ‘content-type’, and then saves the ‘content’ using a call to the object’s save_picture_file() method (this method is documented here).

def some_view(request):

  def save_picture(object):
    if request.FILES.has_key('picture'):
      filename = request.FILES['picture']['filename']
      if not request.FILES['picture']['content-type'].startswith('image'):
        return
    filename = object.user.username + '__' + filename
    object.save_picture_file(filename, request.FILES['picture']['content'])

  post_data = request.POST.copy()
  post_data.update(request.FILES)
  expert_form = UserForm(post_data)

  if expert_form.is_valid():
    default_profile = DefaultProfile()
    save_picture(default_profile)
    default_profile.save()

If you look closely, you’ll notice I created a copy of request.POST first, and then updated it with request.FILES (the update() method is bound to dictionary objects and merges the key-value pairs in the dictionary given in its argument with the source dictionary). That’s pretty simple to understand. When an object of UserForm() is created, it is passed as its first argument a dictionary, which is usually request.POST. This actually creates a bound form object, in that the data is bound to the form. So when the method is_valid() is called against a bound form object, Django’s newforms’ validators perform field validation against all the fields in request.POST that are defined in the form definition, and raise ValidationError exception for each field which does not apparently have the right type of data in it. Now, if you remember, the picture field was created to be a mandatory field. If we didn’t merge the request.FILES dictionary with a copy of request.POST, the is_valid() method would have looked into request.POST and found request.POST[‘picture’] to be missing, and therefore, would have issued an error. This is a nasty, subtle bug, that, I’ve to admit, wasted hours trying to figure out — is_valid() is looking for the field ‘picture’ in request.POST when in reality, for multipart form-data forms, the ‘picture’ field is stored within request.FILES, but since request.FILES was never bound to the form object, and instead request.POST was, it would never find ‘picture’. So, when request.FILES and request.POST are merged together, the resultant object has all the required fields, and is_valid() correctly finds them and does not contain, provided there is no errant data in any of the fields.

I sure do hope the explanation is clear enough. Also, do take care the the “form” tag in the corresponding template has the enctype argument defined as “enctype=multipart/form-data”.

3rd blogging anniversary. Three years ago, I wrote my first …

Posted by

5

I hadn’t noticed. Today marks the passing of three tiresome years of my blogging epoch. The thought of celebrating, however modestly, the first anniversary had never crossed me, and I ended up shamelessly forgetting about it last year and year before. I may well not have remembered today either, were it not for the unexpected sidelong glance that fell on the post archive list and got stuck at “July 2004”. “July 2004” stood there, staring fiercely back at me, fuming, wanting to kill me if I didn’t notice it. It instantly made my ears stand up.

I am hardly happy over my memory, or at the fleeting nature of which. I suspect I could’ve blogged before July 7, 2004, but I have no proof to support that speculation. The oldest post I have intact dates back to July 7, 2004, which can be read here. It is a funny post. I wrote it to vent out anger and frustration that managed to bile up after the first ever semester project I did back at the University. I was a freshman then. I am a graduate now. Time flies.

In three years, I blogged decently. I maintained a balance between blogging because you love to write and blogging because you want to share information. I also kept my composure. A blog, I learned early, is not a personal diary. Anything you publish on the world wide web, perhaps, being a Security Engineer, it would do justice to say instead, anything you publish on the Internet, even when it is not published but kept seemingly hidden, is not personal. Or, it does not retain that status for long. People tend to write about everything on their blogs, from their likes to dislikes, grudges and crushes over someone, personal problems, depressing issues, joyous moments, et cetera. And it is very tempting to do so, too. But one has to remember that what they write, or more generally, make available on the Internet can be read by anyone at any time and tainted and used for any purposes, even, at times, to libel against the very person who wrote it. It is important to realise the importance and sensitive nature of what you make available on the Internet. In blogging, it is important to strike a balance between personal stuff and stuff that is OK for the public to read.

At odd times in my blogging history, I failed to maintain the balance, letting it fall to one side or the other. And, I regret it. However, all in all, what I’ve written has always been carefully screened by myself prior to getting published, and I’ve always made sure that, when droning on anything personal, I don’t let out too much, and when criticising someone or something, I don’t cross lines I am not supposed to cross.

In retrospect, I wrote about a lot of things. I have 159 posts today, the oldest dating back to July 7, 2004. However, of late, and as a good friend who tells me I inspired him to kick off his blogging career usually screams at me with a frown on his face, “You don’t write for your readers”, I’ve tipped the balance more on to the side of disseminating technical information and news. There are a couple of reasons for that, some of which I myself am not sure of. I am indulged in technical stuff more often than ever, plus with my career kicking off, I have hardly time for anything else. Another important reason, I believe, for my not writing about anything non-technical is abstinence. I am taking a lot of hits on the emotional front in my personal life, and I fear if I blog about it, I would trip over the line and go out and expose a lot of things I shouldn’t expose at all. And, no, I am not drinking, nor smoking, nor taking drugs, nor sleeping with anyone. Abstinence. If I blog, I know I will be tempted to write about it. To vent out. To cry. I avoid it, instead. Whatever little I write, it is purely technical. I regret my readers feel the need to leave the theatre a line too early, but, they’ll have to understand.

Blogging is fun. I love to write, so it have been even more fun. Blogging is a great way to fight writer’s block, too. But, again, you have to maintain a balance in what you write and what you should not write but still do out of temptation. When I look back, I can safely look at times when I badly wanted to lash out on someone, over something, but painfully resisted the urge. It was most important to contain the temptation, not only because it wouldn’t have been a nice thing to do, but also because over the years I’ve attracted a big reader base which includes people who are or may be my employees.

I don’t know what else to write. I am glad to have started blogging three years from now at this day, and I am glad to have kept blogging up till now. My blog alone has helped me in various ways. I am truly thankful.

I might celebrate quietly today. I already feel excited. Thank you very much for reading. :-)

Creating custom fields using Django’s newforms Framework

Posted by

2

I simply adore Django’s newforms framework. Creating HTML forms and processing them, both of which, as any web developer would love to tell you with a grimace on their face, are extremely tedious and monotonous tasks, have never been easier with Django.

Part of the reason I like it is that each field in the form can be easily customised further. Take an example of a field in a form I created that accepts a picture. Instead of using one of the built-in fields provided by newforms, I extended newforms.Field, and created a custom PictureField which ensures the file specified has one of a couple of extensions. It raises a validation error otherwise, which automatically gets displayed on the page if the form object is first validated and then loaded into a template.

from django import newforms as forms

class PictureField(forms.Field):
  def clean(self, value):
    if not value:
      return
    file_exts = ('.png', '.jpg', '.jpeg', '.bmp',)
    from os.path import splitext
    if splitext(value)[1].lower() not in file_exts:
      raise forms.ValidationError("Only following Picture types accepted: %s"
      % ", ".join([f.strip('.') for f in file_exts]))
  return value

Within the form, I use it thusly:

class UserForm(forms.Form):
  picture = PictureField(label="Personal Picture", required=False, widget=forms.FileInput)

It is nothing fancy, really, but, at least to me, being Pythonic all the way, it puts the fun back in web development.