Busy making things: github, links, photos, @mc.

Kansas Primary 2008 recap

Posted: August 6th, 2008 | Author: | Filed under: Django, Journalism, Python | 1 Comment »

I’m winding down after a couple of very long days preparing for our coverage of the 2008 Kansas (and local) primaries. As always it’s been an exhausting but rewarding time. We’ve come a long way since the first election I wrote software for and was involved with back in 2006 (where election night involved someone accessing an AS/400 terminal and shouting numbers at me for entry). Our election app has become a lot more sophisticated, our data import process more refined, and election night is a whole lot more fun and loads less stressful than it used to be. I thought I’d go over some of the highlights while they’re still fresh in my mind.

Douglas County Comission 2nd District Democratic primary section

Our election app is definitely a success story for both the benefits of structured data and incremental development. Each time the app gets a little more sophisticated and a little smarter. What once wasn’t used until the night of the election has become a key part of our election coverage both before and after the event. For example, this year we had an overarching election section and also sections for indivudual races, like this section for the Douglas County Commission 2nd district Democratic primary. These sections tie together our coverage of the individual races: Stories, photos and videos about the race, our candidate profiles, any chats we’ve had with the candidates, campaign finance documents, and candidate selectors, an awesome app that has been around longer than I have that lets users see which candidates they most agree with. On election night they’re smart enough to display results as they come in.

Election results start coming in Results rolling in County commission races almost done

This time around, the newsroom also used our tools to swap out which races were displayed on the homepage throughout the night. We lead the night with results from Leavenworth County, since they were the first to report. The newsroom spent the rest of the nice swapping in one or more race on the homepage as they saw fit. This was a huge improvement over past elections where we chose ahead of time which races would be featured on the homepage. It was great to see the newsroom exercise editorial control throughout the night without having to involve editing templates.

More results

On the television side, 6 News Lawrence took advantage of some new hardware and software to display election results prominently throughout the night. I kept catching screenshots during commercial breaks, but the name of the race appeared on the left hand side of the screen with results paging through on the bottom of the screen. The new hardware and software allowed them to use more screen real estate to provide better information to our viewers. In years past we’ve had to jump through some hoops to get election results on the air, but this time was much easier. We created a custom XML feed of election data that their new hardware/software ingested continuously and pulled results from. As soon as results were in our database they were on the air.

The way that election results make their way in to our database has also changed for the better over the past few years. We have developed a great relationship with the Douglas County Clerk, Jamie Shew and his awesome staff. For several elections now they have provided us with timely access to detailed election results that allow us to provide precinct-by-precinct results. It’s also great to be able to compare local results with statewide results in state races. We get the data in a structured and well-documented fixed-width format and import it using a custom parser we wrote several elections ago.

State results flow in via a short script that uses BeautifulSoup to parse and import data from the Kansas Secretary of State site. That script ran every few minutes throughout the night and was updating results well after I went to bed. In fact it’s running right now while we wait for the last few precincts in Hodgeman County to come in. This time around we did enter results from a few races in Leavenworth and Jefferson counties by hand, but we’ll look to automate that in November.

As always, election night coverage was a team effort. I’m honored to have played my part as programmer and import guru. As always, it was great to watch Christian Metts take the data and make it both beautiful and meaningful in such a short amount of time. Many thanks go out to the fine folks at Douglas County and all of the reporters, editors, and technical folk that made our coverage last night possible.


DjangoCon!

Posted: August 1st, 2008 | Author: | Filed under: Django, Journalism | 2 Comments »

I’m a little late to the announcement party, but I’ll be attending DjangoCon and sitting on a panel about Django in Journalism with Maura Chace and Matt Waite. The panel will be moderated by our own Adrian Holovaty.

I think the panel will be pretty fantastic but I can’t help be just as terrified as my fellow panelists. I love that we’ll have both Journalist-programmers and Programmer-journalists on the panel, and I love that Django is so often the glue that brings the two together.

DjangoCon is going to be awesome.


Covering Kansas Democratic Caucus Results

Posted: February 5th, 2008 | Author: | Filed under: Django, Journalism, Projects | 1 Comment »

I think we’re about ready for caucus results to start coming in.

We’re covering the Caucus results at LJWorld.com and on Twitter.

Turnout is extremely heavy. So much so that they had to split one of the caucus sites in two because the venue was full.

Later…

How did we do it?

We gained access to the media results page from the Kansas Democratic Party on Friday afternoon. On Sunday night I started writing a scraper/importer using BeautifulSoup and rouging out the Django models to represent the caucus data. I spent Monday refining the models, helper functions, and front-end hooks that our designers would need to visualize the data. Monday night and in to Tuesday morning was spent finishing off the importer script, exploring Google Charts, and making sure that Ben and Christian had everything they needed.

After a few hours of sleep, most of the morning was spent testing everything out on our staging server, fixing bugs, and improving performance. By early afternon Ben was wrapping up KTKA and Christian was still tweaking his design in Photoshop. Somewhere between 1 and 2 p.m. he started coding it up and pretty soon we had our results page running on test data on the staging server.

While the designers were finishing up I turned my focus to the planned Twitter feed. Thanks to some handy wrappers from James, I wrote a quick script that generated a short message based on the caucus results we had, compared it to the last version of the message, and sent a post to Twitter if the message had changed.

Once results started coming in, we activated our coverage. After fixing one quick bug, I’ve been spending most of the evening watching importers feed data in to our databases and watching the twitter script send out updates. Because we’ve been scraping the Kansas Democratic Party media results all night and showing them immediately, we’ve been picking up caucuses seconds after they’ve been reported and have been ahead of everything else I’ve looked at.

Because we just recently finished moving our various Kansas Weekly papers to Ellington and a unified set of templates, it was quite trivial to include detailed election results on the websites for The Lansing Current, Baldwin City Signal, Basehor Sentinel, The Chieftain, The De Soto Explorer, The Eudora News, Shawnee Dispatch, and The Tonganoxie Mirror

While there are definitely things we could have done better as a news organization (there always are), I’m quite pleased at what we’ve done tonight. Our servers hummed along quite nicely all night, we got information to our audience as quickly as possible, and generally things went quite smoothly. Many thanks to everyone involved.


We’re hiring!

Posted: January 22nd, 2008 | Author: | Filed under: Django, Journalism, Python | 3 Comments »

Wow, the Django job market is heating up. I posted a job opening for both junior and senior-level Django developers on djangogigs just a few days ago, and it has already fallen off the front page.

So I’ll mention it again: We’re hiring! We’re growing and we have several positions open at both the junior and senior level. We’d love to talk to you if you’ve been working with Django since back in the day when everything was a tuple. We’d love to talk to you if you’re smart and talented but don’t have a lot of (or any) Django experience.

Definitely check out the listing at djangogigs for more, or feel free to drop me a line if you’d like to know more.


2008 Digital Edge Award Finalists

Posted: December 20th, 2007 | Author: | Filed under: Django, Journalism | Comments Off

The 2008 DIgital Edge Award finalists were just announced, and I’m excited to see several World Company sites and projects on there as well as a couple of sites running Ellington and even the absolutely awesome Django-powered PolitiFact.com.

At work we don’t do what we do for awards. We do it to serve our readers, tell a story, get information out there, and do it as best we can. At the same time even being nominated as finalists is quite an honor, and evokes warm fuzzy feelings in this programmer.

Here are the various World Company projects and sites that were nominated (in the less than 75,000 circulation category):

Not too shabby for a little media company in Kansas. I’m particularly excited about the LJWorld.com nominations since it hasn’t been too long since we re-designed and re-launched the site with a lot of new functionality. Scanning the finalists I also see a couple of other sites running Ellington as well as several special projects by those sites.

As someone who writes software for news organizations for a living I’m definitely going to take some time this morning to take a look at the other finalists. I’m particularly excited to check out projects from names that I’m not familiar with.


All I want to do is convert my schema!

Posted: January 16th, 2007 | Author: | Filed under: Django, Java, Open Source, Projects, Python, Web Services | 53 Comments »

I’m working on a django in which I want to store GPS track information in GPX format. The bests way to store that in django is with an XMLField. An XMLField is basically just a TextField with validation via a RELAX NG Compact schema.

There is a schema for GPX. Great! The schema is an XSD though, but that’s okay, it’s a schema for XML so it should be pretty easy to just convert that to RELAX NG compact, right?

Wrong.

I pulled out my handy dandy schema swiss army knife, Trang but was shocked to find out that while it can handle Relax NG (both verbose and compact), DTD, and an XML file as input and even XSD as an output, there was just no way that I was going to be able to coax it to read an XSD. Trang is one of those things (much like Jing that I rely on pretty heavily that hasn’t been updated in years. That scares me a bit, but I keep on using ‘em.

With Trang out of the picture, I struck out with various google searches (which doesn’t happen very often). the conversion section of the RELAX NG website. The first thing that struck my eye was the Sun RELAX NG Converter. Hey, Sun’s got it all figured out. I clicked the link and was somewhat confused when I ended up at their main XML page. I scanned around and even searched the site but was unable to find any useful mention of their converter. A quick google search for sun “relax ng converter” yielded nothing but people talking about how cool it was and a bunch of confused people (just like me) wondering where they could get it.

At this point I was grasping at straws so I pulled up The Internet Archive version of the extinct Sun RELAX NG Converter page. That tipped me off to the fact that I really needed to start tracking down rngconf.jar. A google search turned up several Xdoclet and Maven cvs repositories. I grabbed a copy of the jar but it wouldn’t work without something called Sun Multi-Schema XML Validator.

That’s the phrase that pays, folks.

A search for Sun “Multi-Schema XML Validator” brought me to the java.net project page and included a prominent link to nightly builds of the multi-schema validator as well as nightly builds of rngconv. These nightly builds are a few months old, but I’m not going to pick nits at this point.

After downloading msv.zip and rngconv.zip and making sure all the jars were in the same directory I had the tools I needed to convert the XSD in hand to RELAX NG Compact. First I converted the XSD to RELAX NG Verbose with the following command: java -jar rngconv.jar gpx.xsd > gpxverbose.rng. That yielded the following RELAX NG (very) Verbose schema. Once I had that I could fall back to trusty Trang to do the rest: trang -I rng -O rnc gpxverbose.rng gpx.rng. It errored out on any(lax:##other) so I removed that bit and tried again. After a lot more work than should have been required, I had my RELAX NG Compact schema for GPX.

My experience in finding the right tools to convert XSD to RELAX NG was so absurd that I had to write it up, if only to remind myself where to look when I need to do this again in two years.


Pardon the Dust

Posted: December 20th, 2006 | Author: | Filed under: Django, MySQL, PHP, Projects, Python, Weblogs | 7 Comments »

Sorry about the short outage there. I finally consolidated the various co-location, shared hosting, and virtual private hosting services that I was consuming every month in to one VPS account. I still have some legacy URLs to do some rewrite magic for, but the archives back to 2002 is here.

Because my new box is very Django-oriented, I am now running WordPress via PHP5 (FastCGI) and MySQL5 on lighttpd behind perlbal.

One of the things I really enjoyed about the move from WordPress on Apache with a really gnarly .htaccess file for URL rewriting to lighttpd was the simplicity of it all. Getting WordPress to “just work” for me on lighttpd was as simple as adding a 404 handler for the site:

server.error-handler-404 = "/index.php?error=404"

Everything should be smoothing out shortly and of course the eventual goal is to move this blog over to Django trunk. I did just that a few months ago but I need to revisit the code, find the importer, and give it a lot of layout love.


Acknowledging the Mobile Web with Django

Posted: July 26th, 2006 | Author: | Filed under: Django, Mobile, Python | 11 Comments »

KTKA breaking news homepageI was reading up on HowToProvideAlternateViewsForMobileDevices on the Rails wiki this morning and couldn’t help but notice how much easier it is to set up a mobile version of a Django site. At World Online we have stripped-down barebones no frills “all we want are the facts ma’am” versions of all of our sites. They prove extremely useful during KU basketball games or when you’re in downtown lawrence and want to know what restaurants are open. Since our mobile sites are just alternate templates on the same views, setup goes something like this:

In main_site.settings:

TEMPLATE_DIRS = (
'/path/to/templates/mainsite.com/',
'/path/to/templates/default/',
)

In mobile_site.settings:

from main_site.settings import *
TEMPLATE_DIRS = (
'/path/to/templates/mobile.mainsite.com/',
'/path/to/templates/default/'
) 

The first line imports all of the settings from your main site. We then overwrite the TEMPLATE_DIRS setting to point to the mobile version of our templates (and fall back to default templates if there isn’t a mobile specific version). Because every app that we write also gets a default template we can have a complete mobile site up and running by creating just one or two mobile base templates.

While Django can’t help you debate internally the “one web” versus “two webs” philosophies, it can definitely help you produce lightweight mobile-friendly content with minimum effort.


Clickable Bylines

Posted: February 18th, 2006 | Author: | Filed under: Django, Journalism, Python | 49 Comments »

Clickable bylines are the new black in online journalism according to this post and related comments at Poynter. I have to admit that I thought that this was the norm rather than the exception, since this had been the case at the Journal-World long before I arrived in Lawrence.

A few days ago Dan asked me how long it would take to whip up per-writer RSS feeds. Thanks to django’s syndication framework the answer was no time at all. Over the next couple of days and with the direction of Dan and David, we tweaked the feeds to include both per-writer and per-photographer feeds. David made it easy to set up search alerts for every time a staff member posted a story. We also updated the staff bio pages to make all of this information easier to get to.

Here is an example from a recent story by Joel Mathis:

in-story byline

If you click on Joel’s name, you’ll be taken to his bio page. If you click on Contact, you’ll be taken directly to his contact form. There’s nothing new there (for us anyway). The new stuff happens on the bio page:

Joel's bio page

The very top of every bio page contains more metainformation than you can shake a stick at. First and foremost is Joel’s number and contact form. After that we have an RSS feed of his latest stories. Following that is the search alert form that allows you to be notified every time Joel posts a story. Since Joel is such a converged guy and takes pictures too, you can check out the latest photos he has taken or subscribe to an RSS feed of those photos. You can also subscribe to that feed as a photocast in the latest iPhoto.

Joel’s bio page also also contains a short biography that makes me want to head up the street to Rudy’s every time I read it. Below the bio and mugshot is a list of recent stories by Joel and a form that lets you quickly search his stories.

I think these tools go well beyond what other news organizations are just beginning to do. At the same time there is always room for improvement, so don’t be suprised if more information is added to these pages.


mobile.kusports.com

Posted: January 28th, 2006 | Author: | Filed under: Django, Mobile, Python | 48 Comments »

Over the past few days I’ve been spending some free time and downtime tweaking mobile.kusports.com and adding a couple of really cool (IMHO) features. We send out a ton of cel phone updates during each game, but I really wanted to bring our awesome live stats to mobile devices.

The first order of business was to see how the low-tech live stats view looked on the mobile site. Since pretty much all of our templates extend a base template, it looked pretty darn good out of the box (thanks to template inheritance and some great default templates by Wilson and David). Most of the time spent on this template was to condense the stats a little bit to require less scrolling on small devices. Here is the live stats view using Opera Mini on my 6682:

mobile.kusports.com live stats

Having the live stats accessable from mobile devices is great, but it’s important to make it extremely easy for someone to get to them if they come to mobile.kusports.com while a game is in progress. The solution was to present the current score on the home page if and only if a game is in progress. This was accomplished by writing a custom template tag (which took literally a few minutes) and a tweak to the home page template:

KUSports mobile home page

Now the score and the stats are first and foremost while there’s a game on, but nowhere to be found if there isn’t. The last thing I wanted to do was to make sure that the current score (and a link to the live stats) was available from other interior pages in an unobtrusive way. I figure that if someone is reading a story or viewing photos while a game is on, they probably wouldn’t mind knowing the current score either. I decided to present the information without announcing itself quite as loudly as the score on the home page:

KUSports mobile interior page

I hope to do some more tweaking of our mobile sites here and there as free time permits, and I’ll do my best to highlight the changes here if I think they are particularly clever.


We’re Moving to Kansas!

Posted: October 31st, 2005 | Author: | Filed under: Django, Open Source, Projects, Python, Weblogs | 37 Comments »

No really, we’re moving to Kansas. I’ve accepted a position at World Online, the online division of the Lawrence Journal-World. I’ll be working on some award winning sites including LJWorld.com, lawrence.com, KUsports.com using my favorite web framework: Django.

I’m really excited about working with an awesome team of people doing some really cool stuff. And of course I’m completely stoked about working with Django on a daily basis. I’ll talk about what I’m up to when I can but there will be times when I have to keep my lips zipped. I guess now might be a good time to mention that this is my personal weblog and that views/opinions/etc expressed here are mine and do not necessarily reflect those of my employer.

Needless to say I’ve been a bit busy with getting up to speed at work and planning the move. I’ve been meaning to write this post for some time now and had to delete a completely out of date post that I had half written while in Lawrence. Blogging will probably be light until things settle out, but in the meantime keep an eye on my del.icio.us links.

Strap in, Toto!


The Emerging Django Job Market

Posted: August 31st, 2005 | Author: | Filed under: Django, Python | 11 Comments »

A company in Cincinnati, Ohio is looking for a web developer. Check out the framework they’re using:

We are developing e-commerce sites based on Python and Django framework. Experience in Python, object oriented programming, MySql, PostgreSQL, and other web programming technologies is requested. This person would also be responsible for maintaining other scripts on various websites we maintain and host. The main focus will be on e-commerce development.

It’s great to see jobs emerging for this amazing little framework that has been public for just a month and a half. This one popped up on a feedster search for django in my aggregator. I can’t wait to see where we are a year from now.


Django: Big Integer Fields

Posted: August 26th, 2005 | Author: | Filed under: Django, MySQL, Projects, Python | 6 Comments »

I submitted a patch to Django Ticket #399 (request for a bigint field type). It still needs testing but works at a quick glance on mysql. Here’s a shot of them in action from the admin interface (the input is just too small and just too big respectively):

BigInt Admin

Update: BigIntegerField works perfectly on PostgreSQL but because it doesn’t have an unsigned integer type (that I can find), PostitiveBigIntegerField isn’t going to make it all the way up to 18,446,744,073,709,551,615 without using an arbitrary precision NUMERIC or mapping zero to -9,223,372,036,854,775,808. Both solutions are messy and it would be a shame to have the mysql and postgres backends behave so differently. As an aside, it looks like this is already the case with mysql’s IntegerFields being UNSIGNED while Postgres just checks to make sure that the integers are positive before inserting.

The best solution would probably be to employ backend-specific range checking for these monsterous numbers. That way you won’t end up out of range in PostgreSQL but you’re also not penalizing MySQL for being able to count to 18 bajillion. At this point it would be safe to drop in BigIntegerField as is (as soon as I check it out on sqlite), but PostitiveBigIntegerField still needs some pondering.


Django Model Syntax Change

Posted: August 25th, 2005 | Author: | Filed under: Django, Python | 3 Comments »

Adrian has committed the Django model syntax changes. Current models will have to be rewritten — once — but the new syntax looks a lot cleaner and I think it will reduce the learning curve for newcomers to the platform.

Be sure to check out the full documentation as well as a brief screencast highlighting the changes. I will do my best to update my CRUD Generic Views tutorial as soon as possible to reflect the syntax change.

Update: Here’s the updated tasks.py


Django Generic Views: CRUD

Posted: August 17th, 2005 | Author: | Filed under: Django, Projects, Python | 100 Comments »

Note: I’ve not yet updated this to reflect the new model syntax. For the time being you can take a look at the new model syntax for tasks here.

There are lots of gems buried in Django that are slowly coming to light. Generic views and specifically the CRUD (create, update, delete) generic views are extremely powerful but underdocumented. This brief tutorial will show you how to make use of CRUD generic views in your Django application.

One of my first encounters with Rails was the simple todo list tutorial which managed to relate lots of useful information by creating a simple yet useful application. While I will do my best to point out interesting and useful things along the way, it is probably best that you be familiar with the official Django tutorials. Now would also probably be a good time to mention that this tutorial works for me using MySQL and revision 524. Django is under constant development, so things may change. I’ll do my best to keep up with changes.

Getting Started

As with all Django projects, the best place to start is to start with django-admin.py startproject todo. Make sure that the directory you created your project in is in your PYTHONPATH, then edit todo/settings/main.py to point to the database of your choice. Now would be a good time to set your DJANGO_SETTINGS_MODULE to "todo.settings.main". Next move to your apps/ dir and create a new application: django-admin.py startapp tasks and django-admin.py init. The initial setup process is covered in much more detail in tutorial 1.

The Model

Now that we have the project set up, let’s take a look at our rather simple model (todo/apps/tasks/models/tasks.py):

from django.core import meta

# Create your models here.

class Task(meta.Model):
  fields = (
    meta.CharField('title', maxlength=200),
    meta.TextField('description'),
    meta.DateTimeField('create_date', 'date created'),
    meta.DateTimeField('due_date', 'date due'),
    meta.BooleanField('done'),
  )

  admin = meta.Admin(
    list_display = ( 'title', 'description', 'create_date', 'due_date', 'done' ),
    search_fields = ['title', 'description'],
    date_hierarchy = 'due_date',
  )

  def __repr__(self):
    return self.title

The model is short and sweet, storing a title, description, two dates, and if the task is done or not. To play with your model in the admin, add the following to INSTALLD_APPS in todo/settings/main.py: 'todo.apps.tasks',

Feel free to play around with your model using the admin site. For details, see tutorial 2.

URL Configuration

Now let’s configure our URLs. We’ll fill in the code behind these URLs as we go. I edited todo/settings/urls/main.py directly, but you’re probably best off decoupling your URLs to your specific app as mentiond in tutorial 3.

from django.conf.urls.defaults import *

info_dict = {
  'app_label': 'tasks',
  'module_name': 'tasks',
}

urlpatterns = patterns('',
  (r'^tasks/?$', 'todo.apps.tasks.views.tasks.index'),
  (r'^tasks/create/?$', 'django.views.generic.create_update.create_object',
  dict(info_dict, post_save_redirect="/tasks/") ),
  (r'^tasks/update/(?P<object_id>\d+)/?$',
  'django.views.generic.create_update.update_object', info_dict),
  (r'^tasks/delete/(?P<object_id>\d+)/?$',
  'django.views.generic.create_update.delete_object',
  dict(info_dict, post_delete_redirect="/tasks/new/") ),
  (r'^tasks/complete/(?P<object_id>\d+)/?$',
  'todo.apps.tasks.views.tasks.complete'),
)

Note: I had to alter the formatting of the urlpatterns in order to make them fit. It looks a lot better in its original formatting.

We use the info_dict to pass information about our application and module to the generic view handlers . The CRUD generic views need only provide these two pieces of information, but some generic views need more. See the generic views documentation for an explanation.

Let’s look at each of these URLs one at a time, along with the code behind them.

Index

(r'^tasks/?$', 'todo.apps.tasks.views.tasks.index'),

This points to our index view, which is an index function in todo/apps/tasks/views/tasks.py:

from django.core import template_loader
from django.core.extensions import DjangoContext as Context
from django.utils.httpwrappers import HttpResponse, HttpResponseRedirect
from django.models.tasks import tasks
from django.core.exceptions import Http404

def index(request):
  notdone_task_list = tasks.get_list(order_by=['-due_date'], done__exact=False)
  done_task_list = tasks.get_list(order_by=['-due_date'], done__exact=True)
  t = template_loader.get_template('tasks/index')
  c = Context(request, {
    'notdone_tasks_list': notdone_task_list,
    'done_tasks_list': notdone_task_list,
  })
  return HttpResponse(t.render(c))

This view creates two lists for us to work with in our template, notdone_tasks_list is (not suprisingly) a list of tasks that are not done yet. Similarly, done_tasks_list contains a list of tasks that have been completed. We will use the template tasks/index.html to render this view.

Make sure that you have a template directory defined in todo.settings.main (this refers to todo/settings/main.py). Here’s mine:

TEMPLATE_DIRS = (
 "/home/mcroydon/django/todo/templates",
)

Now let’s take a look at the template that I’m using for the index:

{% if notdone_tasks_list %}
    <p>Pending Tasks:</p>
    <ul>
    {% for task in notdone_tasks_list %}
        <li>{{ task.title }}: {{ task.description }} <br/> 
          Due {{ task.due_date }} <br/> 
          <a href="/tasks/update/{{ task.id }}/">Update</a> 
          <a href="/tasks/complete/{{ task.id }}/">Complete</a>
        </li>
    {% endfor %}
    </ul>
{% else %}
    <p>No tasks pending.</p>
{% endif %}
    <p>Completed Tasks:</p>
    <ul>
{% if done_tasks_list %}
    {% for task in done_tasks_list %}
        <li>{{ task.title }}: {{ task.description }} <br/> 
          <a href="/tasks/delete/{{ task.id }}/">Delete</a>
        </li>
    {% endfor %}
    </ul>
{% else %}
    <p>No completed pending.</p>
{% endif %}
<p><a href="/tasks/create/">Add a task</a></p>

Don’t let this index scare you, it’s just a little bit of logic, a little looping, and some links to other parts of the application. See the template authoring guide if you have questions. Here’s a picture to give you a better idea as to how the above barebones template renders in Firefox:

Tasks thumbnail

Create Generic View

Now let’s take a look at the following URL pattern:

(r'^tasks/create/?$', 'django.views.generic.create_update.create_object', dict(info_dict, post_save_redirect="/tasks/") ),

There’s a lot of magic going on here that’s going to make your life really easy. First off, we’re going to call the create_object generic view every time we visit /tasks/create/. If we arrive there with a GET request, the generic view displays a form. Specifically it’s looking for module_name_form.html. In our case it will be looking for tasks_form. It knows what model to look for because of the information we gave it in info_dict. If however we reach this URL via a POST, the create_object generic view will create a new object for us and then redirect us to the URL of our choice (as long as we give it a post_save_redirect).

Here’s the template that I am using for tasks_form.html:

{% block content %}

{% if object %}
<h1>Update task:</h1>
{% else %}
<h1>Create a Task</h1>
{% endif %}

{% if form.has_errors %}
<h2>Please correct the following error{{ form.errors|pluralize }}:</h2>
{% endif %}

<form method="post" action=".">
<p><label for="id_title">Title:</label> {{ form.title }}
{% if form.title.errors %}*** {{ form.title.errors|join:", " }}{% endif %}</p>
<p><label for="id_description">Description:</label> {{ form.description }}
{% if form.description.errors %}*** {{ form.description.errors|join:", " }}{% endif %}</p>
<p><label for="id_create_date_date">Create Date:</label> {{ form.create_date_date }}
{% if form.create_date_date.errors %}*** {{ form.create_date_date.errors|join:", " }}{% endif %}</p>
<p><label for="id_create_date_time">Create Time:</label> {{ form.create_date_time }}
{% if form.create_date_time.errors %}*** {{ form.create_date_time.errors|join:", " }}{% endif %}</p>
<p><label for="id_due_date_date">Due Date:</label> {{ form.due_date_date }}
{% if form.due_date_date.errors %}*** {{ form.due_date_date.errors|join:", " }}{% endif %}</p>
<p><label for="id_due_date_time">Due Time:</label> {{ form.due_date_time }}
{% if form.due_date_time.errors %}*** {{ form.due_date_time.errors|join:", " }}{% endif %}</p>
<p><label for="id_done">Done:</label> {{ form.done }}
{% if form.done.errors %}*** {{ form.done.errors|join:", " }}{% endif %}</p>
<input type="submit" />
</form>
<!--
This is a lifesaver when debugging!
<p> {{ form.error_dict }} </p>
-->

{% endblock %}

Here’s what the create template looks like rendered:

Create Tasks

If we fill out the form without the proper (or correctly formatted) information, we’ll get an error:

Create Tasks

Update

(r'^tasks/update/(?P<object_id>\d+)/?$', 'django.views.generic.create_update.update_object', info_dict),

This URL pattern handles updates. The beautiful thing is it sends requests to tasks_form.html, so with a little logic, 90% of the form can be exactly the same as the create form. If we go to /tasks/create/, we get the blank form. If we visit /tasks/update/1/ we will go to the same form but it will be prepopulated with the data from the task with the ID of 1. Here’s the logic that I used to change the header:

{% if object %}
<h1>Update task:</h1>
{% else %}
<h1>Create a Task</h1>
{% endif %}

So if there’s no object present, we’re creating. If there’s an object present, we’re updating. Same form. Pretty cool.

Warning: It looks like form.create_date_date and form.create_date_time have broken between the time I wrote this and wrote it up. This form will not prepopulate the form with the stored information. There’s a ticket for this, and I’ll update as neccesary when it has been fixed.

Delete

Here’s the URL pattern to delte a task:

(r'^tasks/delete/(?P<object_id>\d+)/?$', 'django.views.generic.create_update.delete_object', dict(info_dict, post_delete_redirect="/tasks/new/") ),

There’s another little Django gem in the delete function. If we end up at /tasks/delete/1 using a GET request, Django will automatically send us to the tasks_form_delete.html template. This allows us to make sure that the user really wanted to delete the task. here’s my very simple tasks_form_delete.html template:

<form method="post" action=".">
<p>Are you sure?</p>
<input type="submit" />
</form>

Once this form is submitted, the actual delete takes place and we are redirected to the main index page (because we set post_delete_redirect.

CRUD Generic Views And the Rest of Your Application

That pretty much covers the basics of the CRUD generic views. The great thing about generic views is that you can use them along side your custom views. There’s no need to do a ton of custom programming for list/detail, date-based, or CRUD since those generic views are available to you. Because we set up our URL patterns, we can make sure that we craft URLs that look pretty and make sense.

For my sample tasks application I decided that I wanted to create links that would immediately set a task as complete and then redirect to the index. This is pretty much trivial and can be accomplished by adding the following URL pattern and backing it up with the appropriate view. Here’s the pattern:

(r'^tasks/complete/(?P<object_id>\d+)/?$', 'todo.apps.tasks.views.tasks.complete'),

And here’s the view that goes along with it (from todo/apps/tasks/models/tasks.py):

def complete(request, object_id):
  try:
    t = tasks.get_object(pk=object_id)
  except:
    # do something better than this
    raise Http404
  try:
    t.done=True
    t.save()
    return HttpResponseRedirect('/tasks/')
  except:
    # do something better than this
    raise Http404

I do plan to actually handle errors and respond accordingly, but it was late last night and I just wanted to see it work (and it does).

Conclusion

Django rocks. Generic views rock. The framework and specifically the generic views make your life easy. My little tasks app took a few hours to put together, but a significant portion of that was reading up on the documentation, trying to figure out generic views using the existing docs and reading the source, and of course pestering the DjangoMasters about generic views and other stuff on #django (thanks all).

I hope this overview of CRUD generic views helps, but if anything confuses you, don’t hesitate to comment or get in touch with me (matt at ooiio dot com). Also expect to see updates to this tutorial as APIs change and I get a little more time to clean up my code.

Feel free to download and play with my little todo app: todo-tutorial.tar.gz or todo-tutorial.zip. Consider them released under a BSD-style license. Above all, don’t sue me.


Rob Curley Blew My Mind

Posted: August 14th, 2005 | Author: | Filed under: Django, Python, Weblogs | 75 Comments »

A few days back I listened to this IMA Keynote by Rob Curley on IT Conversations. It blew my mind. I started out looking for some insight in to the team behind Django, The Lawrence Journal-World, lawrence.com, and so on. I got a whole lot more than I had bargained for.

If you’re involved in old media, new media, any media, or just want a glimpse of the future (I hope), please have a listen. It will be very interesting to see what happens in the next few months as Django takes flight, Rob heads to Naples (with technical backup from Eric) and Adrian telecommutes to The Washington Post


Django Markup Template Tags

Posted: August 10th, 2005 | Author: | Filed under: Django, Python | 13 Comments »

Changeset 467 contains template tags for renderingTextile (using PyTextile), Markdown (using python-markdown), and ReStructured Text (using docutils). You can find usage examples and some basic tests in markup.py.

You need to make sure that you have the correct modules installed to do the heavy lifting, but the tags are extremely easy to use:

  • {{ textile_content|textile }}
  • {{ markdown_content|markdown }}
  • {{ rest_content|restructuredtext }}

Django: Tutorial 4 and RSS

Posted: August 8th, 2005 | Author: | Filed under: Django, Python | 11 Comments »

Today Adrian posted Django Tutorial 4 which covers form processing and generic views. Everyone dive in!

In other news, if you’re looking to generate RSS using Django “the easy way,” look no further than this #django log and associated pastebin.

As always, svn up if you haven’t in the last few hours.

Update: The pastebin dissapeared, so here’s a transcribed version of what was in there. Typos are mine and not Adrian’s of course.


HTTP Headers and Django

Posted: August 8th, 2005 | Author: | Filed under: Django, Python | 8 Comments »

Andrew Brehaut has a hot tip about HTTP headers and Django:

HttpResponse objects contain a headers dictionary, that you can easily add and remove headers from using dictionary notation

You can read more about it and see usage examples in his post.


Migrating Your App to Django

Posted: August 5th, 2005 | Author: | Filed under: Django, Python | 99 Comments »

A few days ago I mentioned Changeset 384 which included a new command, django-admin.py inspectdb <dbname>. It has also been tweaked and improved since it was initially committed. The other day I tried it out on a simple database structure, but I decided to throw a more complex example at it.

I decided to take the final depot application from the excellent Agile Web Development with Rails book. Beta books rule by the way. I executed the SQL in rails-code/depot_final/db/create.sql from the the code tarball to set up the database structure. I then created a new project with django-admin startproject and edited settings/main.py to tell Django how to log in to my mysql database. After exporting the correct DJANGO_SETTINGS_MODULE I ran django-admin.py inspectdb depot_rails which gave me the following model:

# This is an auto-generated Django model module.
# You'll have to do the following manually to clean this up:
#     * Rearrange models' order
#     * Add primary_key=True to one field in each model.
# Feel free to rename the models, but don't rename 
# db_table values or field names.
#
# Also note: You'll have to insert the output of 
# 'django-admin.py sqlinitialdata [appname]'
# into your database.

from django.core import meta

class LineItem(meta.Model):
    db_table = 'line_items'
    fields = (
        meta.IntegerField('id'),
        meta.IntegerField('product_id'),
        meta.IntegerField('order_id'),
        meta.IntegerField('quantity'),
        meta.FloatField('unit_price'),
    )

class Order(meta.Model):
    db_table = 'orders'
    fields = (
        meta.IntegerField('id'),
        meta.CharField('name', maxlength=100),
        meta.CharField('email', maxlength=255),
        meta.TextField('address'),
        meta.CharField('pay_type', maxlength=10),
        meta.DateTimeField('shipped_at'),
    )

class Product(meta.Model):
    db_table = 'products'
    fields = (
        meta.IntegerField('id'),
        meta.CharField('title', maxlength=100),
        meta.TextField('description'),
        meta.CharField('image_url', maxlength=200),
        meta.FloatField('price'),
        meta.DateTimeField('date_available'),
    )

class User(meta.Model):
    db_table = 'users'
    fields = (
        meta.IntegerField('id'),
        meta.CharField('name', maxlength=100),
        meta.CharField('hashed_password', maxlength=40),
    )

Per the comment block at the top, you’re not home free yet, but at lot of tedious work has been done for you. This should definitely jumpstart the porting of existing applications to the Django platform.