Busy making things: github, links, photos, @mc.

All DECT out

Posted: June 5th, 2007 | Author: | Filed under: Mobile | 4 Comments »

A few weeks ago my wife and I decided to replace our 5.8GHz two-handset cordless phone system. We got them for a great price but didn’t realize until after we had trashed the packaging that only one handset could be used at a time. That was a pretty big showstopper as it means one of us is tied to the corded phone in the kitchen.

While ambiently looking at phones at our local Best Buy, I came to a wonderful realization: DECT has finally made it to the US mainstream. I have friends in Europe, so I’ve known about (and been envious of) DECT for years. DECT has more in common with a GSM phone than it does a microwave. There are specs that deal with audio codecs, and there’s a Generic Access Profile which allows for at least a chance of interop between different manufacturers’ devices.

We eventually settled on the Panasonic KX-TG1034S with a total of 4 handsets, and we absolutely love them. I’ve used higher end Panasonic multi-phone systems over the years and I’ve always been happy.

Some of the things I’m particularly pleased with in our unit is the ability to edit the phonebook on one phone and synchronize it to all other phones linked to the base station. I can’t tell you how much time that will save us in the long run. I also like that I can check voicemail from any handset rather than trip over my junk in the den in order to hit the play button. I can also clear the caller ID of a missed call on one phone and that will propogate out to the other devices. These are (I believe) proprietary features that probably won’t work on a non-Panasonic system, but they’re quite nice indeed.

Like every wireless device in the US, DECT phones here run on a different frequency than everywhere else in the world (are you suprised?) They run at 1880-1900MHz in Europe and 1920-1930MHz in the US. It looks like DECT was approved at that frequency range back in 2005 but I’ve only noticed devices on the shelves for a few months now. DECT devices in the US are labeled “DECT 6.0″ which seems utterly silly given that 6.0 has nothing to do with frequency (like 2.4 and 5.8 do), it’s just simply a marketing tactic (6.0 is clearly better than 5.8).

I’m glad that we’re slowly catching up on yet another wireless standard over here and I have nothing but good things to say about our new phones.


Reason 3,287 why I hate setuptools

Posted: June 3rd, 2007 | Author: | Filed under: Open Source, Python | 5 Comments »
root@monkey:~/inst/simplejson# python setup.py install
The required version of setuptools (>=0.6c6) is not available, and
can't be installed while this script is running. Please install
 a more recent version first.

(Currently using setuptools 0.6c3 
(/usr/lib/python2.4/site-packages/setuptools-0.6c3-py2.4.egg))

isbn.erl: My first Erlang module

Posted: May 3rd, 2007 | Author: | Filed under: Open Source, Projects | 63 Comments »

For a few weeks now I’ve been tinkering with, learning about, and falling in love with Erlang. I’ve been noticing the buzz about Erlang over the past few months, but two things won me over: the Erlang video and how amazingly simple and elegant concurrency and message passing is.

For the past few weeks I’ve been reading the Erlang documentation, Joe Armstrong’s new book, reading the trapexit wiki, and lurking in #erlang on irc.freenode.net. If you’re wading in the erlang waters, I highly suggest just about every link in this wonderful roundup of beginners erlang links.

After a few weeks of reading and tinkering in the shell I decided that it was time to come up with a quick project to hone my Erlang skills. I had tinkered with ISBN validation and conversion while writing a small django application to catalog the books on the bookshelf, so I thought that was a good place to start. The Wikipedia page and my collection of ISBN links provided me with more than enough guidance on validation, check digit generation, and conversion.

I found it very easy to build this module from the ground up: start with ISBN-10 check digit generation, then use that to build an ISBN-10 validator. I did a similar thing with ISBN-13, writing the check digit generator and then the ISBN-13 validator. From there I was able to build on all four public functions to write an ISBN-10 to ISBN-13 converter as well as an ISBN-13 to ISBN-10 converter (when that is a possibility). In the process of this very simple module I ended up learning about and applying accumulators, guards, the use of case, and lots of general Erlang knowledge.

Here’s a peek at how the module works. After downloading it via google code or checking out the latest version via Subversion, the module needs to be compiled. This is easily accomplished from the Erlang shell (once you have Erlang installed of course):

mcroydon$ erl
Erlang (BEAM) emulator version 5.5.4 [source] [async-threads:0] [kernel-poll:false]

Eshell V5.5.4  (abort with ^G)
1> c('isbn.erl').
{ok,isbn}

After that we can use any of the exported functions:

2> isbn:validate_13([9,7,8,1,9,3,4,3,5,6,0,0,5]).
true

Let’s trace the execution of a simple function, check_digit_10/1. The function expects a list of 9 numbers (the first 9 numbers of an ISBN-10) and returns the check digit as either an integer or the character 'X'. The first thing that the function does is check to see if we’ve actually passed it a 9-item list:

check_digit_10(Isbn) when length(Isbn) /= 9 ->
    throw(wrongLength);

This is accomplished with a simple guard (when length(Isbn) /- 9). If that guard isn’t triggered we move on to the next function:

check_digit_10(Isbn) -> 
    check_digit_10(Isbn, 0).

This forwards our list of 9 numbers on to check_digit_10/2 (a function with the same name that takes two arguments. We’ll see in a minute that the 0 I’m passing in will be used as an accumulator. The next function does most of the heavy lifting for us:

check_digit_10([H|T], Total) ->
    check_digit_10(T, Total + (H * (length(T) + 2)));

This function takes the list, splits it in to the first item (H) and the rest of the list (T). We add to the total as specified in ISBN-10 and then call check_digit_10/2 again with the rest of the list. This tail-recursive approach seems odd at first if you’re coming from most any object oriented language, but Erlang’s function nature, strong list handling, and tail-recursive ways feel natural very quickly. After we’ve recursed through the entire list, it’s time to return the check digit (or 11 minus the total modulus 11). There’s a special case if we get a result of 10 to use the character ‘X’ instead:

check_digit_10([], Total) when 11 - (Total rem 11) =:= 10 ->
    'X';

Finally we return the result for the common case given an empty list:

check_digit_10([], Total) ->
    11 - (Total rem 11).

Feel free to browse around the rest of the module and use it if you feel it might be useful to you. I’ve posted the full source to my isbn module at my isbnerl Google Code project, including the module itself and the module’s EDoc documentation. It is released under the new BSD license in hopes that it might be useful to you.

I learned quite a bit creating this simple module, and if you’re learning Erlang I suggest you pick something not too big, not too small, and something that you are interested in to get your feet wet. Now that I have the basics of sequential Erlang programming down, I think the next step is to make the same move from tinkering to doing something useful with concurrent Erlang.


Darwin Calendar Server Status Update

Posted: April 19th, 2007 | Author: | Filed under: Open Source, Python | 6 Comments »

Wilfredo Sánchez Vega posted an update on the status of Darwin Calendar Server to the calenderserver-dev mailing list this afternoon with a status update on the project. Check the full post for details, but here’s the takeaway:

We think the server’s in pretty solid shape right now. Certainly there are still some bugs that have to be fixed before we can roll up a “1.0″ release, but the core functionality is all in place now, and we think it’s fairly useable in its current state.

There is also a preview release in the works, so keep a close eye on the mailing list and the Darwin Calendar Server site.


Forum Nokia Remote Device Access

Posted: March 30th, 2007 | Author: | Filed under: Mobile, Web Services | 682 Comments »

I’m really excited about Nokia’s new Remote Device Access program for Forum Nokia members, including free members.

A similar service has been available from Device Anywhere for some time now, but the service isn’t free (but definitely a lot cheaper than purchasing half a dozen test devices). I’m excited that Nokia have opened up a device testing service with a wide array of devices from the 5500 to the N95 to all developers including shareware and open source developers. It looks like I have 40 credits and a half hour with a device costs 2 credits, so it looks like I have the potential to test for up to 20 hours with my free Forum Nokia membership.

Here are some screenshots from the Java-based interface:

RDA List
Nokia Remote Device Access device list

RDA N95
Nokia Remove Device Access N95 information

RDA standby
Nokia Remote Device Access N95 standby screen

RDA home screen
Nokia Remote Device Access N95 home screen

RDA S60 browser
Nokia Remote Device Access S60 browser

RDA maps
Nokia Remote Device Access maps

By default the bit-depth isn’t quite the same as the device (see Heikki’s comment below), so there’s a bit of dithering and as expected there’s a slight delay, but it’s definitely the next best thing to having a device in your hands. I was a bit disoriented when I put the S60 browser in horizontal mode and continued to use it with a vertical keypad, but that’s to be expected.

I think it’s a great testing tool and can’t wait to make use of it in the future.


A mobile take on SXSW 2007

Posted: March 13th, 2007 | Author: | Filed under: Mobile, Web Services | 4 Comments »

With SXSW Interactive 2007 winding down I’ve started reflecting on SXSW from a mobile perspective. First of all I found myself using sxsw.mobi from my phone quite a bit in the beginning of the conference as it allowed me the same overview that the pocket schedule did but also allowed me to drill down in to panel details. As I had more time to research my panel selection later in the week I found myself using my annotated (analog) pocket guide more and the mobile site less.

One of the most invigorating sessions was Brian Fling’s presentation entitled Everything you wanted to know about the mobile web (but were afraid to ask). His slide deck is chock-full of information but only as much jargon as absolutely necessary. There wasn’t a lot of new information in it for me, but I think he’s doing exactly the right thing by firing up this group of alpha designers about mobile design and showing them that it’s not that hard. In fact XHTML-MP is still just XHTML and while there are lots of limitations, CSS is still CSS. He also mentioned several great resources in his presentation including the brand new .mobi mobile web developer’s guide. Also worth reading is the Mobile Web Initiative’s Mobile Best Practices document.

I also caught a mobile panel on Monday called “Mobile Application Design Challenges and Tips.” Dan Saffer took some great notes on the panel which focused on the trials and tribulations experienced when developing a mobile app. It was nice to hear that several apps were quite successful operating “off-deck” or outside of carrier portals. It was great to hear Matt Jones talk about lower level UI bits and revel in the success of ZoneTag, but my biggest takeaway from the panel was from John Poisson of radar.net. His advice was to not worry too much about details. Get it out there as quickly as you can, even if it’s simpler than you plan for it to be. Gather feedback from your users and continue to improve the product. This also seemed to jive with my takeaway from the turning projects in to revenue panel: fail early, fail often.

The other mobile panel that stood out was called There’s no Such Thing as the Mobile Web (Or Is There?) I found a great set of notes on this panel at Eran’s blog. The panel started off discussing if there was a seperate “mobile web” or not and in the end it was hard to come up with a solid answer. It is significant to note that what a user does or expects to do on a mobile device is somewhat different than what a person needs when sitting in front of a computer. Context, location, creating content and retrieving information quickly are essential. It was interesting to get several different viewpoints on the issue: Dan Applequist from the standards and carrier viewpoint, Carlo from the industry analyst side, Michael Sippey from the blogging/content creation point of view, and Dwipal Desai who is focusing on mobile for a little company called YouTube. I was fascinated at how well Six Apart know their users: They focus on richer apps on higher end devices for a service like Vox but emphasize text messaging for LiveJournal because those users tend to be younger with cheaper phones, limited data capability, but usually have unlimited messaging plans. Vodafone are also in a unique position to offer a rich environment with asynchronus communication built around SVGT. Looking forward it’s obvious that there’s a ton of potential for the mobile web (or connected rich applications, or whatever you’d like to call it) but it’s unclear exactly which path we’ll take and what it will be.

I truly hope that the topics discussed at SXSW this year will encourage these alpha designers, UI experts, and coders to take a closer look at mobile apps and push the limits of the mobile web.


PostGIS: From Multilinestring to Linestring

Posted: March 4th, 2007 | Author: | Filed under: Projects | 8 Comments »

Continuing on with my PostGIS tinkering, I’ve been further exploring getting data in and out of PostGIS. After recompiling GEOS to get the C API working so that I could start working with the new features in PostGIS 1.2.1.

One of the problems I ran in to is that gpx2shp really likes to import data as a Multilinestring, while in the simplest case of a single-track GPX file, it should really be a Linestring. After searching a bit, I came across a mention of linemerge which can turn a Multilinestring such as MULTILINESTRING((-95.235071 38.971896,-95.235076 38.971906,-95.235015 38.971848)) in to a LINESTRING(-95.235071 38.971896,-95.235076 38.971906,-95.235015 38.971848), which seems like a saner way to work with single tracks.


Properly serving a 404 with lighttpd’s server.error-handler-404

Posted: February 28th, 2007 | Author: | Filed under: Linux, Open Source, Projects, Python | 1 Comment »

The other day I was looking in to why Django‘s verify_exists validator wasn’t working on a URLField. It turns out that the 404 handler that we were using to generate thumbnails with lighttpd on the media server was serving up 404 error pages with HTTP/1.1 200 OK as the status code. Django’s validator was seeing the 200 OK and (rightfully) not raising a validation error, even though there was nothing at that location.

It took more than the usual amount of digging to find the solution to this, so I’m making sure to write about it so that I can google for it again when it happens to me next year. The server.error-hadler-404 documentation metnions that to send a 404 response you will need to use server.errorfile-prefix. That doesn’t help me a lot since I needed to retain the dynamic 404 handler.

Amusingly enough, the solution is quite clear once you dig in to the source code. I dug through connections.c and found that if I sent back a Status: line, that would be forwarded on by lighttpd to the client (exactly what I wanted!)

Here’s the solution (in Python) but it should translate to any language that you’re using as a 404 handler:

print "Status: 404"
print "Content-type: text/html"
print
print # (X)HTML error response goes here

I hope this helps, as I’m sure it will help me again some time in the future.


Packaging Python Imaging Library for maemo 3.0 (bora) and the N800

Posted: February 11th, 2007 | Author: | Filed under: Linux, Open Source, Projects | 6 Comments »

I found myself wanting to do some image manipulation in Python with the Python Imaging Library on my N800. Unfortunately PIL isn’t available in the standard repositories. Not to worry, after reading the Debian Maintainers’ Guide I packaged up python2.5-imaging_1.1.6-1_armel.deb, built against python2.5 and maemo 3.0 (bora), and installs perfectly on my N800:

Nokia-N800-51:/media/mmc1# dpkg -i python2.5-imaging_1.1.6-1_armel.deb 
Selecting previously deselected package python2.5-imaging.
(Reading database ... 13815 files and directories currently installed.)
Unpacking python2.5-imaging (from python2.5-imaging_1.1.6-1_armel.deb) ...
Setting up python2.5-imaging (1.1.6-1) ...
Nokia-N800-51:/media/mmc1# python2.5 
Python 2.5 (r25:9277, Jan 23 2007, 15:56:37) 
[GCC 3.4.4 (release) (CodeSourcery ARM 2005q3-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from PIL import Image
>>>

Feel free to take a look at the directory listing for full source, diffs, etc. You might also want to check out debian/rules to see where to use setup.py in the makefile to build and install a Python package. If anyone wants a build of this for maemo-2.0/770 please let me know. It would just take a little time to set up a maemo-2.0 scratchbox.


Nokia N800 and camera.py

Posted: February 4th, 2007 | Author: | Filed under: Linux, Mobile, Open Source, Python | 43 Comments »

Nokia N800 and camera.py

I received my Nokia N800 yesterday and have been enjoying how zippy it is compared to the 770 (which has been getting faster with every firmware upgrade).. I got a chance to play with the browser while waiting for my wife at the airport and have been poking around to see how Bora is different than Mistral and Scirocco.

One of the bigger physical differences between the 770 an N800 is the onboard camera. I haven’t set up Nokia Internet Call Invitation yet but I looked around for some camera code samples and was pleasantly suprised. Luckily Nokia is one step ahead of me with camera.py, an example program to show what the camera sees on the screen. It looks like some bindings are missing so saving an image from the camera is off limits at the moment but this is a great start.

To run the above example on your N800, grab camera.py, install Python and osso-xterm, and run camera.py from the console.

It’s time to dust off the Python Maemo tutorial and get my feet wet.

Update: I’ve also been playing with the c version of the camera example code and have managed to get it running and taking pictures after building it in scratchbox and running it with run-standalone.sh ./camera.


Mapping Every airport and helipad in America

Posted: January 24th, 2007 | Author: | Filed under: Projects, Python, Web Services | 3 Comments »

All the airports

After stumbling upon Transtats again today I took my semi-anual visit to the FAA data and statistics page to see if there was anything new to play with. The unruly passenger count still looks like it’s down for 2006 but I was really interested in playing with the airport data that I’ve seen before.

After a little help from Python’s CSV module and some helper functions from geopy, I whipped up a 4 meg KML file for use with Google Earth or anything else that can import KML. Be warned thought that the file contains some 20,000 airports, helipads, and patches of dirt that can lead to some rendering bugs. If you’re interested, here’s the code that generated the KML.


From GPX to PostGIS

Posted: January 20th, 2007 | Author: | Filed under: Open Source, Projects | 51 Comments »

Now that I have a RELAX NG Compact schema of GPX, it’s time to figure out how to get my data in to PostGIS. so I can do geospatial queries. I installed PostGIS on my Ubuntu box with these instructions. If I recall correctly, the latest version didn’t work but the previous release did.

GPX makes a great GPS data interchange format, particularly because you can convert just about anything to GPX with GPSBabel. I’m not aware of anything that can convert straight from GPX to PostGIS, but it is a relatively straightforward two-part process.

The first order of business is converting our GPX file to an ESRI Shapefile. This is easiest done by using gpx2shp, a utility written in C that is also available in Ubuntu. Once gpx2shp is installed, run it on your GPX file: gpx2shp filename.gpx.

Once we have a shape file, we can use a utility included with PostGIS to import the shapefile in to postgres. The program is called shp2pgsql, and can be used as such: shp2pgsql -a shapefile.shp tablename. I tend to prefer the -a option which appends data to the table (as long as it’s the same basic type of the existing data). shp2pgsql generates SQL that you can then pipe in to psql as such: shp2pgsql -a shapefile.shp tablename | psql databasename.

Once the data is in Postgres (don’t forget to spatially-enable your database, you can query it in ways that I’m still figuring out. A basic query like this will return a list of lat/lon pairs that corresponds to my GPS track: select asText(the_geom) from tablename;. I’m llooking forward to wrapping my head around enough jargon to do bounding box selects, finding things nearby a specific point, etc.

The examples in the PostGIS documentation seem pretty good, I just haven’t figured out how to apply it to the data I have. I feel like mastering GIS jargon is one of the biggest hurdles to understanding PostGIS better. Well, that and a masters degree in GIS wouldn’t hurt.


Arduino serial communication with Python

Posted: January 18th, 2007 | Author: | Filed under: Projects, Python | 8 Comments »

The Arduino is here!

I got my shiny Arduino yesterday. The first order of business (after the obligatory “Hello World” led_blink sketch) was interfacing Arduino with my language of choice, Python.

Googling around for python serial led me to pySerial, a cross-platform serial library. I was actually quite suprised that such a wrapper didn’t exist in the Python Standard Library. Nevertheless, I plodded on.

The first order of business was symlinking the default device for the Arduino serial drivers on my mac (for sanity):
sudo ln -s /dev/tty.usbserial-LOTSOFCHARSANDNUMBERS /dev/tty.usbserial. From there I fired up the Python shell and ran the serial hello world sketch on my Arduino:

>>> import serial
>>> ser = serial.Serial('/dev/tty.usbserial', 9600)
>>> while 1:
...     ser.readline()
'1 Hello world!\r\n'
'2 Hello world!\r\n'
'3 Hello world!\r\n'

Writing from Python to Arduino is simple too. Load serial_read_blink and do the following from Python:

>>> import serial
>>> ser = serial.Serial('/dev/tty.usbserial', 9600)  
>>> ser.write('5')

Hooray, it worked! Communicating with the Arduino over serial with Python (just like every other language) is a pretty trivial process.


All I want to do is convert my schema!

Posted: January 16th, 2007 | Author: | Filed under: Django, Java, Open Source, Projects, Python, Web Services | 53 Comments »

I’m working on a django in which I want to store GPS track information in GPX format. The bests way to store that in django is with an XMLField. An XMLField is basically just a TextField with validation via a RELAX NG Compact schema.

There is a schema for GPX. Great! The schema is an XSD though, but that’s okay, it’s a schema for XML so it should be pretty easy to just convert that to RELAX NG compact, right?

Wrong.

I pulled out my handy dandy schema swiss army knife, Trang but was shocked to find out that while it can handle Relax NG (both verbose and compact), DTD, and an XML file as input and even XSD as an output, there was just no way that I was going to be able to coax it to read an XSD. Trang is one of those things (much like Jing that I rely on pretty heavily that hasn’t been updated in years. That scares me a bit, but I keep on using ‘em.

With Trang out of the picture, I struck out with various google searches (which doesn’t happen very often). the conversion section of the RELAX NG website. The first thing that struck my eye was the Sun RELAX NG Converter. Hey, Sun’s got it all figured out. I clicked the link and was somewhat confused when I ended up at their main XML page. I scanned around and even searched the site but was unable to find any useful mention of their converter. A quick google search for sun “relax ng converter” yielded nothing but people talking about how cool it was and a bunch of confused people (just like me) wondering where they could get it.

At this point I was grasping at straws so I pulled up The Internet Archive version of the extinct Sun RELAX NG Converter page. That tipped me off to the fact that I really needed to start tracking down rngconf.jar. A google search turned up several Xdoclet and Maven cvs repositories. I grabbed a copy of the jar but it wouldn’t work without something called Sun Multi-Schema XML Validator.

That’s the phrase that pays, folks.

A search for Sun “Multi-Schema XML Validator” brought me to the java.net project page and included a prominent link to nightly builds of the multi-schema validator as well as nightly builds of rngconv. These nightly builds are a few months old, but I’m not going to pick nits at this point.

After downloading msv.zip and rngconv.zip and making sure all the jars were in the same directory I had the tools I needed to convert the XSD in hand to RELAX NG Compact. First I converted the XSD to RELAX NG Verbose with the following command: java -jar rngconv.jar gpx.xsd > gpxverbose.rng. That yielded the following RELAX NG (very) Verbose schema. Once I had that I could fall back to trusty Trang to do the rest: trang -I rng -O rnc gpxverbose.rng gpx.rng. It errored out on any(lax:##other) so I removed that bit and tried again. After a lot more work than should have been required, I had my RELAX NG Compact schema for GPX.

My experience in finding the right tools to convert XSD to RELAX NG was so absurd that I had to write it up, if only to remind myself where to look when I need to do this again in two years.


iPhone and the new mobile web

Posted: January 10th, 2007 | Author: | Filed under: Apple, Linux, Mobile | 17 Comments »

There are a few things that excite me about the newly announced iPhone (and a few details that I’m unclear on). One thing I’m pretty sure of is that the coming of the iPhone is the tipping point for a new kind of mobile web.

I call it a tipping point more than trailblazing because there are already several devices out there that are just waiting for a new mobile web. These are fairly robust devices that have “real” browsers in them. These browsers are fully capable of reading the desktop web, but just because they can doesn’t mean it’s ideal. I’m specifically talking about Nokia’s internet tablets (the 770 and N800), current and upcoming mobile phones from Nokia that include the S60 browser, other mobile devices with advanced versions of Opera and even non-mobile gaming devices such as the Nintendo Wii.

So what about this new mobile web thing? It’s quite simple. These devices fall somewhere in between the desktop web and the traditional mobile web. It’s sort of a Goldielocks and the three bears situation. Yes, you can view the regular desktop web on this new generation of devices, but you end up having to zoom in to actually read anything or blow everything in to a single column layout (killing the design in the process) if you’re going to read longer passages. The opposite is true when viewing traditional mobile web pages on this new breed of device: the sites look stripped down and barebones in a stark “All we want are the facts, ma’am” way.

There is a middle ground between the desktop and traditional mobile web just waiting to thrive on these new devices. These new sites will feature more visually rich content and design from the desktop side but will also contain the immediacy and usefulness of the traditional mobile web.

Developing for the mobile web has never been easy, and I don’t think it’s going to become as simple as creating a myspace profile any time soon, but there are a few things now and on the horizon that are going to make it a lot easier. First of all, there are more mobile browsers supporting more aspects of CSS and Javascript. Specifically the S60 browser, the versions of Opera on many of these new devices, and the version of Safari on the iPhone all support CSS and Javascript quite well. It is also quite nice that while the Nokia N95 and the Apple iPhone are very different devices, they both have at their core the same basic rendering engine, KHTML. The specifics may be different (Safari vs. S60 browser) but I’m hoping that the core rendering behavior remains relatively consistent across these devices. Writing for the iPhone, S60 browser, and Opera will hopefully become as (relatively) easy as making sure a site renders properly in Firefox, Safari, and Internet Explorer. I definitely think that making the same site look good with KTHML and Opera is going to be a lot easier than making a site look good in Safari and IE.

Welcome to the tipping point, folks. It’s going to be a helluva ride.


My mail setup: Postifx, Dovecot, PostgreSQL, SASL, TLS, Amavis, SpamAssassin, and Postgrey on Ubuntu

Posted: December 23rd, 2006 | Author: | Filed under: Web Services | 4 Comments »

Part of moving from several various hosting locations to one was figuring out my mail setup. I had originally planned to manage mail with a control panel provided by my VPS provider. The 1U server that I had co-located in Maryland for several years is still sitting in the middle of the den so I figured that the easier mail was to manage the more likely I would be to get off my butt and manage it. It turns out that there were some bugs with Ubuntu and the version of the control panel I was using, so I asked the VPS provider to reinstall a fresh clean copy of Ubuntu Server and I’d take it from there.

I have to say that I’ve been doing this Linux thing for some time now (remember downloading A, AP, D, K, etc disk sets?) and it seems like setting up a good mail server still one of the most tedious things to do, but boy does it feel good when you’re done. After quite a bit of research I settled on a virtual mailbox stack built on Postfix and Dovecot.

I’ve found that setting up a mail server is best done in pieces. Configure something, test to make sure that it works, add another piece, break it, fix it, test it again. I started out my setup with a basic Postfix installation as described on the Ubuntu wiki. Once that was working I moved to a virtual mailbox setup with flatfiles but eventually ditched that for a PostgreSQL setup as described at L’Xtreme after trying to get PAM and SASL authentication working with my flatfile setup. If you’re looking to start from scratch with a postfix setup using virtual mailboxes I would highly recommend the L’Xtreme setup.

The only snag I ran in to with the L’Xtreme instructions was generating CRYPTed passwords. I ended up using htpasswd from the apache2-utils package in Dapper Drake. Setting both auth_debug = yes and auth_debug_passwords = yes in /etc/dovecot/dovecot.conf helped me figure out the password mismatch that was going on.

Once I had the basic setup working with TLS and SASL authentication via pam authenticating to Postgres, I set out to lock down the system against spam. The first thing I did was to set up Amavisd-new running it through SpamAssassin and several plugins. That did a pretty good job but spam dropped to near zero as soon as I installed Postgrey. I used these instructions from Debian/Ubuntu tips and tricks. I tweaked the config files to whitelist quicker and reduced the greylist to 60 seconds from the default 5 minutes (to be a little nicer to legit mail servers). I’ve also been using pflogsumm.pl to keep an eye on stats.

Like I said, setting up a mail server can be quite frustrating, but it sure is satisfying once it’s humming along.


Wii Internet Channel Drops

Posted: December 22nd, 2006 | Author: | Filed under: Web Services | 13 Comments »

Wii Internet Channel start page postneo.com on the Wii Channel Wii Internet Channel: Zooming in Google Maps on the Wii Internet Channel

The Wii Internet channel dropped early this morning. I’m rounding up some screen shots on my Wii Internet Channel flickr set.

So far it seems pretty usable. Pages are rendered in fullscreen mode (sometimes wtih just a little scrolling if the page is wide. You scroll by hitting the B (trigger) button and moving the Wii remote. Zooming in and out is as simple as hitting the + or – keys. Due to the resolution of my TV it was necessary to zoom in on just about every page I went to. I found that the predictive text interface used while entering text in form fields to be good, definitely on par with my experiences with T9 and the Nokia 770 onscreen keyboard.

Adding favorites was quite easy (just click the star while you’re on a page then click the add button). After browsing around the web and snapping some pictures I went over to Wiicade to play some flash games.

I don’t see the Wii Internet Channel becoming my primary browser any time soon but I can’t wait to see more content designed for and adapted to the Wii platform.

Update: The demo video from Opera does a really good job at taking Opera on the Wii through its paces.

Opera on the Wii identifies itself as such: Opera/9.00 (Nintendo Wii; U; ; 1309-9; en


Pardon the Dust

Posted: December 20th, 2006 | Author: | Filed under: Django, MySQL, PHP, Projects, Python, Weblogs | 7 Comments »

Sorry about the short outage there. I finally consolidated the various co-location, shared hosting, and virtual private hosting services that I was consuming every month in to one VPS account. I still have some legacy URLs to do some rewrite magic for, but the archives back to 2002 is here.

Because my new box is very Django-oriented, I am now running WordPress via PHP5 (FastCGI) and MySQL5 on lighttpd behind perlbal.

One of the things I really enjoyed about the move from WordPress on Apache with a really gnarly .htaccess file for URL rewriting to lighttpd was the simplicity of it all. Getting WordPress to “just work” for me on lighttpd was as simple as adding a 404 handler for the site:

server.error-handler-404 = "/index.php?error=404"

Everything should be smoothing out shortly and of course the eventual goal is to move this blog over to Django trunk. I did just that a few months ago but I need to revisit the code, find the importer, and give it a lot of layout love.


Web Services with JSON and Lua

Posted: October 5th, 2006 | Author: | Filed under: Web Services | 6 Comments »

I’m still not sure why but today I wrote a web services client for the Yahoo! Traffic API using JSON and Lua.

According to Wikipedia, Lua is a lightweight scripting language commonly embedded for use as an in-game scripting language. While it is most commonly used in gaming, I think it’s a simple but very powerful little scripting language.

While exploring Lua a bit I stumbled upon the socket library and decided to couple that with a JSON parser. The short and sweet program checks and prints out traffic information for Kansas City, Missouri in about 20 decently commented lines. Here is some example output from the program.

As with many short and sweet scripting language programs, this one relies on a few external libraries: Luasocket and JSON4Lua. The socket library uses Lua’s C bindings to work its magic but JSON4Lua (and many other extensions) are written in pure Lua. I’ve always been a sucker for a good library written in pure Python, and as such I love pure Lua extensions too.

The JSON parser and HTTP library were particularly neat to work with, as was wrapping my head around Lua tables. Here’s the bit that parses the JSON response in to a Lua table:

results = JSON.decode(r)[”ResultSet”][”Result”]

Lua tables are neat in that you can access them in dict or attribute style so the above code can be rewritten as such:

results = JSON.decode(r).ResultSet.Result

If you’d like to read up on Lua a bit more I would suggest checking out the following sites:


MobileHCI 2006 Roundup

Posted: September 16th, 2006 | Author: | Filed under: Mobile | 4 Comments »

I felt a certain disconnect between the academic world and the blogging world while trying to find coverage of MobileHCI 2006 in Espoo, Finland last week. The conference schedule [PDF] looked amazing, and I would totally have gone except for those 5000 miles between Kansas and Finland. Luckily there were a few people there blogging, snapping pics, and posting links. Please let me know if I’ve forgotten something.

While checking Technorati and Feedster for blog posts I stumbled upon a few gems that were not directly related to MobileHCI 2006. I’ll be catching up on Fabien Girardin’s thesis blog and reading about the future of Mobile HCI at ACM Queue over the next few days.