Matt Croydon::Postneo

Year: 2006

  • Where’s Matt?

    Holux 236I’ve wanted a Bluetooth GPS device for a long time now. I know it’s a totally geeky thing, but there are so many things that I’ve wanted to do that involve getting a hard lat/long reading. Don’t get me wrong, cel tower information is nice, but nothing is better than knowing *exactly* where you are.

    I decided to spend some tax return money on a nice (but inexpensive) Bluetooth GPS unit. Jim Ley was kind enough to share what he knew about them with me in #mobitopia. After talking to Jim and examining my options, it sounded like I had to choose between feature sets in my price range. I could either have the SiRF III chipset, which by all accounts rocks, is small, low power, and extremely accurate, or I could do on-device logging.

    My decision quickly came down to either the DeLorme Bluelogger with on-device logging but a previous generation SiRFStar IIe chipset or the Holux GPSlim 236 which has the newer chip but no on-device logging.

    I ended up snagging a Holux 236 and a USB data cable for a little over a hundred bucks after shipping on ebay. I’ve since been tinkering with hooking up the GPS to both Meaning and ZoneTag to geocode my flickr photos. They’re the ones filed under geotagged. I’ve also been having fun with the NMEA info python app, Christopher Schmidt’s GPSDisplay, Microsoft Streets and Trips on my wife’s PDA, and GPSDrive on the Nokia 770.

    Now that I’ve had a chance to use it a little, it would be nice to be able to do on-device logging. It’s not the end of the world but in hindsight it would have been nice to log my path on the device and download it later rather than haiving to keep a bluetooth connection open on a second device. I’m still glad to have the latest and greatest chipset though.
    The Holux 236 has been one of the most hassle-free bluetooth devices I’ve used. It doesn’t require explicit pairing before use and getting it to work with various platforms and applications has been a breeze. I do have some trouble getting a fix to transfer from time to time, but it’s extremely well behaved by Bluetooth device standards.

    I hope to play around with this some more but also do some real stuff with it too. I can’t wait to poke at it from within Python for S60 and an open street map of Lawrence would rule. Speaking of Lawrence, you should definitely check out where Tim Hibbard is. He’s been geolocating himself around town with a nice Google Maps interface for some time now.

  • Den Archeology

    I spent part of yesterday cleaning up and organizing the den (the last holdout of the rebel packing box forces) and found quite a few things along the way. Among old papers, reciepts, notes, and general crap there were some interesting stuff. Here is a sampling:

    • 5/31/2000: Invoice for my AMD Athlon 750 CPU. That CPU in an Abit KA7-100 motherboard treated me quite well.
    • 8/05/2000: A packing slip for several Billy Pilgrim albums from the (long gone) MP3.com. I wish that I had snagged a few more before MP3.com went under.
    • 1/3/2002: Registration confirmation to watch a Steve Jobs keynote via satellite at Apple’s Northern Virginia campus.
    • 7/5/2002: My reciept for Radio Userland 8.0.8. That’s also the day that I switched my tech content from LiveJournal to my Radio blog.
    • 7/20/2002: A printout of my Amazon Web Services developer token.
    • 8/26/2002: Windows Beta product key for ITX (.NET Server RC)
    • 10/12/2002: An Airtran bording pass from BWI to Boston for the Web Services DevCon East.
    • 2/28/2003: packing slip for my newly ebay‘d Intel ISP 1100 1U server. I also found reciepts for the processor, memory, and hard drive that I stuck in it.
    • 10/03: In the margin of class notes I wrote down some thoughts on mobile wikis and mobile FOAF.
    • 11/03: Some notes on weathermob, a mobile webapp that I’ve thought about off and on again for years but have never done anything with.

    Per usual, I spent way too much time looking at stuff and not enough time actually cleaning. A trip down geeky memory lane is quite nice every once in awhile though.

  • Don’t Be Complacent

    News Designer:

    The new free Baltimore Examiner tab dropped Wednesday, with a bigger circulation than the Baltimore Sun.

    How awful must it be to wake up one morning and have your paper suddenly and abruptly be #2? It could happen to anyone, any time, and with little or no warning.

    We do our best to be painfully aware of that at the Journal-World. Dolph Simons Jr., Chairman of The World Company is quick to remind us, “No one can afford to be complacent as there always is someone who can come into town and beat you at your own business if you do not remain alert and strong.” That quote is on our about us page, though he echoes similar statements in a recent interview with The Kansan, KU‘s newspaper.

    Still, this gutsy move by The Examiner should remind the entire industry to keep on its toes.

  • Wishlist 2.0

    I’ve been contemplating writing a wishlist app off and on for a few months now but have never gotten around to doing so. While I have an Amazon wishlist, there’s a lot of stuff that I’d love to have that Amazon doesn’t sell. After finding myself keeping a seperate list and periodically e-mailing it to my wife, I though tit would be cool to be able to put together a wishlist using any item that has a URL.

    I waited too long and it looks like gifttagging has done at least 80% of what I was hoping to do. It has the web 2.0 look and feel and a tag cloud on the front page and everything. I have a feeling that I won’t actually use the service but it definitely does almost all of what I was planning to do, so if I tried to pull it off it’d be something of an also-ran.

    A couple of weeks ago I brainstormed the concept (in a rather conversational tone) with the hope of motivating myself to get started on it. That obviously didn’t happen so I thought perhaps I’d share the brainstorming session in case it’s useful to someone.

    So you have an amazon wishlist, and a wishlist with this other site, and you want some stuff that you can’t put on a wishlist. Wouldn’t it be nice if you could put all of this wishlist stuff in one spot? Cue wishlist 2.0 (or whatever it’s called). It gives you one URL you can send to your friends who want to know what you want. Of course it does stuff like pull in (and keep in synch) your Amazon wishlist, but it also works for so much more, like that doodad you want from whatsit.com. It lets you set priorities, keep notes about particular items, and it’s really easy to share with your friends. They can subscribe to an RSS/Atom feed of the stuff you want, you can send them an email linking to your wishlist, they can leave comments and OMG you can tag stuff too.

    So lets get down to some details. You sign up. Confirm your email address, cause you have to have a valid email address (even if it’s mailinator.com). After you confirm you’re sent to your “dashboard” screen. You know, the one you get every time you log in. It lists your wishlist items in whatever order you prefer (but you can reorder them). Since it’s your first time there’s a little bit at the top asking if you’d like a tour of the place, or if you’d rather, just import your shit from amazon.

    The import process is pretty painless. We’re up front about needing some information about you in order to get your wishlist from Amazon. So we get that info from you say “hang on a sec” and go grab your info using Amazon’s APIs. We come back with “Hey, so you’re John Whatshisname from Austin, TX, right? You want this, that, and the other thing. That’s you, right?”

    After we confirm that we’re not pulling in some other dude’s wishlist, we prepopulate your wishlist with the stuff from Amazon. Your quantities and ranking come over, plus everything gets tagged with “amazon”.

    If you don’t have anything to import from amazon, we take you in the other way and show you how easy it is to add items to your wishlist. All stuff needs is a URL in order for you to add it. We’ll do our best to guess what it is, but you can always override that. It gets your default “want it” value unless you override that, plus you can tag it with whatever you want “del.icio.us-style”.

    From there we can point out that “hey, your wishlist has an RSS feed. Or an Atom feed, if that’s how you roll.” You can also do other stuff like tell your friends, browse stuff from other peoples’ wishlists, or access your wishlist from a mobile phone.

    I guess the browsing and social aspect could be fleshed out a bit. Each wishlist item could be able to tell you what other people that want this want. You know, if you want a pink RAZR you might also want a fashionable bluetooth headset. Stuff like that. You can also look at the latest stuff that everyone is wishing for. If you’re on somebody elses wishlist page and you see something that they want that you also want, you can just click “I want this too” and you can add it to your wishlist.

  • Tags are for the Community, not for you

    I’m reminded of this every time I try to retrieve a bookmark from del.icio.us: tags are for the community, not for you.

    No really. Every time I go looking for a link from a few months back I search a tag that I *know* I must have tagged it with. A tag that I always tag stuff like that with.

    Nine times out of ten I forgot to use that tag, whatever it was. They’re really usless for information retireval. But they sure do help out the community.

  • S3 Service Outage

    Yesterday Amazon’s new S3 service served up nothing but service unavailable messages for nearly 7 hours.

    I give Amazon full credit for hopping on their user forums last night and letting us know that they were working on it and letting us know when it was fixed. At the same time I’m a little frustrated that such an outage occured so early on in the history of the service. The whole point of S3 is to treat storage like a utility, metered in gigabyte-hours and gigabytes of data transfered, much like you would treat your water or electricity service.

    How mad would you be if the power company turned off your power for several hours without warning, or if you woke up in the morning to find that you couldn’t take a shower? Pretty mad I imagine. I was just a little bit annoyed last night because my flickr backup wasn’t working. I couldn’t have retrieved anything from S3 if I had wanted to, but thankfully I didn’t need (or want) to.

    What if I were building out a Carson-style startup using S3 for storage? That would have been 7 hours of downtime for my app too. Hopefully the beta testers weren’t too pissed off. Hopefully I wasn’t showing a demo of it to anyone.

    Now might be a good time to read the Amazon Web Services Licensing Agreement and specifically the section on Amazon S3. You’ll note that there aren’t any guarantees about availability or uptime. You can’t count the nines in their SLA.

    I know that Amazon strives to keep S3 and their other web services up as much as possible, and over time they have done an extellent job at it. S3 is still very young and I’m sure that they’re tweaking and improving the service on the fly all the time.

    This incident is by means an indication of long term stability. Just remember that there are no guarantees.

    Update: Amazon continues to keep communication channels open and are taking strides to make sure that this doesn’t happen again. David Barth writes:

    A short note to let you know that we are taking the outage this weekend very seriously, and that once things calm down here we will post something to this thread letting you know what steps we will be taking in the future to ensure this doesn’t happen again.

    Update: David Barth gives us a more detailed update:

    We were taking the low-load Saturday as an opportunity to perform some maintenance on the storage system, specifically on some very large (>100 million objects) buckets in order to obtain better load-balancing characteristics. Normally this procedure is entirely transparent to users and bucket owners. In this case, the re-balancing caused an internal transit link to become flooded, this cascaded into other network problems, and the system was made unavailable.

    Read the full post for more on what Amazon is doing to prevert further outages.

  • Backing Up Flickr Photos with Amazon S3

    I love that I now have an Amazon S3 billing page that reads like a really cheap phone or water bill. I think that they’re silently changing the game (again) without telling anyone else. I really like the implications of this magepiebrain post and decided to start using S3 “for real” myself last night.

    The first ingredient was James Clarke’s flickr.py. Getting a list of my photos is pretty simple:

    import flickr
    me = flickr.people_findByUsername("postneo")
    photos = flickr.people_getPublicPhotos(me.id)

    The second ingredient for getting the job done was a pythonic wrapper around the Amazon example python libraries by Mitch Garnaat called BitBucket. Because it builds on the example libraries, there’s very little error checking, so be careful. Check out Mitch’s site for some example BitBucket usage, it’s pretty slick.

    Once I was familiar with both libraries, I put together a little script that finds all of my photos and uploads the original quality image to S3, using the flickr photo ID as the key. Here’s the complete code for flickrbackup.py, all 25 lines of it.

    After uploading 160 or so photos to Amazon, I owe them about a penny.

    Getting photos back out is really easy too:

    >>> import BitBucket
    >>> bucket = BitBucket.BitBucket("postneo-flickr")
    >>> bucket.keys()
    >>> bits = bucket[u'116201243']
    >>> bits.to_file("photo.jpg")

  • SNAP at Forum Nokia

    A few years ago (mid-2004 I think) I got really excited about SNAP (scalable network application package). I haven’t heard much about it since (not that I had been looking very hard), but I saw it pop up again today at Forum Nokia, and I think it’s worth a fresh look. There’s a flash site explaining the tech, but the really good stuff looks like it just hit Forum Nokia:

    I really wish that I had enough time to read all this stuff, since it’s absolutely fascinating to me. I’ll try to download a couple of these to my 770 and read them when I get a chance. It looks like some really sweet stuff has been done with SNAP while I haven’t been looking. In particular, Nitro Spin Racer reminds me of RC Pro-Am and I so want to play it. There are a couple of other games that look pretty compelling too.

  • Lighty Rocks My PHP/MySQL World

    A few weeks ago I was tasked with moving boards.kusports.com from an old, overloaded XServe to a newly appropriated Xeon box. The boards were notorious for going down after big games and during big news events and usually took other things on the same server down with them.

    Thankfully that is no more. A week or so before the Big 12 and NCAA tournaments we finally bit the bullet and committed some time to move the boards over to their own box with more horesepower and more RAM.

    I put the ubuntu-server version of Breezy Badger on the box and took the opportunity to see if I could get UBB Threads running under lighttpd and php fast-cgi. I’m glad that I took the time, because it’s smoking fast.

    The process was quite painless thanks to the Ubuntu packing system. I had to build lighttpd by hand but everything else was installable with command-line tools (aptitude/apt-get). The changeover process consisted of little more than shutting down the boards on the old box, doing a dump/restore of the database to the new server, updating DNS entries and issuing temporary redirects to get the board users through the day.

    The best thing about the conversion is that lighttpd, mysql, and php-fcgi barely register any load on the server, even at peak usage. While we were scrambling with stom coverage the other weekend, the boards box quietly registered a new high water mark for usage. After a painful first-round loss in the NCAA tournament, the boards registered yet another high water mark (1134 simultaneous users). The server barely flinched with a load anywhere from 0.00 to 0.20 with plenty of resources free. Previously a third of that number could potentially bring the XServe to its knees.

    While we were turbocharging things, I also installed eAccelerator on the box, which was equally painless and yielded about a 20-30% performance increase.

    All in all I’m gaga over lighty, mysql, and php-fastcgi for when the situation warrants it. It really is insanely fast. At the same time I’m really glad that I work in Python all day.

  • Python in Your Pocket: Python for S60

    I’m down in Dallas at PyCon 2006 recovering from a late night slide completing session for the talk I gave today: Python in Your Pocket: Python for S60. I used docutils to transform ReST in to a purdy S5 presentation (ReST source is here).

    I’ll be here through Sunday night so if you’d like to hear more, feel free to find me. I’m also going to whip together a Django lightning talk for tomorrow. While I recover a bit and take some notes, keep an eye on the pycon2006 flickr tag which I’ve been juicing with Meaning metadata.

  • Clickable Bylines

    Clickable bylines are the new black in online journalism according to this post and related comments at Poynter. I have to admit that I thought that this was the norm rather than the exception, since this had been the case at the Journal-World long before I arrived in Lawrence.

    A few days ago Dan asked me how long it would take to whip up per-writer RSS feeds. Thanks to django’s syndication framework the answer was no time at all. Over the next couple of days and with the direction of Dan and David, we tweaked the feeds to include both per-writer and per-photographer feeds. David made it easy to set up search alerts for every time a staff member posted a story. We also updated the staff bio pages to make all of this information easier to get to.

    Here is an example from a recent story by Joel Mathis:

    in-story byline

    If you click on Joel’s name, you’ll be taken to his bio page. If you click on Contact, you’ll be taken directly to his contact form. There’s nothing new there (for us anyway). The new stuff happens on the bio page:

    Joel's bio page

    The very top of every bio page contains more metainformation than you can shake a stick at. First and foremost is Joel’s number and contact form. After that we have an RSS feed of his latest stories. Following that is the search alert form that allows you to be notified every time Joel posts a story. Since Joel is such a converged guy and takes pictures too, you can check out the latest photos he has taken or subscribe to an RSS feed of those photos. You can also subscribe to that feed as a photocast in the latest iPhoto.

    Joel’s bio page also also contains a short biography that makes me want to head up the street to Rudy’s every time I read it. Below the bio and mugshot is a list of recent stories by Joel and a form that lets you quickly search his stories.

    I think these tools go well beyond what other news organizations are just beginning to do. At the same time there is always room for improvement, so don’t be suprised if more information is added to these pages.

  • Mobile Olympics

    I’m a little dissapointed that NBC isn’t featuring their mobile Olympic coverage very well, but once you get there, it’s pretty good. After watching parts of the opening ceremony last night, I hopped on the official NBC Olympics site looking for the mobile coverage that had been hinted at during the broadcast.

    The only mention of mobile on the homepage points you in the direction of the NBC Sports Mobile Store:

    NBC Mobile store blurb

    You have to dodge cheezy ringtones, wallpaper, and j2me games to get to the clearance rack in the back that I call “content.” It turns out that in order to get the secret password that lets you in to the mobile NBC Olympics site, you have to text NBCTV (that’s 62288) the message TORINO. What was that? Oh, the sound of fifty cents being drained from my wallet. I guess that’s not too bad, because they send a carrier-specific link for Sprint and Cingular, and there appears to be a generic default too. I may be a freak, but I would have preferred to type a URL in to my phone than deal with a SMS roundtrip. Oh well.

    Once I made it to the mobile site, I was presented with a simple but useful menu. On the left is the Cingular menu using the built-in browser on my 6682, on the right is the Sprint version using Opera Mini on my 6682:

    NBC Olympics Cingular NBC Olympics Sprint

    It’s interesting to note that the only real difference is the links to the carrier homepage (minor usability bonus points) and the online store link on Cingular. From the front page you can get a list of stories, check the tv schedule, and check up on the medal count. Here are some examples of the interior pages (designed for Sprint) using Opera Mini and the medal count page (designed for Cingular) using the built-in browser on my 6682:

    Sprint Mobile Story List Olympics Story Detail mobile medal count

    While the information presented on the mobile site is pretty useful, it’s a tiny fraction of the information that is on the NBC Olympics desktop site, and almost feels like a thinly veiled excuse to hock ringtones and wallpaper. I really wish that NBC were able to cram some of the in-depth event coverage on to mobile devices. For example, would it be absolutely impossible to slap a mobile template on the live luge coverage?

    NBC Luge Coverage

    To be fair, NBC has set up an email and SMS alert system that lets you receive updates about broad categories all the way down to specific events. They also have an extensive list of RSS feeds, though the few I’ve looked at are not valid and contain partial itunes information as well as thumbnails that my aggregator would pass over because it’s in an ibsys:associatedImage tag.

    I definitely have mixed feelings about the mobile coverage that NBC is doing of the Olympics. On the one hand, mobile design and development isn’t easy, and they do have some content online. On the other hand, they make it pretty hard to get to and it feels like they’re just trying to sell me some vaguely related ringtones while I’m there. I do have to give them some geek props though for using Jakarta Tapestry and its mobile markup capabilities on Gentoo running on AMD64 hardware (see stacktrace).

    If you’d like to check out the mobile NBC Olympics site on your phone (and would rather not shell out for a couple of SMSes), here’s where to go:

    While I wasn’t impressed with the mobile coverage, I was absolutely blown away by the video coverage on Sprint’s Power Vision. If you have a Power Vision phone, check out Sprint TV -> Sprint TV Live -> NBC Mobile Olympics. I checked it out just after NBC finished airing the opening ceremonies in Kansas and they were already covering alpine skiing practice and qualifiers. I can’t put in words how much this blew me away. More on that later.

  • mobile.kusports.com

    Over the past few days I’ve been spending some free time and downtime tweaking mobile.kusports.com and adding a couple of really cool (IMHO) features. We send out a ton of cel phone updates during each game, but I really wanted to bring our awesome live stats to mobile devices.

    The first order of business was to see how the low-tech live stats view looked on the mobile site. Since pretty much all of our templates extend a base template, it looked pretty darn good out of the box (thanks to template inheritance and some great default templates by Wilson and David). Most of the time spent on this template was to condense the stats a little bit to require less scrolling on small devices. Here is the live stats view using Opera Mini on my 6682:

    mobile.kusports.com live stats

    Having the live stats accessable from mobile devices is great, but it’s important to make it extremely easy for someone to get to them if they come to mobile.kusports.com while a game is in progress. The solution was to present the current score on the home page if and only if a game is in progress. This was accomplished by writing a custom template tag (which took literally a few minutes) and a tweak to the home page template:

    KUSports mobile home page

    Now the score and the stats are first and foremost while there’s a game on, but nowhere to be found if there isn’t. The last thing I wanted to do was to make sure that the current score (and a link to the live stats) was available from other interior pages in an unobtrusive way. I figure that if someone is reading a story or viewing photos while a game is on, they probably wouldn’t mind knowing the current score either. I decided to present the information without announcing itself quite as loudly as the score on the home page:

    KUSports mobile interior page

    I hope to do some more tweaking of our mobile sites here and there as free time permits, and I’ll do my best to highlight the changes here if I think they are particularly clever.