Matt Croydon::Postneo

Year: 2005

  • PyCon Day 3: Python for Series 60

    Python for Series 60 Presentation

    Erik Smartt and Jukka Laurila gave an excellent, well attended presentation on Python for Series 60 on Friday morning. It’s a shame that they only had 30 minutes to present, as they could have easily talked for twice that or more on the subject and not run out of material. I headed down to the front row for a good view of it all, so I wasn’t able to take quite as extensive notes as I would have liked.

    Erik started off with an overview of the Series 60 platform, something that he didn’t think he would have to do in a crowd like the the one at PyCon. While the number of Series 60 devices in the market vs. the total number of mobile devices isn’t very impressive, if you look at the smartphone segment, Series 60 dominates.

    After the introduction, Jukka took over with some of the more technical bits. Python for Series 60 isPython 2.2.2 as a Symbian DLL and available in binary-only form (for now anyway) at Forum Nokia. It covers many of the basics that one would need, including sockets, urllib, threading, and an Application UI framework. (Also note that if you drop the Python 2.2.2 source distribution in <INSTALLDRIVE>/SYSTEM/LIBS directory, a lot of stuff “just works”).

    Jukka then spoke about the porting difficulties that they ran in to because of the underlying Symbian architecture. Symbian C++ began before C++ itself supported things like exception handling, so they had to create their own way around it. Jukka then explained the realities of running out of memory on Series 60 and he also explained a little about ActiveObjects.

    After the technical stuff came some demos! Because of the setup, they had to switch between laptop input and video overhead input (to see the phone), but the demos were still cool indeed. Jukka demonstrated the Bluetooth console so that he could pop up an interactive intepreter session on his Laptop (it’s a lot better than t9). After that he showed the requisite one-liner hello world app to pop up a native widget. From there he demonstrated some of the hardware interfaces that are (or will soon be) available. For example, in just a few lines in the console he was able to import the camera module, take a picture, saving the file, and then passing it to Content_handler to show the photo.

    It also has a telephone. Jukka then typed import telephone and after another short line tried to dial a random phone number. He couldn’t actually dial because there was no signal in the Marvin center. For the final demo, Jukka showed the classic Snake game written in 99 lines of Python that utilize the as-yet unreleased 2d graphics package.

    After the demos were over there were tons of questions, so many that the session actually ran over by a few minutes. First and foremost was the question of cross platform development. The guys explained that the emulator was only available for Windows but you could use the bluetooth console using Mac and Linux too. Full instructions on that are linked to on the Python for Series 60 Wiki. Also on everyone’s mind was “where’s the source?” Erik couldn’t give a timeline for that, noting that Nokia is a big company with a lot of lawyers, but it sounds like they are doing their best to move things forward.

    I definitely think that the talk was a success. I really hope that the new modules demonstrated can make their way to the outside as soon as possible. Lots of games developers would likely jump on the 2d package, and give their first born for a crack at the PyOpenGL port demoed at ETech. I would personally love to see Jukka’s Snake make its way to the public under whatever license possible, because it’s addictive and would probably demonstrate the graphics platform quite well. I also think that it’s crucial to get these new bits of API functionality and other updates out to the developer community as quickly as possible.

  • PyCon Catchup

    Please bear with me while I play catchup from PyCon this week. I still have several sessions and notes to write up and will do so as quickly as I can.

  • Q&A With Greg Stein

    My notes on the Q&A after the keynotes are a bit rougher, so I’ll summarize.

    Guido asked when they would open source their (really cool sounding) built system. Greg said that the build system, like a lot of the stuff they use doesn’t make a whole lot of sense if you aren’t running on top of their google platform. Later in the Q&A Greg noted that when they perpare bits of code for release as open source the software usually gets better in the process. They also have some people working on untangling some bits from the google platform for release, so keep an eye on code.google.com

    Someone brought up Boost.Python, which is said to handle templated C++ code better. Greg was unaware of it, but they seem quite happy with SWIG for exposing C++ code to Python.

    Someone else wanted to know if they used Python for network monitoring and SNMP. Greg said that the ops guys keep a close watch on the traffic and that he usually has to inform them in advance when he needs to transfer “big” files.

    Alex Martelli, who starts working at Google in 3 days, wanted to know how SWIG dealt with templated code. It’s not great, but there are ways to get around simple templating.

    Another audience member wanted to know what Greg thought was missing from Python and what could be done about it, and how google dealt with programmers that feel more comefortable in c++ or Java. The first question was answered by the fact that they hired Alex. It was also interesting to learn that they run Python 2.2 on their servers. They would like to upgrade to 2.3, but that’ s a non-trivial task. Each engineering team decides what language they work in, and that’s not a huge problem because they make use of SWIG to cover the C++/Python bridge and they make extensive use of RPC so that it doesn’t matter what language each little bit is written in.

    When asked about how many engineers work at Google, Greg pointed to the public numbers, but wasn’t able to break it down further. A couple hundred perhaps.

    Google uses a derivative of Bugzilla for their bug tracking, but they would like something better, and are investigating other options.

    Everyone seems to equate Python with slow, but that hasn’t really been a problem with Greg or Google. When eShop got bought by Microsoft a lot of Python code was rewritten in ASP/COM and the resulting code was slower than Python.

    When asked about the total number of lines of code written in various languages, Greg reckoned that there were probably more lines of C++ than anything else, followed by Python and then Java (Blogger is written in Java).

    David Asher asked Greg about a patch to Python 1.4 a few years back and how that might be useful in the future with multiprocessors becoming so popular. Back in 1996 Greg patched Python 1.4 to remove the global interpreter lock and keep track of things that needed to be locked in other ways. His patches worked great on a single processor machine, 2 processors was a bonus, but once you got to 3-4 processors it was slower. He hasn’t run in to trouble with the global interpreter lock at google.

    Google doesn’t have debugging tools per se, but they do extensive logging (Greg likes “print”) and have good tools to analize those logs.

    Greg’s work projects include code.google.com as well as some internal stuff. For fun he works on Subwiki (in Python of course) along with the ton of other projects that he’s worked on over the years.

    They didn’t use an off the shelf web application framework for code.google.com, they built on top of the Google http server written in c++. GMail was written in C++ not Python.

    When asked how they stage, Greg said that they can route a small amount of traffic (say 1%) to “Canary Servers”. If these servers don’t fall over, they can slowly pump more traffic to the new version but can easily and quickly set a previous version as the live version.

  • PyCon Day 3: Greg Stein Keynote

    The keynote this morning was given by Greg Stein. Audio/video of his talk should be available at some point, but his slides are not available for download. Because of that I took extensive notes in Tomboy, which should be about as close to the slides as I could get with bad eyesight a the back of the room:

    GregSteinKeynote

    Notes taken during the Friday keynote at PyCon.

    Note before the keynote: Python success stories: pythonology.org.

    —

    Quote by Peter about python. [[sorry, the text was too small for me –Ed. ]]

    Python 10 years ago, contributed to Python, authored some modules and apps. Open source, contributed (c and python). Current chairman of ASF. viewcvs.py. Subversion, Apache httpd.

    “We consider Python to be our secret sauce.” –Paul Everett

    —

    Python at eShop:
    1995 “What in the world is python?”
    1996 “this is great stuff”

    [[eShop gets assimilated. –Ed.]]

    Python at Microsoft:
    1996 “it’s called what?”
    1997 “You actually shipped Python code?”
    1998 “Nice prototype. We’ll rewrite it in the next version.”

    Python at CollabNet:
    2001 “No we don’t really use Python here”
    2003 “Definitely write that in Python”

    Python at Google:
    2004 “Of course we use Python. Why wouldn’t we?”

    —

    Small companies eventually “got it” ahread of the curve
    – Champion was needed

    Larger companies follow Python’s growth curve
    – Supporting environment was needed

    Python had to grow for it to becoming business acceptable
    – Large enough talent pool
    – Support services
    – Books
    – Consulting
    – World wide web
    – Follow the trail-blazers

    Python passed the tipping point years ago.

    —

    Highly adaptable
    – Changing requirements
    – Changes in computing environment

    Rapid Development
    – For new and experienced developers

    Easy to maintain

    —

    Primary languages
    – C++
    – Java
    – Python

    Miscellaneous
    – Some perl used by Operations
    – PHP creeps in for internal webapps
    – Saw Ruby sneaking around
    – Small amount of C#

    —

    SWIG: Simplified Wrapper Interface Generator
    – www.swig.org
    – Started by David Beazley

    Multi-language Environment
    – SWIG pulls these “islands” together
    – Very fast mechanism for integration

    Integrated into build system

    —

    Where do we use it?

    – Across our internal network
    – Across a system lifecycle
    – Live services

    —

    Basic network

    development cloud to infrastructure to a whole bunch of servers. “we have quite a few servers”

    In development build system
    – wrappers for version control (they use Perforce). No, you don’t really understand Java code. Forces code reviews. Sends mail out.
    – build system (written in Python)
    – Packaging. Bundles of data packaged up and sent to servers. All built on Python. 3rd generation, it’s a complex problem on a gigantic scale.

    —

    Some usage in the network infrastructure
    – Binary data publisher
    – package repository

    This is written in Python too of course. Push it out. They keep increasing the scale of the problem.

    —

    Some usage on production servers
    – Monitoring (health, temp, hardware, etc)
    – Auto-restart

    —

    Complete the Lifecycle
    Log reporting
    – We generate a “large” amount of log information
    – Data is pulled back from the servers
    – Analyzed using lots of Python tools
    – Easy to alter the reports based on ever-changing needs

    —

    Python-based services

    Google groups
    – “Python old-timers” Jeske and Long (of eGroups and ClearSilver)

    code.google.com
    – Stein and DiBona

    Others? We have so much going on..

    —

    How code.google.com was built:
    front end stuff on top
    code.google.com server
    SWIG
    google stuff on bottom

    —

    code.google.com

    goopy package
    – Functional stuff to start with
    – Place to put future modules

    —

    Closing
    We have lots of Python code covering a broad range of needs

    Python has helped Google for many, many years

    SWIG is underrated

    We are now starting to open-source some of the code.

  • Python for Series 60 BoF

    We held a small BoF session in one of the Open Spaces this morning for Python for Series 60. Jukka demoed some oneliners, excercising the native widgets and excercising some of the available APIs. I was also able to take a look at someone’s FOMA phone running a Final Fantasy-like MMORPG running in Java on his phone.

    One attendee had questions about cross platform development. The SDK tools that allow you to run code on the emulator are win32 only, but the bluetooth console works on Windows, OSX, and Linux. More information can be found on the wiki.

    I haven’t seen a huge number of Series 60 devices (aside from Erik’s 7610, Jukka’s Charlie, my Taco, and a few others). Hopefully more people will come out of the woodwork after hearing Erik and Jukka’s talk tomorrow.

  • Profiling and Visualizing Python Program Behavior

    This morning I sat in on a session by Derek Jones and Richie Saunders who have developed tools for visualizing and profiling applications written in Python. Their company does a lot of work in digital signal processing. Python is great for productivity, but isn’t always fast enough for everyones needs.

    There are two libraries available for profiling Python: profile, written in javaPython and hostpot, written in C. They did some cool stuff by using the profiling hooks available to them. For example, they have created a python top that mimics the unix version by showing what is being executed and how much time is being spent in the various parts of your program.

    They are working on making the source code for their visualizations available, but have not been able to do so yet. There is no source availble (yet) but hopefully we’ll be able to see some of these tools down the road.

    Update: Drew pointed out the obvious mistake above that profile is written in Python. What can I say, I needed more sleep when I wrote this post.

  • PyCon DC 2005 Day 2

    I managed to sleep in this morning after a great first day of PyCon. I missed Guido’s State of Python talk, but I’m catching up by reading the excellent SubEthaEdit notes on his keynote. I hopped on the metro and made it for the tail end of the Chandler session. It looks like Chandler is definitely worth revisiting as it has improved significantly since the last time I looked at it. They had a slick demo that utilized flickr and it sounds like they also have the ability to bring in data from del.icio.us.

    Stay tuned to this post for links to all of my day 2 coverage. Other entries from today:

  • PyObjC Slides

    Bob Ippolito has posted the slides to his talks about PyObjC. I hear that there were some neat hacks in the second talk. I’m going to take a look at these slides later this evening.

  • More PyCon Coverage

    The PyCon Blog looks like another good place to keep tabs on what is going on here at PyCon. Another excellent resource is the collected SubEthaEdit notes for various sessions as gathered by Ted Leung. Also worth keeping an eye on are the searches and tages that I mentioned in my Tracking PyCon post.

  • PyCon: SQLObject

    Several groups ranted about how much they loved SQLObject while they were recapping what got done at the sprints before the conference. SQLObject allows you to map a database table to a Python object so that you can interact with it in a simple table.field (or object.property) way. It aparently increased the productivity of several of the sprinting teams.

  • PyCon: Matplotlib

    Mathplotlib

    After lunch I caught a great session on matplotlib which is a Python library that allows you to plot 2d graphcs in a simple Matlab-like way. The library takes advantage of several graphical frontends, including GTK, Tk, wxWindows, and FLTK. Rumor has it that someone is 90% done with a native Cocoa frontend too.

    Take a look at the screenshots for examples of what it can do. It looks like a very easy way to generate really nice looking plots and graphs. The demos took advantage of an enhanced Python shell called IPython, which I had heard about before but hadn’t looked at closely. IPython allows you to work with matplotlib in the interpreter. You need to do some custom handling if you would like to work with it from the standard interpreter. Its outout is highly configurable and looks extremely polished.

  • PyCon: Scripting the Mac with Python

    Before lunch I sat in on a session on using Python to interface with Applescript. Appscript is used to create a bridge between Python code and Applescript. Here’s a quick code snippet to give you an idea what Appscript can do:

    from appscript import *
f = app.("Finder.app")
f.help # shows what you can do with it, very nice.
f.desktop.disk[1].free_space.get()
# gives you an insane number of bytes free

    Slick indeed. There was also a slick demo to script OmniGraffle to generate a nice looking sitemap using Python, Appscript, and Beautiful Soup. For more cool hacks, check out these examples.

  • Keynote: IronPython

    The keynote this morning was by Jim Hugunin recently of Microsoft. The topic was IronPython. Jim took us for a spin around the now BSD-style implementation of Python running on top of the .NET CLR.

    Jim started out trying to prove that the CLR was a horrible platform for dynamic languages, but after running some tests realized that quite the opposite was true. When the proof of concept was presented last year, IronPython was about 70% faster than CPython. With advances in the .NET CLR and the fact that IronPython targets version 2.0 of the platform, IronPython is now just about 80% faster than CPython.

    Speed doesn’t come without a cost though. IronPython is quite a memory hog compared to CPython. Jim thinks that it’s not a big deal though. Jim demoed a few very interesting things about the platform. In the first demo, he drove Peety the parrot from the interactive Python interpreter. That was neat. The real amazement was when he called a C# extension from the interpreter that in turn called Python code. I’m sure that Jim was releived when it didn’t crash. The ability to extend IronPython is pretty big. Using C# extension are as simple as loading the assembly and importing the namespace.

    IronPython’s ability to leverage Avalon widgets and bling isn’t that cool. Pretty yes, cool no. However, the ability to import XAML and then walk the tree and interact with it is extremly cool.

    Jim was asked about IronPython and if it might eventually run on Mono and Rotor. He says “not right now” but by the time they’re done implementing .NET 2.0 in Rotor, it should work just fine. It sounds like there is also some interest in the Mono camp about getting IronPython to run on the platform.

    All in all it was a thought provoking presentation. I didn’t realize how polished the implementation had become in such a short time. Microsoft now as 1.5 people working on the project, although it constantly benefits from speed boosts and optimization done by the CLR team.

    Go ahead and Grab IronPython 0.7. It is released under a BSD-style license, although toward the end of the talk there were some questions raised about patents pertaining to IronPython that might make your life… interesting. Read the FAQ for more.

  • Tracking PyCon 2005

    I’ve tried to gather a list of resources that will allow people who are not attending PyCon to keep track of the various things that are going on. There are also RSS feeds associated with many of the resources below:

  • Pycon DC 2005 Day 1

    I’m gathering up various bits and pieces of gear before heading out the door and taking the Metro down to PyCon. The schedule is impressive, and that’s not even counting a lot of the ad-hoc goodness that will be going on in the open spaces.

    I plan on blogging as much as possible and will keep link to my posts throughout the day from this meta-post. The official backchannel is #PyCon on irc.freenode.net, and I’ll be there.

    Day 1 posts:

  • In Search of a Javascritpt WYSIWYG HTML Editor that Doesn\’t Suck

    The title says it all. I’m looking for a Javascript-based What-You-See-is-What-You-Get HTML editor that doesn’t suck. I fell in love with HTMLArea when I bought my copy of Radio back in 2001. I’ve since used HTMLArea 2 which is pretty decent but is IE only. Until recently I had been tracking the HTMLArea 3 pre-releases which worked in both IE and Mozilla and were beginning to deal with things like copy/paste content from newer versions of Microsoft Office products that embed all that nasty XMLish markup. The other day I went to htmlarea.com to see if there was a new beta release only to find that the lead contributor had mothballed the project:

    htmlarea 2 and 3 have been discontinued. They were free wysiwyg editors that were distributed on this site. Visit the directory to find similar products.

    What a bummer. The source is still in CVS, and I’m hoping that a leet Javascripter will take over the project, because HTMLArea 3 seemed pretty darn stable and quite near release. I’ve looked quickly at a bunch of possible replacements, but each one seems to have a showstopper. RTE is quite good for when you need a small, simple, compact editor, but it has its quirks. For instance, it’s an editable iframe instead of a layer over textarea, so there are some limits to the script-fu you can do with it. Don’t get the post title wrong, RTE does not suck at all, but it has its limitations.

    And so I ask you, dear reader, what the heck are you using these days for something like this? While I take the time to look closer at the lists of editors that I’ve glossed over before, I’d love to hear from you. Feel free to drop me a line (matt at ooiio dot com) or take a look at my JavascriptWyswiygHtmlEditors page and add your two cents. Please help, I’m at my wits end!

  • My New Printer: File, Print FedEx Kinkos

    File, Print FedEx Kinkos

    I’ve got a crappy little inkjet printer. It does a darn decent job at printing a page or two in black and white, but anything beyond that is just too much for it. Stuff usually prints, but not always. Sometimes it’s a bit smudged. Other times the paper jams up. Sometimes the cats run off with or otherwise mangle the printed page before I can get to it. The darn thing seems to take a lot of union breaks.

    A few weeks ago I was doing research on a paper for my computer organization class. I did much of my research at the University of Maryland’s awesome Engineering and Physical Sciences library, taking notes, xeroxing some pages, and checking out a book. There was also a weath of information available to me at the ACM Digital Library. I ended up downloading 150 or so pages of papers and articles from the vast ACM library.

    As much as I love technology, I’m just not able to skim text and read for long periods of time on a computer screen. I decided to print out the 150 or so pages, but had absolutely no desire to do that on my crappy little inkjet printer.

    I’ve wanted to give Mimeo a try ever since I learned about them a few years ago. They allow you to upload documents to their servers, preview them online, and then they print ia try. I stumbled upon them a few years ago and ship it to you. It’s a very cool idea, but it was Saturday and my paper was due Wednesday.

    Enter Fedex Kinkos. They have a similar service called File, Print Fedex Kinkos. After you download their software (win32 only, sorry), it creates a printer driver and integrates itself with Office. They offer the option of shipping your order to you, but more importantly they allow you to pick it up at your local Fedex Kinkos location.

    I spent a bit of time seperating the various articles I wanted to print with bibliographic information and eventually combined them all in to one big PDF file using Adobe Acrobat. I sent the job off to the File, Print Fedex Kinkos printer and chose options for my order. Since I was going to be flipping through all of the pages, I decided to go with double sided printing on the cheapest possible paper with three holes already punched. The el-cheapo worked out to somewhere around 6-8 cents or so (I forget) per printed page. I flipped through the preview and entered my billing information. It took a few minutes to upload the document to their servers, but after that was done I got a receipt to print and an email in my inbox. About an hour and a half later I got another email saying that the order was ready to be picked up.

    It’s not perfect though. Their premium laser paper is about 20 cents a pop and their options for binding are a bit limited. Their system is only set up to deal with one document at a time, so if you want to make a bunch of different copies of a bunch of different documents, you’re going to have to upload, set up, and pay for each one seperately. If you’re looking for more binding and layout options or for better handling of multiple documents, I’d strongly suggest that you check out Mimeo. Non-Windows users should check out the FedEx Kinkos web interface which looks to be multiplatform aside from some basic Javascript requirements.

    FedEx Kinkos is doing a very smart thing with this service. They’re taking advantage of the fact that they’ve got locations all over the US for pickup. They can also call on the FedEx infrastructure for shipped documents. They’re also making it easier for users to send them orders, reducing employee time spent on taking in orders. They are also probably keeping printers busy that might have otherwise been idle.

  • Open Screencasting

    I’ve been a big fan of Jon Udell‘s screencasts of various products and hacks for as long as he’s been doing them. Screencasts can be curious oddities but are more often extremely helpful. For example, this screencast covering the setup of Ruby [quicktime] adds a ton of value to the HOWTOs and install documentation on the Ruby on Rails site.

    That’s why I was especially excited by this flash screencast by Dan Winship demonstrating the basics of Stetic, a Mono-based Gnome GUI designer. After viewing the screencast, I noticed that he produced it with a program called vnc2swf:

    Vnc2swf is a screen recording tool for X-Window (X11), Windows and Mac OS Desktop. Vnc2swf captures live motion of a screen through VNC protocol and converts it a Macromedia Flash(TM) movie (.swf).

    The program is available in source and binary form for OSX and a few flavours of Linux. It’s defeinitely not a program polished for end users (it looks like installation may be tricker than many casual Linux users would be comefortable with), but it’s great to be able to produce quality screencasts using open source tools. SWF isn’t the most open format on the planet, but let’s look past that for now. I’m sure that you could convert it over to mpeg or something a little more “open” if you really wanted to.

    I see screencasting as a possible “next big thing.” I don’t think it’s going to be the next weblogging or podcasting, but it has tons of potential. I’d love to see little thumbnail screencasts fly by my aggregator when I read the freshmeat feed.

  • Divester: Google Ads as Design Element

    Divester

    I caught the announcement of another Weblogs, Inc. weblog the other day: Divester. You guessed it, it’s all about SCUBA diving. I used to dive, though it’s been a few years since the last time I got wet with a tank strapped to my back. Fearing information overload, I haven’t subscribed to the Divester RSS feed yet, but I’m going to pop in every day or so and check out the website, and reevaluate my RSS position in a few days.

    The design is quite striking. Big splashes of blue (as one might expect on a diving blog) and the sillouette of a diver. The coolest thing (I think) is the use of the Google Ad up top as a full fledged design element rather than something grafted on later.

  • Come on Ride the D-BUS (hey), Ride it (woo woo!)

    The February 2005 issue of Linux Journal contains a gem of an article by Robert Love called Get on the D-BUS. I didn’t notice it until I was trawling through the ACM Digital Library while working on a paper for my computer organization class. All tangents aside it’s a great article and anyone who uses Linux on the desktop should check it out, as I think that you’re going to see D-BUS do a lot of heavy lifting over the next few years.

    I’d strongly suggest reading the LJ article for a full definition, but to summarize: D-BUS is an interprocess communications system that is (or soon will be) used in both Gnome and KDE environments. It allows apps to send and receive messages to and from each other in a happy object-oriented easy-as-it-should-be way. The Gnome Mono codemonkeys are using it with Beagle, and probably other apps. There’s a D-BUS package and several apps that use it backed in to Hoary. There’s even a Security Enhanced DBUS built in to Security Enhanced Linux.

    The APIs and the software are under constant development, but there are already working libraries for C, Glib, and (yes I checked) even python. Here’s example-client.py (edited slightly to fit in my layout):

    #!/usr/bin/env python
import dbus
bus = dbus.SessionBus()
remote_service = bus.get_service("org.designfu.SampleService")
remote_object = remote_service.get_object("/SomeObject",
     "org.designfu.SampleInterface")
hello_reply_list = remote_object.HelloWorld("Hello!")
hello_reply_tuple = remote_object.GetTuple()
hello_reply_dict = remote_object.GetDict()
print (hello_reply_list)
print str(hello_reply_tuple)
print str(hello_reply_dict)

    It looks like D-BUS (or DBUS if you would prefer to abbreviate it that way) is going to be adopted in both the Gnome and KDE camps, which is A Good Thing. I think that it is going to lead to better interaction with applications on the desktop. I imagine a sexy Growl workalike telling me about all kinds of things that I may or may not want to know in a cute and unobtrustive way. As the technology is adopted, I see apps talking to one another and reacting to one another more and more. I see apps and frameworks taking advantage of external web services flowing over DBUS.

    Keep an eye on this list of apps that use D-BUS expand quickly.