Busy making things: github, links, photos, @mc.

Mobile Screen Scraping with BeautifulSoup and Python for Series 60

Posted: March 28th, 2005 | Author: Matt Croydon | Filed under: Mobile, Projects, Python, Web Services | 65 Comments »

BeautifulSoup 2 BeautifulSoup 3

I haven’t had enough time to work up a proper hack for this, but I though I would pass along an interesting discovery that I made the other day before heading out to PyCon. After hearing about how great BeautifulSoup is at scraping HTML and making it easy to get little bits from it that you need, I thought I’d have a go at running it on my taco. You know what? It worked. I was expecting it to barf on import, but no, it chugged along just fine.

Now unfortunately BeautifulSoup won’t work out of the box with the standard .SIS install of Python for Series 60. It relies only on SGMLParser, string, and types, but those three libraries have some dependencies themselves. Here is what BeautifulSoup requires according to modulefinder.py running on my Debian box:

  • array
  • copy_reg
  • markupbase
  • re
  • sgmllib
  • sre
  • sre_compile
  • sre_constraints
  • sre_parse
  • string
  • strop
  • sys
  • types

These dependencies can be easily taken care of by dropping the python modules from the source distro in the appropriate libs directory on the drive you installed Python on.

One reason that BeautfulSoup “just works” on Series 60 is that the author strives to keep imports to a minimum and that the author srives to keep BeautifulSoup backwards compatible all the way back to Python 1.5.2. There are probably many modules out there like BeautifulSoup that are designed to be backwards compatible and platform independent that should work just fine on Series 60. As I find them, I will definitely point them out. I also hope to do some hacking on a few screen scraping apps that use BeautifulSoup and appuifw to present web data using native widgets.

65 Comments on “Mobile Screen Scraping with BeautifulSoup and Python for Series 60”

  1. 1 Sandeep said at 1:07 am on March 28th, 2005:

    Yeah, BeautifulSoup is… well, beautiful.

    I tried it awhile back – check out my blog post and wrote an app that does up-to-the-minute currency conversion (or at least as frequently as x-rates.com updates it).

    Damn, if I had timed it, I could’ve posted about that as well. As far as I recall, I spent a little less than three hours (that includes setting up BeautifulSoup to work on S60 and googling for how to parse HTML tables with it).

  2. 2 Salim Fadhley said at 4:45 am on March 23rd, 2006:

    I’m thinking of doing some series 60 hacking; Will any Nokia series 60 do the trick? Can Series 90 phones run Python?

  3. 3 DAIVIK said at 9:43 am on September 24th, 2007:

    cfdghsdfhsdhgsdh

  4. 4 Subeen said at 1:51 pm on June 2nd, 2008:

    I have started using Python in my S60 phones few days ago. I have written a mobile screen scraping app using urllib and regular expressions.

  5. 5 aaa hotel discounts said at 6:35 am on June 10th, 2010:

    aaa hotel discounts
    …

    aaa hotel discounts
    …

  6. 6 girls pick up said at 2:14 pm on November 12th, 2010:

    I’ve meant to write something like this on one of my blogs and you gave me an idea. TY.

  7. 7 car Graphics said at 5:42 pm on November 30th, 2010:

    Took me time to read all the comments, but I truly enjoyed the article. It proved to be Very helpful to me and I am certain to all the commenters here It’s always great when you can not only be informed, but also entertained I’m sure you had fun writing this write-up.

  8. 8 Building Cleaning Companies said at 6:07 pm on November 30th, 2010:

    Youre so right. Im there with you. Your blog is definitely worth a read if anyone comes throughout it. Im lucky I did because now Ive got a whole new view of this. I didnt realise that this issue was so important and so universal. You undoubtedly put it in perspective for me.

  9. 9 Gorilla trekking said at 6:12 pm on December 2nd, 2010:

    Dude, please tell me that youre heading to write more. I notice you havent written another blog for a while (Im just catching up myself). Your blog is just as well important to be missed. Youve acquired so very much to say, such knowledge about this topic it would be a shame to see this blog disappear. The internet needs you, man!

  10. 10 fharris said at 8:23 pm on December 4th, 2010:

    I believed it was going to be some dull aged post, however it definitely compensated for my time. I will submit a hyperlink to this page on my weblog. I’m positive my website visitors will come across that pretty useful.

  11. 11 Arnita Kamman said at 2:11 am on January 6th, 2011:

    Thanks a ton for this post. I totally loved going through it and will share it with people I know.

  12. 12 Apply for a credit card said at 6:36 am on January 20th, 2011:

    For card not present transactions exactly where the card is not shown (e.g., e-commerce, mail order, and phone sales), retailers in addition confirm that the customer is in bodily possession of the card and is the authorised consumer by asking for additional info these kinds of as the security code printed on the back again of the card, date of expiry, and billing tackle.Amex credit cards

  13. 13 how to increase breast size said at 9:35 pm on January 20th, 2011:

    Nice post. I study something more challenging on completely different blogs everyday. It will at all times be stimulating to read content from different writers and apply somewhat something from their store. I?d choose to make use of some with the content on my weblog whether you don?t mind. Natually I?ll provide you with a hyperlink in your internet blog. Thanks for sharing.

  14. 14 propecia said at 10:53 pm on January 23rd, 2011:

    Let’s talk, to me is what to tell.

  15. 15 Taylor Serafine said at 6:50 am on January 25th, 2011:

    Some truly excellent blog posts on this internet site,thankyou for contribution.

  16. 16 KcpOziRZ said at 10:53 am on February 1st, 2011:

    XPNekH

  17. 17 biuro rachunkowe wrocÅ‚aw said at 3:30 am on February 4th, 2011:

    :)

  18. 18 top ipad apps said at 9:07 pm on February 5th, 2011:

    It had been a while since I visited website with such high quality information. Thansk so much for the useful data

  19. 19 playbook cases said at 10:17 pm on February 5th, 2011:

    Great blog you have here, hope you won’t mind if I retweet your blog posts to my twitter’s account.

  20. 20 guess watch said at 12:04 am on February 6th, 2011:

    My husband and I started doing this a few years ago. It was a leap of faith for me because I found it to be humbling to admit on paper (or in our case a bulletin board) my big dreams..what if they dont come true? (Weve had friends and family happen to see it when they are in our bedroom.) I continue to believe in this as a powerful exercise in faith. We have seen so many of our dreams accomplished and have traded many pictures and words that have happened for new dreams and visions. Our dream board is one of the first things I see before I fall asleep and the first thing I see in the morning. In the center of the board we have a picture of Jesus. Weve also found that creating custom desktop wallpaper with words and images is also a good tool.Have fun with it!

  21. 21 Sheryll Durette said at 12:16 pm on February 6th, 2011:

    Hey I was just looking at your site in IE 8 and the graphic at the top of the link doesnt show up right. Just thought I would let you know.

  22. 22 Life Plus said at 1:59 pm on February 6th, 2011:

    Please message me with any tips about how you made this site look like this, I would appreciate it!

  23. 23 biuro rachunkowe said at 8:56 pm on February 6th, 2011:

    Dzieki za info.

  24. 24 Tyson F. Gautreaux said at 11:43 pm on February 6th, 2011:

    Hello! Bookmarked it to my PC^^ Sweet post.

  25. 25 diet plans that work fast said at 10:34 am on February 9th, 2011:

    Greetings from Idaho! I’m bored to tears at work so I decided to browse your website on my iphone during lunch break. I really like the knowledge you present here and can’t wait to take a look when I get home. I’m shocked at how fast your blog loaded on my cell phone .. I’m not even using WIFI, just 3G .. Anyhow, good blog!

  26. 26 Make Money Website said at 2:43 am on February 10th, 2011:

    very interesting information! .

  27. 27 Resor said at 11:24 am on February 10th, 2011:

    actually liked the article you published actually. it just is not that easy to find great text toactually read (you know.. really READ and not just browsing through it like some zombie before going to yet another post to just ignore), so cheers man for really not wasting any of my time! :)

  28. 28 iphone 5 cases said at 2:32 pm on February 10th, 2011:

    I like the way you present the post. Very clear and useful for a newbie like me. Absolutely great post with valuable tips. I enjoyed reading your blogs. Thanks for sharing such points. I am impressed!! ;)

  29. 29 Enlever Rayure said at 5:40 pm on February 10th, 2011:

    Thanks for each of your hard work on this blog. My mother takes pleasure in managing internet research and it’s obvious why. My partner and i learn all of the powerful manner you create rewarding tactics on this website and therefore cause participation from website visitors on the situation then our favorite daughter has always been being taught so much. Enjoy the rest of the year. You’re the one conducting a tremendous job.

  30. 30 Rayure Verre said at 1:04 am on February 11th, 2011:

    Thank you for all of the effort on this web site. My mom take interest in working on investigations and it is simple to grasp why. Many of us notice all relating to the compelling means you render simple steps via this blog and in addition invigorate contribution from other ones on the concept then our own simple princess is always starting to learn a great deal. Have fun with the remaining portion of the new year. You are always doing a very good job.

  31. 31 xoom accessories said at 6:20 am on February 11th, 2011:

    I like the way you present the post. Very clear and useful for a newbie like me. Absolutely great post with valuable tips. I enjoyed reading your blogs. Thanks for sharing such points. I am impressed!! ;)

  32. 32 Immobilier Bretagne said at 10:53 am on February 11th, 2011:

    There are some attention-grabbing time limits on this article however I don’t know if I see all of them heart to heart. There’s some validity but I’ll take hold opinion till I look into it further. Good article , thanks and we want extra! Added to FeedBurner as properly.

  33. 33 awnings said at 5:01 am on February 12th, 2011:

    hdeiduhqpke qeihi zvef.

  34. 34 indy blog said at 6:20 am on February 12th, 2011:

    Thank you for the auspicious writeup. It in fact was a amusement account it. Look advanced to more added agreeable from you! However, how can we communicate?

  35. 35 biuro nieruchomosci poznan said at 4:16 pm on February 12th, 2011:

    Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed, but also entertained!

  36. 36 sankozmetik.com said at 6:12 pm on February 12th, 2011:

    Thanks nice post..

  37. 37 Sortir said at 7:28 pm on February 12th, 2011:

    I enjoy you because of all of the effort on this web site. Ellie enjoys working on research and it’s really obvious why. My spouse and i know all relating to the compelling method you provide very helpful guides on your web blog and therefore improve response from some others on that subject plus our favorite simple princess is undoubtedly discovering so much. Take advantage of the remaining portion of the new year. You’re performing a really good job.

  38. 38 thailand said at 5:21 pm on February 13th, 2011:

    continue with the the good work on the site. I appreciate it. Could use some more frequent updates, but i’m quite sure that you have got more or better things to do like we all do. =p

  39. 39 Veta Sadak said at 6:44 pm on February 13th, 2011:

    Can I quote you on my website if I link back to your blog?

  40. 40 Issac Maez said at 1:36 pm on February 14th, 2011:

    Hey, I searched for this blog on Bing and just wanted to say thanks for the excellent read. I would have to agree with it, thank you again!

  41. 41 budapest reseguide said at 2:42 pm on February 14th, 2011:

    continue with the the good work on the blog. I like it. Could use some more frequent updates, but i am sure you got some more or better things to do , hehe. =)

  42. 42 how to buy a new car said at 3:05 pm on February 14th, 2011:

    Hello there, You have done a great job. I will certainly digg it and personally suggest to my friends. I am confident they’ll be benefited from this web site.

  43. 43 Senator Carl Kruger said at 5:47 pm on February 14th, 2011:

    I do not even know how I ended up here, but I thought this post was great. I don’t know who you are but definitely you are going to a famous blogger if you aren’t already ;) Cheers!

  44. 44 LevineMARYANNE19 said at 7:46 pm on February 14th, 2011:

    I received 1 st business loans when I was very young and that aided me a lot. However, I require the sba loan as well.

  45. 45 Emtek 8050-P-US26 said at 9:30 pm on February 14th, 2011:

    I happen to be commenting to make you understand of the outstanding discovery my friend’s princess encountered using your web page. She picked up plenty of things, which include what it’s like to have a very effective giving nature to make most people completely gain knowledge of several grueling topics. You truly surpassed my expectations. Thank you for giving the helpful, trustworthy, explanatory and cool tips on that topic to Gloria.

  46. 46 Kids Tory Burch Flats said at 11:07 pm on February 14th, 2011:

    tory burch shoe sale …

    Tory Burch flip flop online sale.For the younger working expat women,look at these tory burch Shoes.these tory burch flip flops are definitely fun,yet the glossy design and round-toe stay these lovely.These tory burch shoes with multiple.New arrivel To…

  47. 47 Constructeur Vannes said at 5:12 am on February 15th, 2011:

    I enjoy you because of all of the work on this web page. My daughter take interest in doing investigations and it is simple to grasp why. We all learn all about the powerful mode you create good secrets via the web blog and even boost response from the others on this situation so my simple princess is really being taught so much. Take advantage of the rest of the year. You are doing a wonderful job.

  48. 48 Linnie Beato said at 6:40 am on February 15th, 2011:

    I found your weblog site on google and verify just a few of your early posts. Continue to keep up the very good operate. I just further up your RSS feed to my MSN News Reader. Seeking ahead to reading extra from you in a while!…

  49. 49 porno izle said at 11:32 am on February 15th, 2011:

    Good post admin. :) Thanks :)

  50. 50 Sharleen Simkin said at 4:06 pm on February 15th, 2011:

    I am pleased that I discovered this web site, exactly the right info that I was looking for! .

  51. 51 Senator Carl Kruger said at 8:46 pm on February 15th, 2011:

    Excellent post. I was checking continuously this blog and I’m impressed! Extremely useful info specially the last part :) I care for such info a lot. I was seeking this certain information for a long time. Thank you and best of luck.

  52. 52 Carl Kruger said at 1:57 am on February 16th, 2011:

    Unquestionably believe that which you said. Your favorite justification seemed to be on the internet the simplest thing to be aware of. I say to you, I definitely get annoyed while people consider worries that they plainly don’t know about. You managed to hit the nail upon the top and defined out the whole thing without having side effect , people can take a signal. Will probably be back to get more. Thanks

  53. 53 resor said at 3:53 am on February 16th, 2011:

    i have checked this cool site a few times now and i have to say that i find it quite great actually. keep the nice work up! =p

  54. 54 payday loans said at 7:09 am on February 16th, 2011:

    Great Post, I love to read articles that are informative and acutally have good content. Thank you for sharing your experiences and I look forward to seeing more.

  55. 55 praca nieruchomosci poznan said at 10:19 am on February 16th, 2011:

    An interesting discussion is worth comment. I think that you should write more on this topic, it might not be a taboo subject but generally people are not enough to speak on such topics. To the next. Cheers

  56. 56 guild wars 2 guides said at 11:50 am on February 16th, 2011:

    Really wanted to tell you that I really enjoyed your blog, what kind of theme do you use? Is it a custom one? Mind to share with me? Thanks in advance!

  57. 57 KO Propo 17004 said at 8:01 pm on February 16th, 2011:

    The article at http://www.wadeindustries.com/article/Baby-Shower-Invitation-Help-What-To-Write.html explains my view on Baby Shower Invitation Help What To Write

  58. 58 phone number lookup said at 12:25 am on February 17th, 2011:

    This is some agreeable material. It took me some time to locate this site but it was worth the time. I noticed this content was buried in yahoo and not the number one spot. This weblog has a ton of decent stuff and it doesn’t deserve to be burried in the searches like that. By the way I’m going to save this site to my favorites.

  59. 59 hostgator coupon said at 11:02 am on February 17th, 2011:

    My sis informed me about your web site and the way nice it is. She’s right, I am really impressed with the writing and slick design. It appears to me you’re just scratching the floor by way of what you possibly can accomplish, however you’re off to a great begin!

  60. 60 car buying said at 1:48 pm on February 17th, 2011:

    Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a bit, but instead of that, this is magnificent blog. An excellent read. I will definitely be back.

  61. 61 Hb Ladies Shoes said at 2:39 pm on February 17th, 2011:

    Hi there, just became aware of your blog through Google, and found that it is really informative. I am going to watch out for brussels. I’ll be grateful if you continue this in future. Many people will be benefited from your writing. Cheers!

  62. 62 Vente maison Concarneau said at 8:44 pm on February 17th, 2011:

    I enjoy you because of all your valuable effort on this blog. My mother really likes participating in investigations and it is simple to grasp why. Most people hear all concerning the powerful means you provide informative guides via your website and improve response from other people on that subject matter while our own daughter is truly becoming educated so much. Enjoy the remaining portion of the new year. You are always conducting a wonderful job.

  63. 63 japanese import car insurance said at 3:48 pm on February 18th, 2011:

    Starting a article kind of like this one forced me to do some research and I found your post to be quite helpful. My website is centered around the idea of knowledge, fun and sharing. I wish you good quality luck with your internet site in the future and you can be sure I’ll be following it.

  64. 64 Film solaire said at 9:13 pm on February 18th, 2011:

    Thank you for all of the hard work on this web site. My aunt take interest in setting aside time for investigations and it’s really obvious why. Many of us notice all concerning the dynamic form you make rewarding information by means of this web site and as well improve participation from other people on this matter then my girl is really learning a great deal. Have fun with the rest of the year. Your conducting a terrific job.

  65. 65 mini celular 2 chips said at 5:58 pm on February 19th, 2011:

    You really make it seem so easy with your presentation but I find this matter to be actually something that I think I would never understand. It seems too complicated and extremely broad for me. I’m looking forward for your next post, I will try to get the hang of it!