Mobile Screen Scraping with BeautifulSoup and Python for Series 60


Posted

in

, , ,

BeautifulSoup 2 BeautifulSoup 3

I haven’t had enough time to work up a proper hack for this, but I though I would pass along an interesting discovery that I made the other day before heading out to PyCon. After hearing about how great BeautifulSoup is at scraping HTML and making it easy to get little bits from it that you need, I thought I’d have a go at running it on my taco. You know what? It worked. I was expecting it to barf on import, but no, it chugged along just fine.

Now unfortunately BeautifulSoup won’t work out of the box with the standard .SIS install of Python for Series 60. It relies only on SGMLParser, string, and types, but those three libraries have some dependencies themselves. Here is what BeautifulSoup requires according to modulefinder.py running on my Debian box:

  • array
  • copy_reg
  • markupbase
  • re
  • sgmllib
  • sre
  • sre_compile
  • sre_constraints
  • sre_parse
  • string
  • strop
  • sys
  • types

These dependencies can be easily taken care of by dropping the python modules from the source distro in the appropriate libs directory on the drive you installed Python on.

One reason that BeautfulSoup “just works” on Series 60 is that the author strives to keep imports to a minimum and that the author srives to keep BeautifulSoup backwards compatible all the way back to Python 1.5.2. There are probably many modules out there like BeautifulSoup that are designed to be backwards compatible and platform independent that should work just fine on Series 60. As I find them, I will definitely point them out. I also hope to do some hacking on a few screen scraping apps that use BeautifulSoup and appuifw to present web data using native widgets.

Comments

65 responses to “Mobile Screen Scraping with BeautifulSoup and Python for Series 60”

  1. Senator Carl Kruger Avatar

    Excellent post. I was checking continuously this blog and I’m impressed! Extremely useful info specially the last part 🙂 I care for such info a lot. I was seeking this certain information for a long time. Thank you and best of luck.

  2. Carl Kruger Avatar

    Unquestionably believe that which you said. Your favorite justification seemed to be on the internet the simplest thing to be aware of. I say to you, I definitely get annoyed while people consider worries that they plainly don’t know about. You managed to hit the nail upon the top and defined out the whole thing without having side effect , people can take a signal. Will probably be back to get more. Thanks

  3. resor Avatar

    i have checked this cool site a few times now and i have to say that i find it quite great actually. keep the nice work up! =p

  4. payday loans Avatar

    Great Post, I love to read articles that are informative and acutally have good content. Thank you for sharing your experiences and I look forward to seeing more.

  5. praca nieruchomosci poznan Avatar

    An interesting discussion is worth comment. I think that you should write more on this topic, it might not be a taboo subject but generally people are not enough to speak on such topics. To the next. Cheers

  6. guild wars 2 guides Avatar

    Really wanted to tell you that I really enjoyed your blog, what kind of theme do you use? Is it a custom one? Mind to share with me? Thanks in advance!

  7. phone number lookup Avatar

    This is some agreeable material. It took me some time to locate this site but it was worth the time. I noticed this content was buried in yahoo and not the number one spot. This weblog has a ton of decent stuff and it doesn’t deserve to be burried in the searches like that. By the way I’m going to save this site to my favorites.

  8. hostgator coupon Avatar

    My sis informed me about your web site and the way nice it is. She’s right, I am really impressed with the writing and slick design. It appears to me you’re just scratching the floor by way of what you possibly can accomplish, however you’re off to a great begin!

  9. car buying Avatar

    Its like you read my mind! You appear to know a lot about this, like you wrote the book in it or something. I think that you can do with some pics to drive the message home a bit, but instead of that, this is magnificent blog. An excellent read. I will definitely be back.

  10. Hb Ladies Shoes Avatar

    Hi there, just became aware of your blog through Google, and found that it is really informative. I am going to watch out for brussels. I’ll be grateful if you continue this in future. Many people will be benefited from your writing. Cheers!

  11. Vente maison Concarneau Avatar

    I enjoy you because of all your valuable effort on this blog. My mother really likes participating in investigations and it is simple to grasp why. Most people hear all concerning the powerful means you provide informative guides via your website and improve response from other people on that subject matter while our own daughter is truly becoming educated so much. Enjoy the remaining portion of the new year. You are always conducting a wonderful job.

  12. japanese import car insurance Avatar

    Starting a article kind of like this one forced me to do some research and I found your post to be quite helpful. My website is centered around the idea of knowledge, fun and sharing. I wish you good quality luck with your internet site in the future and you can be sure I’ll be following it.

  13. Film solaire Avatar

    Thank you for all of the hard work on this web site. My aunt take interest in setting aside time for investigations and it’s really obvious why. Many of us notice all concerning the dynamic form you make rewarding information by means of this web site and as well improve participation from other people on this matter then my girl is really learning a great deal. Have fun with the rest of the year. Your conducting a terrific job.

  14. mini celular 2 chips Avatar

    You really make it seem so easy with your presentation but I find this matter to be actually something that I think I would never understand. It seems too complicated and extremely broad for me. I’m looking forward for your next post, I will try to get the hang of it!