Backing Up Flickr Photos with Amazon S3



, , ,

I love that I now have an Amazon S3 billing page that reads like a really cheap phone or water bill. I think that they’re silently changing the game (again) without telling anyone else. I really like the implications of this magepiebrain post and decided to start using S3 “for real” myself last night.

The first ingredient was James Clarke’s Getting a list of my photos is pretty simple:

import flickr
me = flickr.people_findByUsername("postneo")
photos = flickr.people_getPublicPhotos(

The second ingredient for getting the job done was a pythonic wrapper around the Amazon example python libraries by Mitch Garnaat called BitBucket. Because it builds on the example libraries, there’s very little error checking, so be careful. Check out Mitch’s site for some example BitBucket usage, it’s pretty slick.

Once I was familiar with both libraries, I put together a little script that finds all of my photos and uploads the original quality image to S3, using the flickr photo ID as the key. Here’s the complete code for, all 25 lines of it.

After uploading 160 or so photos to Amazon, I owe them about a penny.

Getting photos back out is really easy too:

>>> import BitBucket
>>> bucket = BitBucket.BitBucket("postneo-flickr")
>>> bucket.keys()
>>> bits = bucket[u'116201243']
>>> bits.to_file("photo.jpg")


711 responses to “Backing Up Flickr Photos with Amazon S3”

  1. Jeff Milner Avatar

    This seems like a pretty nifty thing to do, but other than the novelty of it, why would you want to? In case you decide you don’t want to renew your pro account or something?

  2. Backing up Flickr images using Amazon S3…

    This is a cool show off application for Amazon’s S3 (the Amazon Simple Storage Service). It downloads……

  3. Thomas Hawk Avatar

    If you use the Slickr screensaver it automatically downloads high res versions of all of your photos to your hard drive for free. Given that 10,000 photos is only about 5 gigs of storage, seems easy.

  4. Franck Arnulfo Avatar

    Don’t need Slickr to backup your photos, use Flickr Backup : instead.

  5. Andy Avatar

    I’ve been trying to get all my Flickr photos ~3,000 on my hard drive for a couple of days. FlickrBackup doesn’t seem to work for me. It usually crashes or hangs when trying to get all 3,000 and if i try to download a set I just get the public ones, even though I’m logged in and authenticated.

  6. Andy Avatar

    Anyone know of any other Flickr Backup options? (to local hard drive…eventually)

  7. Armaan Avatar

    I have an honest question… why would one ever need to copy all their photos from flickr _back_ onto their hard drive again? That’s where they all started… why not just save the original copies?

  8. Matt Croydon Avatar


    Actually most of my photos originate and only exist on an RS-MMC card on my mobile phone. I just had my previous card crap out on me, so an extra backup gives me that warm fuzzy feeling.


  9. Ajay Avatar

    You can try more cool flickr stuff with these third party tools

  10. Jason Avatar

    This is nice, but how about a script that will back up any files you want to the S3 service?

  11. Mike Avatar

    I enjoyed your post and the thought process behind what you’re doing. I work for a company called Streamload. We recently launched our own web services API for our online storage platform. For non-commercial uses it’s totally free and any user can have 25 GB of online storage free. In the coming weeks, we’ll be exposing other really cool features via the API (like digital video transcoding, photo re-sizing and rotating and music bitrate conversion), so that whatever you’re building can go well beyond just tapping into an existing storage and bandwidth infrastructure. In fact, we built our new consumer online media center service, Streamload MediaMax, using it. You can find more information on it at

  12. Andy Avatar

    At one time they were all on my computer, but i’ve had a lot of hard drive crashes and computer changes over the last couple of years. Eventually I just figured they were safe on Flickr so I’d just upload new ones from whereever and keep them their, but I’d like to have them all locally again so I could upload them to and give it a try.

  13. S3 + Rails…

    Well, I spent most of today obsessively coding a simple Rails front-end to S3 called s33r (pronounced “seer”). It’s very incomplete, but mostly intended as a proof of concept. It allows you to perform the following operations:

    Create and delete …

  14. Daniel Drucker Avatar
    Daniel Drucker

    I’d like to see this include the all-important metadata as well – I spend many many hours tagging my photos and putting them into proper sets (and in the case of older scanned photos, properly dating them).

    I’d like to see this utility save the metadata via the Flickr API to an XML file, and upload that information to S3 too. A backup isn’t really a backup if you can’t restore from it – and you can’t restore from your utility as it stands now, because you don’t have the metadata saved.

  15. Iain Hindshaw Avatar

    Just found your home page its great, it looks like you folks do great service keep up the good work.

  16. Priyadarshan Avatar

    Unfortunately it seems the tags, the description and all that kind of metadata will not be backed up. Does the Flickr API not allow that? What about the comments?

  17. Daniel Drucker Avatar
    Daniel Drucker

    This is really NOT a good way to back up your Flickr photos if you care about your tags, metadata, etc.

    The Flickr API absolutely DOES allow all that stuff to be backed up – I do it once a month via a perl script, using Net::Flickr::Backup. Using that, all my Flickr photos are backed up to files, and all the metadata for each photo is backed up to an associated RDF file that contains tags, notes, comments, exif data – absolutely everything Flickr knows about the photo.

    There is nothing preventing this script from doing the same!

  18. Smith Avatar

    Thanks so very much for taking your time to create this very useful and informative site. I have learned a lot from your site. Thanks!!

  19. Michel Bohn Avatar
    Michel Bohn

    I found this one better for windows. Based on .net.

    Very easy to use and stable.


  20. inside Avatar

    Hi, this site is very nice…

  21. AFRif Avatar

    Not bad, it really can occur

  22. MaryJames Avatar

    Hello all

    How I can change avatar in this forum?

  23. MaryJames Avatar

    Hola all

    How I can change avatar in this forum?

  24. MaryJames Avatar

    Wazzup all

    How I can change avatar in this forum?

  25. cbmeeks Avatar

    I just recently destroyed years worth of work (well, doodle work but still) the other day. Totally forgot about it. It was mostly pictures, old OLD programs, etc.

    Not too terribly important. But I formatted the drive without thinking (or caring).

    Only to suddenly remember all of the stuff I lost. My bone-head mistake.

    So, I took what I do have and started backing it up everywhere. The more the better.

    So for me, automated python scripts that take stuff from my server and put them everywhere is really cool.

  26. eemwclfgth Avatar

    Hello! Good Site! Thanks you! folbtedrkxpbg

  27. Jennie Finch Photo…

    I Googled for something completely different, but found your page…and have to say thanks. nice read….

  28. does antivert work effectively…

    adding to my bookmarks… …

  29. datindkrn Avatar

    My name is Elena. To me of 20 years.
    I would like to get acquainted with you if you not against.
    I Look forward to hear you soon with impatience. Dating[url=]Dating[/url]

  30. stream video…

    I do think you are right on the spot with this post, I could use a lot of stuff for my new study, thank you very much….

  31. datindkjg Avatar

    My name is Elena. To me of 20 years.
    I would like to get acquainted with you if you not against.
    I Look forward to hear you soon with impatience. Dating[url=]Dating[/url]

  32. datindkjg Avatar

    My name is Elena. To me of 20 years.
    I would like to get acquainted with you if you not against.
    I Look forward to hear you soon with impatience. Dating[url=]Dating[/url]

  33. women and men difference…

    It is so important that we in the younger generation and our children learn from first hand experiences..thanks again…..

  34. penis enlargement…

    your blog is so important. you are the new media…

  35. careybagsbon Avatar

    Greetings to all.

    Prompt the best online shop on sale of Books.

  36. Katy Avatar


    As for a software to download images from Flickr (to your hard drive first) and then to other online services, I found Migratr very useful.

    I also have Slickr, and find it a neat way to share photos with my Dad. I installed it on his computer, and now whenever I tag a photo in Flickr as “fordad” it shows up in his screensaver.

  37. bob Avatar

    EXSTOY hi nice site thx

  38. download file mp3 Avatar

    adobe insurance nc insurance renewal notification state mandate
    [url=]selective auto insurance company of new jersey[/url]
    state of florida legislation auto insurance

  39. zithromax for urinary tract infection Avatar

    levaquin allegra
    [url= ]zyvox[/url]

    linezolid mrsa treatment

  40. bob Avatar

    CHC9xD hi good site thx

  41. bob Avatar

    JnQUV5 hi nice site thx

  42. bob Avatar

    vySA75 nice site thx

  43. mona Avatar

    hi great site 10x

  44. ben Avatar

    interesting site man