I love that I now have an Amazon S3 billing page that reads like a really cheap phone or water bill. I think that they’re silently changing the game (again) without telling anyone else. I really like the implications of this magepiebrain post and decided to start using S3 “for real” myself last night.
The first ingredient was James Clarke’s flickr.py. Getting a list of my photos is pretty simple:
import flickr
me = flickr.people_findByUsername("postneo")
photos = flickr.people_getPublicPhotos(me.id)
The second ingredient for getting the job done was a pythonic wrapper around the Amazon example python libraries by Mitch Garnaat called BitBucket. Because it builds on the example libraries, there’s very little error checking, so be careful. Check out Mitch’s site for some example BitBucket usage, it’s pretty slick.
Once I was familiar with both libraries, I put together a little script that finds all of my photos and uploads the original quality image to S3, using the flickr photo ID as the key. Here’s the complete code for flickrbackup.py, all 25 lines of it.
After uploading 160 or so photos to Amazon, I owe them about a penny.
Getting photos back out is really easy too:
>>> import BitBucket
>>> bucket = BitBucket.BitBucket("postneo-flickr")
>>> bucket.keys()
>>> bits = bucket[u'116201243']
>>> bits.to_file("photo.jpg")
Comments
711 responses to “Backing Up Flickr Photos with Amazon S3”
This seems like a pretty nifty thing to do, but other than the novelty of it, why would you want to? In case you decide you don’t want to renew your pro account or something?
Backing up Flickr images using Amazon S3…
This is a cool show off application for Amazon’s S3 (the Amazon Simple Storage Service). It downloads……
If you use the Slickr screensaver it automatically downloads high res versions of all of your photos to your hard drive for free. Given that 10,000 photos is only about 5 gigs of storage, seems easy.
Don’t need Slickr to backup your photos, use Flickr Backup : http://sunkencity.org/flickrbackup instead.
I’ve been trying to get all my Flickr photos ~3,000 on my hard drive for a couple of days. FlickrBackup doesn’t seem to work for me. It usually crashes or hangs when trying to get all 3,000 and if i try to download a set I just get the public ones, even though I’m logged in and authenticated.
Anyone know of any other Flickr Backup options? (to local hard drive…eventually)
I have an honest question… why would one ever need to copy all their photos from flickr _back_ onto their hard drive again? That’s where they all started… why not just save the original copies?
Armaan:
Actually most of my photos originate and only exist on an RS-MMC card on my mobile phone. I just had my previous card crap out on me, so an extra backup gives me that warm fuzzy feeling.
–Matt
You can try more cool flickr stuff with these third party tools
http://www.quickonlinetips.com/archives/2005/03/great-flickr-tools-collection/
This is nice, but how about a script that will back up any files you want to the S3 service?
I enjoyed your post and the thought process behind what you’re doing. I work for a company called Streamload. We recently launched our own web services API for our online storage platform. For non-commercial uses it’s totally free and any user can have 25 GB of online storage free. In the coming weeks, we’ll be exposing other really cool features via the API (like digital video transcoding, photo re-sizing and rotating and music bitrate conversion), so that whatever you’re building can go well beyond just tapping into an existing storage and bandwidth infrastructure. In fact, we built our new consumer online media center service, Streamload MediaMax, using it. You can find more information on it at http://webservices.streamload.com.
At one time they were all on my computer, but i’ve had a lot of hard drive crashes and computer changes over the last couple of years. Eventually I just figured they were safe on Flickr so I’d just upload new ones from whereever and keep them their, but I’d like to have them all locally again so I could upload them to Riya.com and give it a try.
S3 + Rails…
Well, I spent most of today obsessively coding a simple Rails front-end to S3 called s33r (pronounced “seer”). It’s very incomplete, but mostly intended as a proof of concept. It allows you to perform the following operations:
Create and delete …
I’d like to see this include the all-important metadata as well – I spend many many hours tagging my photos and putting them into proper sets (and in the case of older scanned photos, properly dating them).
I’d like to see this utility save the metadata via the Flickr API to an XML file, and upload that information to S3 too. A backup isn’t really a backup if you can’t restore from it – and you can’t restore from your utility as it stands now, because you don’t have the metadata saved.
Just found your home page its great, it looks like you folks do great service keep up the good work.
Unfortunately it seems the tags, the description and all that kind of metadata will not be backed up. Does the Flickr API not allow that? What about the comments?
This is really NOT a good way to back up your Flickr photos if you care about your tags, metadata, etc.
The Flickr API absolutely DOES allow all that stuff to be backed up – I do it once a month via a perl script, using Net::Flickr::Backup. Using that, all my Flickr photos are backed up to files, and all the metadata for each photo is backed up to an associated RDF file that contains tags, notes, comments, exif data – absolutely everything Flickr knows about the photo.
There is nothing preventing this script from doing the same!
side effects of too much caffeine
caffeine free hydroxycut
cipralex
Thanks so very much for taking your time to create this very useful and informative site. I have learned a lot from your site. Thanks!!
I found this one better for windows. Based on .net. http://greggman.com/pages/flickrdown.htm
Very easy to use and stable.
Mike
Hi, this site is very nice…
Not bad, it really can occur
Hello all
How I can change avatar in this forum?
Hola all
How I can change avatar in this forum?
Wazzup all
How I can change avatar in this forum?
I just recently destroyed years worth of work (well, doodle work but still) the other day. Totally forgot about it. It was mostly pictures, old OLD programs, etc.
Not too terribly important. But I formatted the drive without thinking (or caring).
Only to suddenly remember all of the stuff I lost. My bone-head mistake.
So, I took what I do have and started backing it up everywhere. The more the better.
So for me, automated python scripts that take stuff from my server and put them everywhere is really cool.
http://www.signaldev.com
http://xoomer.alice.it/tosterive/
http://fomube.tripod.com/
http://hometown.aol.com/felixsmith69/
http://users.aol.com/wileygaines37/
Hello! Good Site! Thanks you! folbtedrkxpbg
Jennie Finch Photo…
I Googled for something completely different, but found your page…and have to say thanks. nice read….
does antivert work effectively…
adding to my bookmarks… …
My name is Elena. To me of 20 years.
I would like to get acquainted with you if you not against.
I Look forward to hear you soon with impatience. Dating[url=http://toplop.com/?idAff=59]Dating[/url]
stream video…
I do think you are right on the spot with this post, I could use a lot of stuff for my new study, thank you very much….
My name is Elena. To me of 20 years.
I would like to get acquainted with you if you not against.
I Look forward to hear you soon with impatience. Dating[url=http://toplop.com/?idAff=59]Dating[/url]
My name is Elena. To me of 20 years.
I would like to get acquainted with you if you not against.
I Look forward to hear you soon with impatience. Dating[url=http://toplop.com/?idAff=59]Dating[/url]
women and men difference…
It is so important that we in the younger generation and our children learn from first hand experiences..thanks again…..
penis enlargement…
your blog is so important. you are the new media…
Greetings to all.
Prompt the best online shop on sale of Books.
Hi,
As for a software to download images from Flickr (to your hard drive first) and then to other online services, I found Migratr very useful.
I also have Slickr, and find it a neat way to share photos with my Dad. I installed it on his computer, and now whenever I tag a photo in Flickr as “fordad” it shows up in his screensaver.
EXSTOY hi nice site thx http://peace.com
adobe insurance nc insurance renewal notification state mandate
[url=http://www.conectarnos.com/search.php?query=site:greatlistquotes.com&pr=&image=&www=true&start=20]selective auto insurance company of new jersey[/url]
state of florida legislation auto insurance
levaquin allegra
[url=http://forum.bestechvideos.com/users/67/posts ]zyvox[/url]
linezolid mrsa treatment
CHC9xD hi good site thx http://peace.com
JnQUV5 hi nice site thx http://peace.com
vySA75 nice site thx http://peace.com
upskirt
hi great site 10x
interesting site man