A Photographer’s Guide to Backup
I have an e-book on backup strategies for the photographer—its called “Quick Before They’re Gone” and you can find it on Amazon or buy directly from me. I thought I’d give you all a little preview of the book here in the blog – the following is from the introductory chapter and should give you an idea of the detailed contents of the book. You can also find another post on the general subject here. Below you’ll find the introductory chapter covering the main concepts of archiving and backup covered in detail in the book:
——————————————
Important Concepts
Before I dive into setting up an organizational system, there are a number of ideas pertaining to asset management we need to explore first.
- A “backup” is for day-to-day recovery from disasters
- An “archive” is for long-term storage of assets
- Maintain at least 3 copies of anything important
- Have at least one insurance copy off-site
- Cloud storage is the future of archiving
Backup-
Most people are familiar with the idea of backing up your computer. We have been admonished to maintain a copy of the files on our computer for as long as there have been computers. This goes hand in hand with the idea of saving your work: save frequently as you go along in case the computer crashes, so you won’t lose too much of you work. The same thing applies on a general hardware level: you need to maintain copies in case your computer or hard drive dies, not just momentarily crashes! For our purposes, a backup is the copy of your day-to-day work that allows you to recover instantly from a serious hardware failure. A good backup happens every day before you shut down your computer. Ideally, your backup should reside on a separate disk that you can re-boot from to get back to work. This means that the backup should include the necessary system resources to run your computer. The backup needs to maintain the most current state of your computer, and working files. It essentially mirrors everything on your computer, and changes to reflect the ongoing changes in your system.
Archive-
An archive is different from your backup. This is an insurance copy of important documents or other assets that you wish to store long term. Typically, you only add to an archive, never take away. The archive maintains copies of your assets that you might need to retrieve at some unspecified date in the future, and, in many cases, there can be multiple versions of files that you keep in the archive. The archive does not need to store everything on your computer, only documents that need to be secured. Applications and system resources can be ignored if you maintain a regular backup. The assets in an archive can be easily moved from one system to another if need be.
Three Copies-
Any good archive or backup should maintain at least three copies of everything: one working copy, and two backups. Ideally, one of the backups should live off-site for extra insurance against unusual catastrophic disaster, like a meteor striking your house (or a fire or a busted water pipe), that could take out your working copy and immediate backup copy. The three copies principle should be applied throughout the system, so that you have three copies of anything important at any given time. For instance, when ingesting captured images from a camera card, you’ll want to make an immediate backup in the field, so that the files reside on two hard drives and the card, until you get back home and you can make more permanent copies…
Off-Site Copy-
I’ve already mentioned the need for an off-site copy, but it bears repeating. Full protection against disaster is only possible if you keep a copy in a secure location off-site. Some photographers use a safety deposit box! Recycle one of your copies off-site at least once a week. I keep my extra copies in a fireproof box in my garage. It may not be the best solution, but at least it is separated from my main archive. The ideal solution may involve…
Cloud Storage-
The ultimate off-site solution will eventually be cloud storage: your image files uploaded to a remote server accessible over the internet. At the moment, internet speeds being what they are, the upload process is prohibitively slow for large image archives. Nevertheless, several services have become available for backup, and archive through the internet. I have been experimenting with a couple of services by storing hero shots and portfolio images online as extra insurance. As technology improves, this service will get faster and less expensive, and I have no doubt that this will become a dominant strategy for image archiving one day.
Step One
Get Organized
The first step to asset security is to get organized. Unfortunately almost no photographer organizes before they have way too many images strewn willy nilly across their hard drive. So the problem becomes: how does one organize a number of images that have been stored without any organization? I realize that there isn’t just one way to organize a group of images and you may have begun a process of organization already. I am going to suggest a strategy that may or may not seem good to you, depending on the exact nature of your work: I do know that this works for me and I’ve used many different methods in the past before settling on this one…
Any organizational system has to meet at least 3 requirements:
- It must be scaleable
- It must be portable
- It must be consistent
Scaleable means that you can start small, and grow easily. No system can work if you need to redo it every time your archive grows beyond a certain size. Your system must be able to scale to whatever size media you have available, and be expandable when larger capacity becomes available (as it inevitably will).
Portable means that your whole system can be easily copied from one computer to the next, and from one storage media type to another, without having to reformat or otherwise change your organizational system. It also implies that the overall system can be segmented into smaller parts and re-assembled if need be. Flexibility is key to making a system work over an extended time frame.
Consistent means that your system uses the same organizational criteria for every asset that you archive. It has to be based on something that can apply to any and all images or other documents that you may want to associate with your images, and it has to be repeatable without variation.
Beyond these 3 requirements your organizational system should be simple. A simple system will be easier to implement, and more importantly, easier to maintain and stick to. I use a system that I believe offers all of these things. Whether this approach works for you is something that only you can decide, but I suggest you give it a try unless you are already married to another approach.
————————————————
The book goes on to examine all the necessary components of an image asset management system and provides detailed information about:
- Organization
- Files
- Folders
- Drives
- Hardware
- Hard drives
- RAID
- Optical disks
- Cloud services
- Software
- Backup software
- Image Ingestion
- Cataloging Software
- Putting it all together
- KISS system day-to-day
- Emergencies – recovery from disaster
- Moving forward
Get a copy of my e-book: Quick Before They’re Gone —here!
Lee you are the man! If I don’t win, this book will still be added to my collection of your amazing books. Thanks for the opportunity and the knowledge you share!
Hi Lee,
This is very helpful information, I’m going to share it with all my viewers/readers/followers. More people need to know about this. I can’t wait to get the e-book!
Some really good information. I look forward to the book release!
Hmmm…I came over here because of Erin Mann’s suggestion. You have given me things to thing about. I am doing at least one thing right, though. I have a copy in the shed in the back yard, but it’s not in a fire proof box, yet. I need seriou help with organization though, so I will be watching out for the full release!
Thank you for that timely reminder and sharing your great advice! We recently had our main/home computer crash and lost quite a bit of our files and family photos. We’ve been meaning to implement a 3 layered backup system but haven’t quite figured it out yet. I need to get my ‘butt’ in gear!
Can’t wait for the e-book. We all need to be better prepaired: for it’s not if it will happen but when it will happen. Thankss Lee.
Hi! I, too, visited at Erin Manning’s suggestion. I know from experience that back ups are important. The key thing, I can agree, is that is has to be simple for us to follow through consistently. Look forward to the e-book.
I’d like to share with you all some experiences I had as an IT Auditor – the processes and theory discussed here are covered by existing best practices.
I was very interested in ‘cloud’ backups. The advertisements for the Mac seemed strangely inexpensive – but if they did the job properly — Great!.
The important thing to ask a vendor whom you’re planning to entrust with your data is if he can GUARANTEE the restoration of your data (or version of your data if that’s your arrangement) under all conditions.
I spoke with senior people at CrashPlan who told me it was too expensive to have data copied to multiple sites, although they would like to do so one day. They would not guarantee the availability of my data. For me, that removes them as a potential solution. Similar experience with Carbonite.
There’s a reason off-line storage is so expensive (in comparison) and that has to do with infrastructure management, controls and security, backups and development.
Check the prices of Amazon’s S3 solutions, or IBM’s, or Iron Mountain or Cisco.
If you have terabytes of data to be stored, it’s a business expense figured into your overhead.
If you’re not a business (or can’t afford the cost of doing it properly (meaning you KNOW you can recover your data in an emergency) — do the next best thing: Buy several 3TB external drives from a vendor you trust who is offering at least a 3 year warranty on them. Get a storage box in a bank and export your catalog(s) to a 3TB drive or two and put them in the vault. Next month, use two more of your new drives for export, bring them to the bank vault and retrieve the ones there. You get the idea?
During the second year, replace all of the drives. This is a pre-emptive action because those drives will eventually fail and this action reduces risk.
This process is much cheaper than uploading your data to a server farm, however it has a fatal flaw: it’s completely dependent upon someone doing the export and the trip to the bank. Any system that can be automated, should be automated (backups are a prime candidate for automation) — this is done to reduce the risk of the backup NOT being done.
I’m a hobbiest and, while my data is very important to me, vitally important, in fact, there’s no business and customer base (and potential law suits) involved. I take my stuff to the bank.
If this is your livelihood, you can’t afford to be so cavalier. You need a technical business solution for a technical business problem.
If you don’t have an internal IT department, you might consider hiring a qualified consultant to help you examine the best alternatives for your specific situation.
Just my 2¢
Thanks for your input Mark,
Most of your points are well covered in the book. I’ve been experimenting with cloud backup solutions only as extra backup options for “hero” and portfolio images – it is way too expensive and time consuming to rely on cloud services for the bulk of an archive and I agree that you have to be careful about guarantees and access to your data. I do feel, however, that, at some point in the future, cloud storage will make sense so I’m trying to see where it can be integrated into a personal backup system. The key here is that, either way, cloud storage can only be considered one of the three necessary copies and in my current approach it is actually 4th or 5th in line for important images only!
I didn’t see any format recommendations. Maybe the book will go into that? I’m wondering format for archival storage: Our library uses JPEG 2000 and I thought JP2 died years ago. There it is again. The LOC still uses TIFF. I like PNG- it holds lots of information and travels well on the net. Will I have to open everything and re-save if I use my camera’s RAW format? Would DNG make more sense? I hope it’s all there, Lee.
Well… the short answer is to archive whatever you would consider your originals and your master files! Its your archive so the choice is up to you! I don’t see any reason to re-save something in a different format other than what you work with… DNG is another issue entirely and I do talk about that in the book but I don’t see it as an absolute necessity.
Lee your timing is impeccable! I’m doing this very thing just as your update came into my email. Glad to know I’m doing it right.
One thing to add to all this. Keeping one’s computer “tuned up” is also important, especially the Mac’s “Beachball-of-Death”. That little spinning wheel can tell us a lot about our computers. Of course I found out the hard way, or rather my wife did and she ended up with that final tiny folder with a question mark in the middle. Good thing I did back up some of her files. But it had me thinking on how many photographers do spend the time to back up their systems, but I also noticed how many did not understand how to use their Disk Utilities or the “fsck -fy” in Linux.
Will you include such info with your work?
I’ll be calling you soon to tell you more about my site and our relaunch with our new look.
Thanks for the immense effort you create and the guidance you give to all of us.
Ken
Hi Lee,
Thanks for your passion and for sharing your knowledge. Look forward to the book release!
Hi Lee, looking forward to the complete eBook. Judging by the digital Zone-system pdf, this promises to be super helpful. Cheers Joern.
Lee;
I’m on vacation and finally have time to catchup on stuff I never have time to read. Reading your wisdom and advice presented in a simplistic format does serve to reinforce what I live every day in terms of the vast number of clients I interface with, not all photographers, that have not put in place even the most basic backup process. Most of whom I find on our doorstep are individuals that have taken photos of family and events in their lives that potentially have been lost forever, save the possibility that we can perform a costly data recovery. While I always preach the future need to backup, I think your words personally give me renewed content to embellish on my future efforts to educate our clients. Thanks Lee. Look forward to your book an will suggest it be read to our customers.
Norm
Hi Lee, I started using ImageingesterPro3 per your recommendation and love the software. Looking forward this new e-book.
Thanks for the great lecture at OCC this last Spring.
Alex E.
Delighted that the stuff will be in the ebook format. The last thing I need is more CDs or DVDs.
This is just what I need to efficiently backup and archive my photo collection! Thanks for your efforts and I’m sure the book will be a winner among photographers.
Hi Lee, thank you 🙂
I can’t wait to see the full ebook. I always look forward to your resources. Just what I need WHEN I need it!! Thanks for your work.
Great – i was just about to research back up options myself. Great timing.
Nicely done. This is something that is hard for people to get their heads around alot of the time until it is too late.
thanks for the reminder about back-up…always needed, mostly forgotten until it’s too late. I look forward to seeing your solutions to storage, since that is one thing I haven’t done well so far and as the number of images increases the problem grows.
Congratulations Lee! This is an area that most of us forget about until one minute after the computer won’t boot.
I have been wanting to read something about this for a long time. I’ve tried various methods of organisation, but never seem to be able to stick with them. I can’t wait to read your book!
This is a very timely post for me as I just had a corrupt hard drive. I had a full system backup so was able to restore no problem, but now, due to another system glitch that backup got erased! Good thing I had another. But I need a better overall system as my photography business grows. Would love a copy of your e-book!
tis something we dont think too often or oftenn enough about i feel ….would love the chance to win a copy of it
Thanks for the insights Lee. Looking forward to reading the rest of the book when it is released!
Great post Lee.
My strategy – I keep all NEF files, people ask me why and I explain I’d rather have the raw rather than a jpeg as storage is cheap and the benefit for keeping the raw out weighs the jpeg.
So, three external HDD which are sync’d and overnight I upload to my free mediafire account.
Lee,
An in-depth treatment of this topic would certainly be welcome by me. What is it about your upcoming ebook that will elevate it above the general advice and recommendations one routinely receives about image backup and archiving?
Best regards,
John
Did you read the blog post? At the end of the post is an outline for the contents of the book. I give specific instructions on setting up an archive from organization through hardware, software, cataloging methodology and real world scenarios based on my personal experience. If you’d like an exhaustive treatise on digital asset management you should also check out Peter Krogh’s “DAM book” – my book is more specific to the needs of an individual user and outlines a simpler approach than Peter does in his book. I only talk about hardware, software and methodology that I am using right now. I am on a Macintosh so most of the detailed information pertains to that platform specifically, although all of the concepts can be applied to Windows with other software recommendations and the general methodologies apply equally well to both platforms.
I’m looking forward to reading more about the different Hard Disk options. I agree that the cloud will be the future as bandwidth increases. Your method of keeping a backup in a fireproof box is excellent! I think my most likely scenario is fire or earthquake living in SoCal. My laptop and back up drive might get destroyed but if I have another copy safe, I should recover. Thanks for the great thoughts Lee!
Coming up with a logical way to organize your folders/files is something every photographer (amateur or pro) should dive into and embrace with open arms.
I’m looking forward to reading about your thoughts and discovering what works for you and employing it into my workflow.
Thanks for jumping into this very important topic!
I work for a large bank and once a quarter, we have to prove to the Feds that we can bring up our entire operation at a remote site with full functionality within 24-hours. This is serious stuff and anyone not backing up their digital assets has probably never seen a hard drive bite the dust or lived in an area prone to fires, floods and/or earthquakes.
I back up my data religiously but I am always looking for a more efficient system including the use of offsite and cloud storage. Nice to see another book on this subject. I think it is as important a skill to manage as exposure. Thanks Lee!