Just mixing it up. Garrido's Restaurant.
A measured response. Garrido's Restaurant.
I used to back up in two different ways. I would write the files from a job to a DVD or multiple DVDs. When files were smaller I would burn two sets of DVDs as a back up. One from a particular manufacturer's brand of DVDs and one from an alternate brand, just to make sure someone at the factory wasn't having a bad day that might make my life miserable down the road. After burning the DVDs I'd make a back up onto two different external hard drives. It was a miserable use of time but you could always write a blog or a novel while you were waiting to feed the DVD burner....
But then there came a day when, because of the increase in raw file sizes, every job was requiring four or five DVDs and by the time you made a back up set you'd spent some real quality time sitting in the office trying to future proof work for a client. And many times (most times?) the advertising work has a more limited life than the DVDs or the hard drives themselves. The sheer volume of work in a good year means going through some respectable number of terabytes to get all the original and finished files from every video and still photography projects safely tucked away.
When fast 7200 rpm drives became more $$ available I started using them in the video editing cycle and in the image or edit processing cycle of a job and then writing everything out to additional, large but slower hard drives for back up.
I noticed on my last trip to my local Costco that 4 terabyte drives could be had, on sale, for $114. These are Seagate USB 3 drives in nondescript enclosures and they work with Macs and PCs. I bought some. Now I am in the process of making a third point of back up for the work I've done in the last quarter. I am transferring 842.95 gigabytes of job files from one of the previous 3TB drives. The files will also remain on that drive and the original drive on which they were ingested and edited. Of course the back-up is automatic at this point but it still boggles the mind that it will take three hours to complete the task. I have to admit that it's a heck of a lot faster than the original USB1 ports I started out with. After SCSI that is...
I tend to buy the hard drives in pairs and now that we're under the $150 mark for a 4TB the transactions are not as painful. I would council everyone who uses a method like this for backing up to get a slap of white tape and write the drive "name" and date of beginning and end of service (taken off system because it is full) and put it on the drive. I also recommend making a list of all the files on the drive and putting it in a notebook for quick reference should you need to find a precious file in a moment of desperation.
I liked making DVDs much better. While it took longer it's been exceedingly rare for me to have a CD or a DVD go "stupid" on me. I recently pulled some Kodak Gold CDs from 1999 of work I'd done in Madrid for Tivoli Systems and all the disks worked well. No corrupted files. But I do like the speed and ease of the HD back-ups. I just hope they are making them more and more reliable. But, unlike the film days I find myself not caring that much about older, client work files. We've instituted a policy of telling all clients in writing that when we hand them files the storage and maintenance of the files (including future migrations) is their sole responsibility. I make every effort to keep my hands on the work and the work on a working set of disks but no one can guard entirely from multiple, coincidental failures. And I might drop over dead just when they need their files the most. I will not delay my departure just to do more client tasks. :-) (Not planning on leaving any time soon).
I listened to Vincent Laforet talk to a group of our students a few years ago he dived into incredible detail about his back up strategy which calls for storing all files in RAID arrays in closet sized enclosures as well as making multiple tape back ups of everything. At the time he was estimating that his cost just to cool all the drives and the servers related to the drives was about $800 per month. Added to that one of his office assistants was constantly in the process of migrating terabytes of information from older drives to newer drives. It was stunning to listen to him and calculate the total monthly outlay he blew through just on back ups.
I've outlined my current back-up strategy above. I know some of you have far more technical knowledge about storage than I do. If you have any suggestions that might benefit all of the readers don't hesitate to chime in. If it's cost effective and better we're ready to change.
In the meantime I looked around Amazon to see what kind of deals I could find on external HDs. I limited my search to USB3's and Thunderbolts. Now might be the right time to add another layer to the back-up strategy or at least its redundancy.
This is the model I just picked up. You might be able to get a slightly better price at Costco but if you hate braving the crowds and already have an Amazon Prime membership AND you want to support my work here at the site this is not a bad price.....
And while you are waiting around for your back ups to happen it might be nice to have something exciting and photography oriented in your hands to read. Here's nice novel with a photographer as the protagonist. He's also a Leica aficionado. The Kindle version is on sale from $3.99 for the rest of this year. If you prefer to get your intrigue and adrenaline on paper we've got that too....
18 comments:
Just curious as to whether you have tried cloud storage, Kirk.
You might want to store one or two copies of your backed up files offsite. The small 2TB drives at Costco fit nicely into a bank safe deposit box.
This way you won't lose everything to a fire, flood, or...
I note Dave's comment about cloud storage. I have seen many cloud storage providers disappear or change their terms and conditions to become unuseable.
I'd avoid them for long term storage.
To transfer to a client, sure use them. Just don't expect the cloud provider to be there in 20, 7 or even 2 years from now.
I just ordered a wireless 1TB hard drive that has a SD slot built-in for backing up files directly from the card. Seems like a nice improvement over the old "digital wallets."
Amazon says it will be delivered on Sunday by USPS - this I gotta see.
I not so long ago blogged about my approach to workflow and archiving work. It can be seen here:-
http://paulamyes.com/2014/09/14/workflow/
Hm. Brings up the question about whether our "work" (for you it's work, I'm only a hobbyist) will survive us, and for how long.
I think if I ever had a picture which was worth of surviving me (in my opinion, think of a photo which the world *has to* see), I'd go and ask the professionals in that arena - like Guggenheim or MoMA. Surely these guys *have to* have a proper backup strategy.
And to Dave, above: I'm working at IBM, and "the cloud" is one of the current buzzwords there. But would I trust "them" (whoever it is who's responsible for your data)? Not really. Call me old fashioned, but I think that if you want a job being done properly, you'll eventually have to do it yourself...
Kirk, I am a big believer in multiple redundant backups of anything that is both important and digital. This link has very informative statistics for hard drive failure rates that I found very surprising given that most of the drives have very similar prices.
One of the hats I wear in my company is as the sysadmin, and therefore I am responsible for ensuring that approximately 5TB of data is never, ever inaccessible or lost. Because this is not my main function in the company, one of the essential attributes of my backup plan is that it does not require me to exercise any diligence on a day-to-day basis. All the backups are so-called "set and forget" whereby they run automatically and send an email to me on successful completion (or failure).
The other paradigm I use when thinking about all this stuff is to look at points of failure. For instance, a RAID-5 backup on first glance looks pretty good. One disk can fail and data can still be recovered. But there is another possible failure point in a RAID system - the RAID controller itself. If the controller fails, it is quite possible to have 4 or 5 healthy disks that are still unusable. I have seen this exact scenario occur more than once.
So multiple single disk backups of everything are also an essential part of the mix. With two of them stored in a safe deposit box and rotated frequently. This obviates the risk from fire, flood or theft or malicious software virus (another failure point).
I am pretty sure that neither Amazon nor Google (the two predominant cloud guys today) nor IBM are likely to be out of business in any time frame that matters to our photographs. Most companies today have much of their data in the cloud and by most metrics that data is significantly important. That being said, we have lots of disks spinning here to catalog not only photos but lots of video footage as well, but a cloud backup is also part of the plan if there is ever a fire (flood seems very unlikely here in Southern CA).
I agree that cloud is the solution. Once copy on HDD and second copy pon amazon, microsoft or google servers. It's actually very cost efficient solution. Arbuz.
Make analog copies of your work. This works with both stills and video. Analog film will last for at least 100 years, and can be read with 2114 scanners.
Edit your work and just print the keepers, no reason to keep thousands of non-keepers (losers ??)
I just pulled the trigger on a highly reviewed external Blu-Ray burner, discounted this black/cyber weekend to $70.
A 50 disc spindle of quality 50gb Verbatim bd-r is $115 on Amazon. That's a theoretical 2.5 TB of data (more like 2.2TB after formatting). 25 GB discs are cheaper per GB, but I want to back up my HDDs with something more permanent, so the time cost of swapping less discs makes $ense to me. 100 GB discs are still cost prohibitive, but the burner can do them when they get reasonable.
My main storage is a double disc redundant (raidz2) Freenas nas, that I back up in smaller data sets to single hdds, and to the cloud via the freenas Crashplan plugin. Even with all that I ordered the Blu-Ray burner. Single points of failure scare me, and having multiple types of back ups is the only way I know to reduce the anxiety.
This is partly tongue-in-cheek but have you ever done the calculation of what it would have cost you if you never backed up anything. How many images would you have lost, how many lost re-sales of images?
Less tongue-in-cheek, could someone make a case for a business model that never did back-ups? That is they forego future use of images but save the time and energy devoted to doing back-ups?
Kirk,
My family has used Synology products for 6+ years now. We've had several hard drives to fail, but we've always had the RAID setup, so we simply poped in a new drive and kept on going, so we have never lost ANY data. They come in many different sizes - 2 disk, 4 disk, or more. It does NOT require a great deal of technical knowledge to run one of these. Given the current size of affordable hard drives, you could very quickly and easily have a 12TB to 18TB solution, 1/2 the size of a breadbox attached directly to your iMac. They are fast and will also sync with cloud storage accounts if you are interested. If you have a tablet or smartphone, you could even link directly to it to show work to clients, or if you desire, you can create guest/client accounts that allow them to preview the job without your ever leaving your studio.
There are many additional features that the Synology offers which my family uses (Time Machine backup, iTunes server, video server) that you do not have to use, but are available. I would highly, highly recommend your investigating one of their products. ( I sound like a salesman, but I feel about their product the way many people to about their Macs. It just works).
Finally, some form of offsite storage is a must in case of fire, theft or some other natural disaster. We have a large external HDD that plugs in and syncs with the Synology that we keep in our safety deposit box.
Robert, That's a very interesting question even if you weren't 100% serious. I've only lost one hard drive and everything on it was backed up to DVDs. That's in nearly 17 years of dancing with digital. Most of our contracts now call for the clients to accept responsibility for archiving the work we license to them. Legally I suppose I could trash the files the minute we finish work on them... Then I could just store my fun stuff in a couple of places. Not sure it will mean anything 10 minutes after I'm gone!
I use two Netgear ReadyNAS. One at home and another at work. I setup an offsite backup job where photos are backed up from the one at home to the one at work on a nightly basis. Both NASes are also running in a RAID5 configuration. I can access my files from any web browser if I choose to as well.
You can kind of look at it as my own "cloud".
Looking to incorporate burning photos onto bluray discs in the near future as DVDs really can't hold that much nowadays.
I backup in three places myself. First to a fast internal drive on my PC, then a nightly automated job to back that drive up to an external NAS, and a cloud storage provider.
The nice thing about the cloud storage is they archive my files back 30 days, so if I should inadvertently contract a nasty virus that corrupts all my files, I can restore the previous unaffected versions.
Hi Kirk,
A few years ago I got tired of the backup shuffle. I also started to wonder about the use of a backup that is physically in the same risk zone as the originals (fire, water damage etc.).
After investigating offsite backups and trying several(too many !) of them i settled on one called Crashplan (www.code42.com).
Crashplan can be configured either where they host your data or where you host your data yourself (obviously at another location). I opted to host myself and have my backup server sitting in a friends house. In short I have 5 devices and TB's of images all quietly backing up to an off site backup. It even notifies me by email if any of the devices haven't managed to backup. Probably equally important, restores are easy fast and reliable.
Free to try and for a single user way cheaper than your disks to run.
-Ian
I'm very low-tech. I have all my files on external hard drives, and regularly clone each drive to another drive. Carbon Copy Cloner is easy, cheap and effective, and simply updates the backup, rather than starting from scratch. I could use Time Machine, but it's too sophisticated and I can never really be sure what it's doing! It's a bit like preferring manual focusing. Anyway, that way, everything's on two drives, and I rely on them not going pear-shaped at the same time.
Incidentally, so far all the problems I have had with hard drives relate to malfunctions of the enclosures, not the drives themselves. Getting someone to rip the enclosures apart and reinstall the drive itself in a generic enclosure can work wonders!
I don't trust the Cloud.
Post a Comment