[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Escaping Data Rot

Steven Hirsch wrote:

Michael J. Mahon wrote:


Hard disks come with their read/write technology, so the only
technology "generation" issue is their attachment--ATA for decades,
only now transitioning to SATA--clearly destined to be pervasive for
decades to come.



Or at least the next five years... :-).  But I take your point.
Prices for used LTO-2 drives and tapes have hit the "sweet spot" in the past year. I can backup my entire server (300GB RAID-5 array) on a single tape in less than an hour. The tape goes in my briefcase and travels to the office with me so there's always an off-site copy. 


How about 1.5TB?  That fits on one <$90 drive and backs up in under
two hours.  And I *always* have three copies of my data, so I can
tolerate the rare failure of a single drive.  (A multiple-drive
failure, if they are stored apart, is astronomically unlikely.)
It all hinges on one's comfort level. I'm simply not comfortable betting on hard disks. The larger they are, the more eggs in one basket. 

The trick is never to trust *anything* in one basket--that's why I
use at least three.  And with that level of redundancy and physical
dispersal, the only real risk is global catastrophe.  ;-)

And a 3.5" hard drive puts 1.5TB (at the cost per byte sweet spot) in
a jacket pocket--not that I'd ever keep one there.  ;-)  Hard disk
data volume densities trump anything else available, and it is instantly
accessible (with a USB adapter) on a random computer.


Educate me on the costs:  How much for a drive?  How much for each tape?
What is the "sweet spot" tape capacity for incompressible data?
A bare LTO-2 drive should set you back about $100 from ePay, less if you frequent hamfests and computer swap meets. A lot of them are showing up on the retired equipment market and none that I've picked up are anywhere near their service limits in terms of POH or tape cycles (they all appear to have counters in non-volatile memory). 

All this reminds me of my use of DAT backup tapes in the early 1990s.
It was unusual (I was the only one I knew with a DAT drive), slow
(compared to hard disk rates), and inconvenient to use--particularly
for recovering, say, eight photos that I had accidentally overwritten.

I found that I used it three times, and that was it.

The joy of hard drive backup is that it is fast *and* convenient, both
for backing up and browsing.  And all the panoply of disk tools can be
used to peruse and map the data stored.  With the DAT drive, I was
locked into using the manufacturer's minimal programs.
I picked up a large plastic storage bin of LTO-2 tapes with reasonably low pass counts (yes, the tapes keep track of this also). Figure about 85 tapes for $20. If you are loathe to trust used tapes, new ones are about $25 each nowadays. Capacity on LTO-2 is 200GB uncompressed and 400GB (at best) using the internal hardware compression. LTO-3 is double that and LTO-4 is, I think, double that again. If you are archiving source code and text, you'll get darn close to 2:1 compression. I have no trouble fitting 300GB on a single tape and that represents a reasonable mix of file types. 

Virtually all the volume of my data is already compressed: JPEGs and
MPEGs, so compression doesn't do me much good.  I have tens of thousands
of compressible files, but their combined volume is a small fraction of
the total.


I presume that you read back after each backup to verify that both drive
and tape are actually working?  (A software division of my company also
used tape backup, and found that for six months, they had been making
unreadable backups!)  This is a case where just making multiple backups
doesn't solve the problem!
Absolutely! The bash script that drives the backup does that automatically. Some types of tape drives do read-after-write to check for bit errors, but off the top of my head I cannot state whether or not LTO drives do this (I'm betting they do). 


I can't help but think that the practices of large IT departments are
strongly shaped by tradition (to put it politely).  It seems to me that
most high-availability database systems would be best backed up using
a "floating triple-mirror" scheme, in which the system is never off-line
and the backups are all onto (other) hard disks on off-site mirrored
systems.  A journaled database and file system mesh beautifully with
this approach.
On a technical level, perhaps, but all the high-capacity SANs and fiber-based online storage systems use their own proprietary file systems and communication protocols at some level. A tape written with 'tar' or other common backup tool can be read back on anything and does not require logical 
stitching together of stripes from an array.


The problem, in most cases, is "stopping the world" or opening an
availability loss window for the shortest possible time.  Specialized
techniques are much better at this.


The inherent "browsability" of a hard disk backup makes it very well
suited to the recovery of particular files or groups of files, and to
periodic "syncing" of file groups, which is a good match to the way
individual users maintain their files.

I'm sure tapes can be used in this way, but this is not the typical
"backup" scenario.  Certainly data cannot easily be updated in place.



In place?  No, of course not.
Random recovery is painful with vanilla 'tar', but I ported a variant called 'gtar' from OS/2 to Linux about 16 years ago. Gtar writes an index file during backup and can use the stored block offsets to do a fast file search for restore. Typical access time to anything on an LTO-2 tape is less than a minute. It's saved my butt more than once :-).
Again, all this comes down to comfort level. I'm not trying to perpetuate a "..my backup is better than your backup" thread by any means! I've simply experienced a significant number of scary disk failures over the years. Most seem to occur on the power-up side of things, BTW. Cannot recall ever losing one while it was on (a good argument for never shutting the system down, which is what I do with the server and RAID-5 array). 

Sounds like a stiction problem, or possibly contamination.  Both are
extremely rare in modern drives, so that redundancy lowers the chance
of loss to a miniscule level.  Presumeably, RAID masked any loss of data
as a result of hardware failure.

In my experience (and there's been plenty of that ;-), most catastrophic
failures have nothing to do with disk hardware.  It is almost always an
errant program or operator that crashes the file system or database.
(And twice it was the hardware connection from the system to the drive.)

Modern disk drives provide abundant sensing of impending permanent
errors long before they actually occur, and keeping multiple copies
reduces risk of data loss below the level of "meteor collision" loss.

-michael

NadaNet and AppleCrate II: parallel computing for Apple II computers!
Home page: http://home.comcast.net/~mjmahon

"The wastebasket is our most important design
tool--and it's seriously underused."