[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Escaping Data Rot
Michael J. Mahon wrote:
Hard disks come with their read/write technology, so the only
technology "generation" issue is their attachment--ATA for decades,
only now transitioning to SATA--clearly destined to be pervasive for
decades to come.
Or at least the next five years... :-). But I take your point.
Prices for used LTO-2 drives and tapes have hit the "sweet spot" in
the past year. I can backup my entire server (300GB RAID-5 array) on
a single tape in less than an hour. The tape goes in my briefcase and
travels to the office with me so there's always an off-site copy.
How about 1.5TB? That fits on one <$90 drive and backs up in under
two hours. And I *always* have three copies of my data, so I can
tolerate the rare failure of a single drive. (A multiple-drive
failure, if they are stored apart, is astronomically unlikely.)
It all hinges on one's comfort level. I'm simply not comfortable betting on
hard disks. The larger they are, the more eggs in one basket.
Educate me on the costs: How much for a drive? How much for each tape?
What is the "sweet spot" tape capacity for incompressible data?
A bare LTO-2 drive should set you back about $100 from ePay, less if you
frequent hamfests and computer swap meets. A lot of them are showing up on
the retired equipment market and none that I've picked up are anywhere near
their service limits in terms of POH or tape cycles (they all appear to have
counters in non-volatile memory).
I picked up a large plastic storage bin of LTO-2 tapes with reasonably low
pass counts (yes, the tapes keep track of this also). Figure about 85 tapes
for $20. If you are loathe to trust used tapes, new ones are about $25 each
nowadays. Capacity on LTO-2 is 200GB uncompressed and 400GB (at best) using
the internal hardware compression. LTO-3 is double that and LTO-4 is, I
think, double that again. If you are archiving source code and text, you'll
get darn close to 2:1 compression. I have no trouble fitting 300GB on a
single tape and that represents a reasonable mix of file types.
I presume that you read back after each backup to verify that both drive
and tape are actually working? (A software division of my company also
used tape backup, and found that for six months, they had been making
unreadable backups!) This is a case where just making multiple backups
doesn't solve the problem!
Absolutely! The bash script that drives the backup does that automatically.
Some types of tape drives do read-after-write to check for bit errors, but off
the top of my head I cannot state whether or not LTO drives do this (I'm
betting they do).
I can't help but think that the practices of large IT departments are
strongly shaped by tradition (to put it politely). It seems to me that
most high-availability database systems would be best backed up using
a "floating triple-mirror" scheme, in which the system is never off-line
and the backups are all onto (other) hard disks on off-site mirrored
systems. A journaled database and file system mesh beautifully with
this approach.
On a technical level, perhaps, but all the high-capacity SANs and fiber-based
online storage systems use their own proprietary file systems and
communication protocols at some level. A tape written with 'tar' or other
common backup tool can be read back on anything and does not require logical
stitching together of stripes from an array.
The inherent "browsability" of a hard disk backup makes it very well
suited to the recovery of particular files or groups of files, and to
periodic "syncing" of file groups, which is a good match to the way
individual users maintain their files.
I'm sure tapes can be used in this way, but this is not the typical
"backup" scenario. Certainly data cannot easily be updated in place.
In place? No, of course not.
Random recovery is painful with vanilla 'tar', but I ported a variant called
'gtar' from OS/2 to Linux about 16 years ago. Gtar writes an index file
during backup and can use the stored block offsets to do a fast file search
for restore. Typical access time to anything on an LTO-2 tape is less than a
minute. It's saved my butt more than once :-).
Again, all this comes down to comfort level. I'm not trying to perpetuate a
"..my backup is better than your backup" thread by any means! I've simply
experienced a significant number of scary disk failures over the years. Most
seem to occur on the power-up side of things, BTW. Cannot recall ever losing
one while it was on (a good argument for never shutting the system down, which
is what I do with the server and RAID-5 array).
Steve