[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Rescuing old 5.25" disks

Ben Rudiak-Gould wrote:

Michael J. Mahon wrote:


One of the reasons that image files don't contain internal checksums
is that they can also be "written to" by emulators, and re-computing
a checksum for an arbitrary internal change is non-trivial.
True. I've been thinking mostly of read-only images. But I'd argue that in the case of canonical images of existing disks, writing to them from an emulator *should* invalidate the checksum. 

How about just keeping a "changed" bit to distinguish the file as
changed from its original capture?  If error detection is a good thing,
then you shouldn't want to be without it on disks you update.  ;-)


In the same way that .dsk is the default, and .nib is used only when
.dsk is insufficient, I'd propose that the universal format only be
used when both .dsk and .nib are insufficient.
I like that idea except for that fact that I don't know when to trust a .dsk or .nib image. What I want is a format that's only produced by tools that verify everything they read, so that you can see an image of program X in that format on Asimov and have some reasonable level of assurance that X is now imaged and needn't be imaged again. Maybe that's the wrong approach to the problem, but I feel like something needs to be done. 

The vast majority of disk images will *always* be .dsk images.  It's
too late in the history of the Apple II to change that fact.
Here's another idea: use plain .dsk and .nib, but put an encoded checksum in the filename, e.g. "name.{581BE17A}.nib". That's better than an in-file checksum, since it can be used to distinguish different versions of the same disk. Problem: it's not ProDOS-compatible. Even if it was a CRC-16 without the braces, you'd only have six characters left for the disk name. Why did we ever switch from DOS 3.3? 

Or don't worry about the fact that the image names are not ProDOS-
compatible names.  Images are generally used by emulators, or by
transfer programs that can present any "name" they want to an
attached or emulated Apple II.
Another idea: use 2mg, and put the checksum in the comment (in a distinctive textual encoding). Emulators won't see the checksum, so writing to the disk will invalidate the checksum, which is good. 


Aside from mode issues, I have never encountered file corruption in a
FTP/HTTP transfer that was reported successful.
Well, files get truncated during upload and often the uploader doesn't have permission to delete them (assuming they even try). That's detectable after the fact for .dsk and .nib, as you say. But I've also seen bit-flip errors. I've had bit-flip errors when copying from one hard drive to another on the same machine, due to some weird bus timing problem. Cosmic rays happen. There are lots of things that can go wrong en route, and the difficulty/impossibility of reimaging a disk years later seems to justify an end-to-end checksum (i.e. added at imaging time, and not removed until the file is "safely" archived somewhere). 

An overall checksum is good, but we have managed for quite a while
without them.  In fact, most computer files don't have checksums,
though they may have internal consistency checks.

If you have random bit-flip errors, it's time to work on your
hardware, since computers (and their users  ;-) are completely
intolerant of such things.


What we really need is a Disk II to USB adapter [...]



...and there's a thread on this very topic.



Huh, so there is. I hadn't been following c.s.a2.
But the discussion doesn't seem to be converging on a workable design. I don't see how it can possibly be as hard as people are making it out to be...
Code for accurate bit reading already exists---in software like Copy II+ if nowhere else---and I'm sure that somebody here knows exactly how it works. 


But, of course, you know that Copy II Plus is unsuccessful in copying
*many* disks without detailed parameter information tantamount to a
deprotection of the disk.
I assumed this was because of writing limitations rather than reading limitations, but maybe I'm wrong. 

Most of it is because there is no algorithmic way to determine what
data is relevant on a disk--some of it is the need for tailored
routines for writing.

The alternative of saving *everything*, including whether each nibble
(bit?) is "hard" or "soft" adds up to an order of magnitude more data
than a normal disk image (yes, I know most of the hard/soft bits can
be compressed).

I think most people who have considered this approach have weighed
the cost of additional storage versus the marginal value of saving
protected forms of programs, most of which have been deprotected,
and decided not to bother.

But it would still be interesting to do, and would enable a level of
preservation that is currently impossible for many disks.


And I think that many would like to be able to re-create physical
disks from "universal" images.
I do rather like the idea of hacking Copy II Plus's bit copier into two versions, one that writes to a file instead of a physical disk, and one that reads from a file instead of a physical disk. Then Copy II Plus could act as an archiver for any disk that it was capable of copying (with or without a parameter file), and the archives could be copied back to physical disks. Has anyone disassembled the bit copier, by any chance?
Problems: some of the param files patch the target to deprotect it, so this doesn't really make pristine copies in every case. Also, I seem to remember that Copy II Plus wasn't very reliable. Some of the parameter files say "repeat until boots". 

Correct.  Some parms are partial or full deprotects, and some are only
"copy" parms.

The reliability part comes from not having computer control over the
speed of the drive, and therefore the length of a track in bits, and
from a few other possibilities.

The real issue is that a well protected programs cannot be understood
by Copy II Plus without a lot of human assistance.


I published some code by Stephen Thomas that could read whole tracks,
including the "nibble lengths",



Thank you! I finally found it:

  http://groups.google.com/group/comp.sys.apple2/msg/c2fc21fb904df954
That's very pretty code. Kind of too pretty, actually. I can't see any way to modify it without breaking it completely. And as he says, it would be nice to fix some things. 

That's what's fun about cycle-counted code.  ;-)  If you want to change
it, you have to invent something new!


The problem is not in capturing the data, but in deciding what *is*
data.
Well, in that case, there's no problem at all, as far as I'm concerned. My only worry was that it wasn't possible to capture the data. 


My vote is to capture *everything* and let the program that reads the
image interpret it.



I agree -- that was my only goal in this thread.


But that's what gets us to a big factor size increase, too.
You don't need to simulate the full Woz machine, just add half track and quarter track seeking and update the data register at four-cycle intervals. 


But even that updating needs to be done in accordance with the
Woz state machine, since the reading program may be doing nibble
timing, and the Woz machine doesn't uniformly shift the register.
Implicit in my statement above was that the data register should be updated *correctly* at four-cycle intervals. :-) My point was just that the original state machine has to deal with jitter, whereas an emulator can assume a perfectly uniform data stream, which makes things much simpler. I don't see how any copy protection scheme could depend on jitter. 

That's not what I said.

The shift register is *not* updated every four cycles.  The update time
depends on the state of the state machine.   The "final" (high bit set)
state is held an extra-long time to allow the Apple II to read it
easily, and this syncopation would need to be emulated for programs
that poll the shift register and change state based on what they find.

Almost nothing about the Disk ][ is "simple"--except the hardware.  ;-)

Complicated software allows simple hardware.
I'm not going to worry about emulator support for writing to the canonical format, or about converting from the canonical format back to a physical disk. 


But it is vital to consider these requirements in designing a new
image "standard" for the Apple II.
Well, I was thinking that .nib would be good enough for disks that were going to be written to. Am I wrong? (I might well be.) 

I meant the part about converting from the canonical format back to
a physical disk.  And .nib's are not good enough for any disk that can't
be precisely represented by a .nib (sounds pretty tautological ;-).

As for "updates" to a canonical format disk image, I think that we
should only have to have *one* universal standard, so everything
reasonable that might be done to it should be considered in its
design.  Tradeoffs come later.

-michael

NadaNet networking for Apple II computers!
Home page:  http://members.aol.com/MJMahon/

"The wastebasket is our most important design
tool--and it's seriously underused."