Re: Rescuing old 5.25" disks

["Bryan Parkoff" <nospam@nospam.com>]

Ben,

    I appreciate your concern.  I agree with your view.  Unfortunately, it 
is not going to solve this issue.  It is good news for CFFA card with 
compact flash card.  It would be a big project for each DOS 3.3 images and 
copy-protected images be converted to ProDOS by modifying or rewrite 6502 
source code from scratch to support ProDOS.  It is bigger project to 
preserve all old game images.
    We should build DVD-ROM card for Apple II so all ancient images can be 
read from DVD-ROM disc into hard drive while playing.
    I think that DSK and PO without copy-protection is very good.  2MG is 
identical to DSK except a header is appended before DSK image.  I think this 
header may have important information about write-protection and 
track/sector structure.
    My goal is to build 2MHz board without depending on 6502 CPU.  It would 
read bit by bit stream from each track on disk and be converted to binary 
image which it is very accurate than cracking or deprotect copy-protection 
image such as Where in the world is Carmen Sandiego?  Earlier posts have 
discussed about this, but nothing is heard.
    It would be nice when someone is retired from work is willing to 
volunter and do a lot of work on this project like I do.

Bryan Parkoff

"Ben Rudiak-Gould" <br276deleteme@cam.ac.uk> wrote in message 
news:eh3518$3em$1@gemini.csx.cam.ac.uk...
> [note crossposting]
>
> I think everyone agrees that as many old Apple II disks as possible should 
> be imaged as soon as possible, before they degrade to the point of 
> unreadability. And many people are imaging their disks and putting them on 
> archives like Asimov, which is good.
>
> But most of those images are in the dsk/do/po or nib formats. That's bad. 
> It's bad because those formats fail to preserve information about the 
> disks that is sometimes important, and it's *really* bad for two other 
> reasons:
>
>   * Most of the tools that generate these formats don't do nearly enough 
> error checking, and the formats themselves have no checksum to guard 
> against corruption subsequent to imaging. There are probably dozens or 
> hundreds of subtly corrupt images on Asimov, and by the time anyone 
> notices the corruption it may be too late to reimage them. In many cases 
> it's probably already too late.
>
>   * Many disks can't be represented at all in either format. As far as I 
> know most games have been archived only in cracked versions, and while it 
> may be worth preserving those too, surely we shouldn't lose the originals 
> just for lack of an appropriate format.
>
> The reason these anemic formats dominate is obvious enough: they're by far 
> the most widely supported, including by many old tools that will probably 
> never support anything else. But that's entirely beside the point. The 
> first and most important goal here is to save as much information as 
> possible about the original disks, while that information is still 
> available. Once the disks are imaged in *any* reasonable format, it's easy 
> to write portable open-source tools for modern machines that can 
> interconvert between that format and formats like dsk and nib, for 
> compatibility with existing tools. With a good disk image format, 
> everybody wins.
>
> == format ==
>
> The canonical format needs to record "all relevant information" about the 
> disk. In general this means bit-accurate track data (including length) for 
> each full, half, and quarter track, inter-track synchronization 
> information, and the status of the write-protect tab. It might be nice to 
> have a field to hold information about the disk label also (at least text, 
> maybe even a picture). Am I forgetting anything? Of course, the half and 
> quarter tracks on most disks are just random noise, so it should be 
> possible to leave them out. (The problem being figuring out when it's safe 
> to leave them out.) In the case of ordinary unprotected 16-sector (and 
> 13-sector?) disks, it's probably sufficient to store the decoded sectors. 
> This does lose information, but probably not information that really needs 
> to be saved. Again, though, it's hard to detect algorithmically that this 
> is safe.
>
> I think that the format needs a checksum. In theory this should be the 
> responsibility of storage and transfer protocols, but it's a sad fact of 
> life that common protocols like FTP and HTTP POST, as implemented in 
> practice, frequently corrupt files. An easy way to add a checksum and also 
> keep file size down is to zip or gzip the image after the fact. That's 
> probably a good idea in any case, but I still think the format itself 
> needs an internal checksum.
>
> Are any suitable formats already defined?
>
>   * FDI 2.0 supports bit-accurate tracks. I think it has enough support 
> for synchronization. I think it could support half and quarter tracks by 
> putting 96 or 192 in the TPI field. It has a comment field, but it's 
> limited to 80 characters. Frustratingly, it doesn't have any checksums.
>
>   * 2MG supports very little right now, but could easily be extended. The 
> main problem is that it then becomes unclear what "supporting 2MG" means. 
> You couldn't tell from the filename whether it's a legacy 2MG or the new 
> format. Probably it would be a good idea to invent a new file extension.
>
> == imaging tools ==
>
> What we really need is a Disk II to USB adapter which samples and buffers 
> the raw disk signal in much the same way as a sound card, for subsequent 
> signal analysis on a modern PC. But that's way outside my area of 
> expertise, so I'll talk about utilities running on a real Apple II.
>
> The canonical imaging utility needs to work, automatically and reliably, 
> for basically any disk, protected or not. That may make it slow, but no 
> matter how slow it is, it's much faster than cracking copy protection by 
> hand, or, worse, trying to figure out whether an image is corrupt after 
> the fact. More importantly, most people don't have those skills, and those 
> people probably have disks that the people with the skills don't.
>
> I don't think that this is terribly hard. Code for accurate bit reading 
> already exists---in software like Copy II+ if nowhere else---and I'm sure 
> that somebody here knows exactly how it works. The problem we're trying to 
> solve is much easier than the one solved by Copy II+, because we don't 
> have to write the data back to a physical disk.
>
> I seem to recall that some bit-reading code was posted to c.e.a2 recently, 
> but diligent searching on Google Groups has failed to turn it up. I did 
> find this article,
>
> 
> http://groups.google.com/group/comp.emulators.apple2/msg/7ff7480ee467fed6
>     Message-ID: <2tvt57$7ge@lace.Colorado.EDU>
>
> in which Chris Redmond wrote, way back in 1994,
>
>> If you are serious about writing an emulator that can use a raw bitstream
>> disk image, I can write a disk reader for the Apple// that will produce
>> a disk image containing an exact bitstream that can be calibrated for
>> spiral tracks &c.
>
> which at least suggests that it's possible to do all this without special 
> hardware. So, did he or anyone else ever write this code? It's frustrating 
> that nothing like this seems to exist over a decade later.
>
> The imaging utility also needs to be able to either save the image to a 
> local disk (3.5" or hard), or send it over a serial connection. The former 
> is easy. The latter is already done by ADTPro, which is an open source 
> project. At least, it's open-source in theory, but I haven't actually 
> found the source code for the client. The package on Sourceforge only has 
> a dsk-format image, which contains only the binary. Assuming source code 
> for the client exists, it seems most sensible to add an accurate-imaging 
> extension to ADTPro rather than write a whole new application.
>
> Unless ADTPro already does this, it should also be modified to read every 
> block twice for verification when making a cooked image, and it should not 
> be possible to turn this off. Yes, this will slow things down, but it's 
> really really worth it. It could be sped up by writing an efficient 
> whole-track reader, using code like this:
>
> 
> http://groups.google.com/group/comp.emulators.apple2/msg/fe3909783963507d
>     Message-ID: <t4eoa2hg9bavhmhgpdsuic0fr5noppio54@4ax.com>
>
> == conversion tools ==
>
> There are lots of format-conversion utilities on Asimov, but unfortunately 
> most of them are closed-source or system-specific or both. There needs to 
> be a free (as in speech), portable conversion utility written in C that 
> supports at least dsk, nib, 2mg, and the canonical format, whatever it is. 
> I can write this from scratch if necessary.
>
> == direct support ==
>
> Obviously it would be nice if some emulators supported the canonical 
> format directly, partly because of convenience but mainly because it's the 
> only way to support many copy-protected disks. I don't think that adding 
> this support would be very hard. You don't need to simulate the full Woz 
> machine, just add half track and quarter track seeking and update the data 
> register at four-cycle intervals.
>
> I'm not going to worry about emulator support for writing to the canonical 
> format, or about converting from the canonical format back to a physical 
> disk. Not that those wouldn't be useful features, but in the short term 
> I'm interested mostly in archiving old disks.
>
> == conclusion ==
>
> This needs to be done. It should have been done a decade ago, but it's 
> never too late. I don't think it's very hard. Comments requested.