Re: A little compression...

["Jeff Blakeney" <jeff.blakeney@a2central.com.remove-izv-this>]

  To: Harry Potter
Instead of posting multiple replies to this thread I figured I would
gather all my replies together in a single one.

On Fri, 23 May 2008 08:43:53 -0700, Harry Potter wrote:

> I am in the process of creating a file compression program.  It
> already does better than PKZip's deflate/maximum technique, but I
> still have work to do on it.  It is a 32-bit technique, but I think I
> can manage on a 6502/C02, although very slowly.  My goal is an average
> 3:1 compression ratio.  If I succeed, would anybody here be interested?

Back before the LZW compression (the one used by ShrinkIt) algorithm's
patent expired, I wanted to write a IIgs archiving program that would
have plug ins for various formats.  The ShrinkIt or NuFX format would
have been the first plug in I would have created but then I would have
liked to add others like ZIP and the older formats of ARC, LHA, LZH and
so on.

It would probably still be a handy thing to have on the IIgs.

On Fri, 23 May 2008 13:24:33 -0700, Sam Hobbs wrote:

> "Dombo" <dombo@disposable.invalid> wrote in message 
> news:4837057b$0$24399$5fc3050@news.tiscali.nl...
> > Sam Hobbs schreef:
> >>
> >> I would search the internet for the various algorithms.
> >
> > You also need data sets, certain compression algorithms (or 
> > implementations thereof) work better with certain data than others.
> 
> I don't understand that. I especially don't understand why we need data sets 
> to understand algorithms.

Different types of data compress differently.  Text files are usually
highly compressable by pretty much any means but audio and video data
tend to require specialized compression algorithms to get good
compression ratios.

For example, the MNP5 compression used by modems back before the
Internet became wide spread would compress text files up to four times
smaller.  Binary files would usually max out around two times.  The big
advantage of MNP5 over its predecessors was that it was smart enough to
stop compressing data when it saw that it couldn't compress it further
because generally, compressing already compressed data would end up
making the data larger.

On Fri, 23 May 2008 15:50:10 -0700, Harry Potter wrote:

> I have to apologize to CBM and Apple 2 owners: this compression
> technique won't work on 8-bit computers, and probably not on 16-bit
> computers as well: it uses a 32k dictionary size, and uses C++.  It is
> also memory-intensive: the Win32 executable is of now 63k in length.
> I need to convert some code to assembler for speed--I used C++ code
> for testing.

If your 32 bit code is at 63 KB in length it doesn't put it out of the
realm of an 8 bit machine.  Keep in mind that 32 bit code also uses 32
bit opcodes and can use 64 bit address operands.  If you drop those down
to 8 and 16 bits, you save a lot of space right there so as long as you
don't need four 8 bit instructions for every 32 bit instruction you
currently have, the executable will be smaller on an 8 bit machine.

Memory use can also be controlled by only dealing with files by reading
in small chunks, compressing that, writing out the compressed data to a
new file and repeating.  If you are dealing with only 1 KB of data at a
time, the memory requirements go down but all the extra disk access will
slow the program execution down.

For 16 bit machines, I don't see there being a problem at all.

On Fri, 23 May 2008 21:34:17 -0700, Sam Hobbs wrote:

> It is something that is off-topic for this newsgroup. The person asking the 
> question will get more meaningful discussion elsewhere.

Apple II machines use various compression formats all the time so I
don't see why this would be considered off topic.  SHK, ZIP, and GIF as
well as other formats are used on Apple IIs quite regularly.
--- Synchronet 3.14a-Win32 NewsLink 1.85
A2Central.com - Your total source for Apple II computing.