Re: Integer Basic Tokenization

[pausch@saaf.se (Paul Schlyter)]

In article <98rh8a$ead$1@news7.svr.pol.co.uk>,
Thug <thug5@optusnet.com.au> wrote:
 
> I'm currently writing a utility to manage my Apple ][ disk images. As part
> of this I'm going through a lot of my old personal disks that I created when
> I was first learning computers in the early 80s, including a lot of BASIC
> programs I typed in from magazines and so on. (Remember when you used to be
> able to do that? Computer Magazines came with "Special 16 page pull-out
> program supplements", not DVDs full of code ready to go.)
> 
> Anyway, I digress...
> 
> I've sucessfully written a program to take an APPLESOFT program and
> de-tokenize it back into readable text, and it works an absolute treat. But
> INTEGER BASIC programs are proving a little bit more difficult, and I was
> wondering if somebody out there could shed some light on the process it
> uses.
> 
> So far I have the structure being like this:
> 
> 1 Byte: Length of Line
> 2 Bytes: Line Number (Lo/Hi Order)
> ? Bytes Tokenized Program
> Last Byte: 01 (End of line token)
 
Correct
 
> That's not too bad and is quite similar to Applesoft except that things like
> quotes are tokenized and plain text has the high bit set. But once numbers
> start appearing in the code, things get really messy. INTEGER appears to
> encode all numbers too, whereas APPLESOFT just has them as plain text.
 
Correct: Integer Basic eocnides all integer constants as 16-bit
binaries, prefixed with a byte B0-B9.  Interpretation gets quicker if
integer constants need not be converted from ASCII each and every
time they are encountered in the code.
 
..............
 
> Can anyone help? (And just a list of tokens would be helpful!)
 
Visit my apple 2 page at:
 
    http://hotel04.ausys.se/pausch/apple2
 
and download my utility FID, which comes with C source.  It contains
de-tokenizers for Applesoft Basic, Integer Basic, and S-C Assembler
source files (the latter are stored as "I" type files too!).  That
code should help you figure out how INteger Basic programs are
tokenized.
 
-- 
----------------------------------------------------------------
Paul Schlyter,  Swedish Amateur Astronomer's Society (SAAF)
Grev Turegatan 40,  S-114 38 Stockholm,  SWEDEN
e-mail:  pausch at saaf dot se   or    paul.schlyter at ausys dot se
WWW:     http://hotel04.ausys.se/pausch    http://welcome.to/pausch