UNIX/Apple textfile/newline problems? Some remedies follow..

[sbdocker@acs.ucalgary.ca (Sean Brendan Dockery)]

A little while ago, Brian Tao complained about munged files on comp.-
binaries.apple2.  I tried to post this from another site, but it
apparently wasn't circulated -- it didn't show up at the university.

In article <CBpywC.9EJ@wave.scar.utoronto.ca>
90taobri@wave.scar.utoronto.ca (TAO BRIAN T) writes:

|      ARGH!!!!!  You are posting CR-terminated Binscii files!  UNIX text
| files end with LF's!  News software will see your Binscii files as one
| long line and break it up into 255-character chunks.  Same thing
| happened the second time you posted it, so _no_ you are _not_ getting
| any better at this!  Not to mention posting an outdated version (0.6) to
| begin with... Wanna know how much money you just wasted?
| 
| (ignore me, I'm in a pissy mood today...)

Well, rather than ignore you -- how about a couple of shell scripts
for the benefit of the wider audience?  I have posted C shell and Korn
shell versions for those people with various flavours of unices.

Following are three scripts:

- striplf: concatenate lines ending with UNIX newlines (LF's) caused
	by being truncated by news software.
- aii2u: convert Apple // newlines (CR's) to UNIX newlines (LF's).
- maketext: strip bit seven of every byte in a text file making it
	completely text.

They are invoked from the command line by

	<scriptname> <file(s)-to-convert>

Output files retain the same names as input files.

NOTE: All of these scripts are for use on text files only.  Because of
their nature, they will corrupt binary files (or at least make them
quite unpredictable).

----------------------- striplf (csh version) -----------------------
#!/bin/csh

# striplf (csh version): strip UNIX newlines from a file that has had
#  line lengths truncated to 255 characters.

if ( $#argv == 0 ) ; then
	echo "usage: $0:t <filespec>"
	exit 1
endif

foreach file ($*)
	(cat $file | tr -d '\012') > /tmp/xxx
	mv xxx $file
end
----------------------- striplf (ksh version) -----------------------
#!/bin/ksh

# striplf (ksh version): strip UNIX newlines from a file that has had
#  line lengths truncated to 255 characters.

if [ $# = 0 ] ; then
	echo "usage: ${0##*/} <filespec>"
	exit 1
fi

for file in $*
	cat $file | tr -d '\012' > /tmp/xxx
	mv xxx $file
done
---------------------------------------------------------------------

The second one will convert Apple // newlines (CR's) to UNIX newlines
(LF's).

------------------------ aii2u (csh version) ------------------------
#!/bin/csh

# aii2u (csh version): script to convert files with Apple // newlines
#  to files with UNIX newlines

if ( $#argv == 0 ) ; then
	echo "usage: $0:t <filespec>"
	exit 1
endif

foreach file ($*)
	(cat $file | tr '\015' '\012') > /tmp/xxx
	mv /tmp/xxx $file
end
------------------------ aii2u (ksh version) ------------------------
#!/bin/ksh

# aii2u (ksh version): script to convert files with Apple // newlines
#  to files with UNIX newlines

if [ $# = 0 ] ; then
	echo "usage: ${0##*/} <filespec>"
fi

for file in $*
	cat $file | tr '\015' '\012' > /tmp/xxx
	mv /tmp/xxx $file
done
---------------------------------------------------------------------

I also have problems with some of the files that I FTP.  Some of them
come down the line with the eighth bit of every byte set (making text
files quite unruly when trying to edit for brevity with emacs).

----------------------- maketext (csh version) ----------------------
#!/bin/csh

# maketext (csh version): strip bit seven of every byte in input
#  file(s). 

if ( $#argv == 0 ) ; then
	echo "usage: $0:t <filespec>"
	exit 1
endif

foreach file ($*)
	(cat $file | tr '[\200-\377]' '[\000-\177]') > /tmp/xxx
	mv /tmp/xxx $file
end
----------------------- maketext (ksh version) ----------------------
#!/bin/ksh

# maketext (ksh version): strip bit seven of every byte in input
#  file(s). 

if [ $# = ] ; then
	echo "usage: ${0##*/} <filespec>"
	exit 1
fi

for file in $*
	cat $file | tr '[\200-\377]' '[\000-\177]' > /tmp/xxx
	mv /tmp/xxx $file
done
---------------------------------------------------------------------

NOTE: In some cases, people have their "noclobber" variable set in
their C shell environment (or the equivalent for the Korn shell).  In
these cases, these scripts will fail.

It is necessary to make the following modifications to "force" them to
work in such conservative environments.

For C shells, change

	from			--->	to
	-----------------------		---------------------
	... > /tmp/xxx		--->	... >! /tmp/xxx
	mv /tmp/xxx $file	--->	mv -f /tmp/xxx $file

For Korn shells, change

	from			--->	to
	-----------------------		---------------------
	... > /tmp/xxx		--->	... >| /tmp/xxx
	mv /tmp/xxx $file	--->	mv -f /tmp/xxx $file

All of these scripts just need to be put in a directory in your $PATH
variable on the UNIX host.  If you haven't already set up a $HOME/bin
directory for yourself (created the directory and added $HOME/bin to
your $PATH variable), now would be a good time.

For csh, add the following to your .cshrc file:

	set path=($HOME/bin $PATH)

For ksh, add the following to your .profile file:

	export PATH=$HOME/bin:$PATH

I did not post this to comp.sources.apple2 in light of the fact that
some people who don't have access to that newsgroup, but who might be
using GNO, would be able to use these.  The csh scripts should be
completely compatible.

I hope that *someone* out there finds these scripts useful.  :)
-- 
 internet: dockery@griffin.cuc.ab.ca
      -or- sbdocker@acs.ucalgary.ca
      -or- dockery@pro-calgary.cts.com
 pro-line: dockery@pro-calgary