Re: NadaNet 3.1 Released

[mdj <mdj.mdj@gmail.com>]

On May 29, 12:13 pm, "Michael J. Mahon" <mjma...@aol.com> wrote:
> Hi, Matt!

Greetings :-)

> > (Of course, another barrier is having enough 'spare' Apple IIe
> > motherboards to build a 'Crate; I rationalised my collection and
> > probably want to keep the remaining IIe's 'intact'. If I happen upon
> > an auction for a large set and can bare the shipping expense, I'll add
> > them to (yet another) box.)
>
> It's worth underscoring the fact that it is not necessary to disassemble
> Apple II's to use them as part of a logical 'Crate.  NadaNet connects
> all machines on the net as peers, and any can serve as a "board" of an
> AppleCrate, if you will.
>
> In fact, the AppleCrate is *only* a convenient mechanical packaging of
> a number of //e machines.  It does make it necessary to deal with the
> *inconvenience* of not having any attached I/O (except NadaNet) by
> requiring that NadaNet boot ROMs be installed!  None of this is needed
> for a "normal" Apple II.
>
> Any Apple II with a game port can participate in a NadaNet as a full
> peer simply by booting a NadaNet startup disk.  (It can then continue
> to be used separately as a normal //e, participating in the network
> only when: 1) a NadaNet command is entered manually or executed as part
> of a program, or 2) the "service loop" is started on the machine so
> that other machines can talk to it.

True enough. I guess I can find the desk space to stack a couple of
machines on.. Now to find my *desk* ...

> > Without wanting to be in any way critical (at least for negative uses
> > of the term), there are issues with having CrateSynth as the primary
> > "marketing tool" for NadaNet. From an end-user perspective it's a
> > "passive" application; it plays at you rather than you playing with
> > it, and you can enjoy its fruits straight from your website without
> > ever having to build one yourself ;-)
>
> ;-)  It's a demo!  ;-)

And a brilliant one!

> > From a developers perspective, which I think is the more pertinent,
> > CrateSynth is an "embarassingly parallel" application. By virtue of
> > the Apple II's deterministic timing, the workers only have to
> > rendezvous at the start of playback. This is of course very important
> > to demonstrate since parallel deterministic processes is NadaNet's
> > most unique quality, but it doesn't serve as an inspiration to
> > developers who may be considering problems that aren't embarrassingly
> > parallel.
>
> True.  In fact, there are a wealth of embarrassingly parallel problems,
> so one still has lots of choices...  But, of course, problems with the
> most interest are those in which the parallelism must be organized
> carefully to minimize communication, and those are not the easy ones!
>
> > As we've discussed at length over the years, parallel programming
> > really is the new frontier in software development, and the vast
> > majority of the programming fraternity remains hopelessly unable to
> > exploit the potential power. This includes, unfortunately, current
> > consumer class machines that contain 2-8 processing units with shared
> > fast memory! Having the technology available to begin to experiment
> > has resulted in very little software in the wild having anything more
> > than 'token' support multiple procesors (embarrassingly parallel
> > problems excluded), so the adoption issues you're suffering from
> > certainly aren't unique to NadaNet.
>
> Amen to that.   As we've discussed, that problem is perhaps a generation
> away from a sweet solution (likely involving new programmers to write
> the new programs ;-).

I think you're probably right. There's been a couple of tentative toes
placed into the water by some - Apple's recent C compiler work for
instance has introduced 'blocks' (think closures) that can be passed
around as is the norm in functional languages, and the latest version
of MacOS can exploit multiple cores to realise the implicit
parallelism of evaluating a closure within the current execution
frame. The marketing name given to this is "Grand Central Dispatch"

(NB Real Grand Centrals like the one in NYC probably do a better job
of handling concurrency than the computing fraternity)

> > Since sequential programming already demands so much of our spatial
> > reasoning, I guess it shouldn't come as a surprise that there's so
> > little left over to consider time as a two dimensional quantity. My
> > own experience with parallel application development over the years
> > has taught me that even the most innocuous looking construct, say
> > using a mutex to serialise access to a resource (the proverbial
> > critical section) is fraught with hidden complexity.
>
> Absolutely.  Most people underestimate the growth rate of N squared,
> let alone N factorial!  The number of possible things that can happen
> at even a mutex grows very rapidly with contention, and the best
> algorithms for handling sequential re-use change, too.

For this reason, I'm almost convinced that many of the cores we're
about to sprout will end up doing static analysis to determine 'on the
fly' how to put the other cores to use.

> > Given that, I think it would probably beneficial to provide a small
> > collection of contrived examples. How do we implement serialised
> > access to a resource? What about many readers but only one writer? A
> > NadaNet equivalent of the Dining Philosophers, etc.
>
> The Message Server is a very simple mechanism for dealing with queuing,
> with single or multiple writers and single or multiple readers.  The
> messages are limited to 255 bytes, but only a few bytes are needed to
> orchestrate exclusive serial re-use, or even more complex interactions.
>
> For those who are willing to trade the simplicity of message passing
> for the efficiency of direct, synchronous communication, there are
> primitives like &PEEKPOKE for semaphoring and &PEEKINC for simple
> allocation.
>
> Essentially all of my programming efforts have been directed toward
> the creation of tools, which may not appeal to some on the face of it,
> but if you look into the structure of the tools, you find methods for
> handling very common problems.  ATTACH, the "remote console" program,
> is a good example.  It uses 256-byte circular buffers to pass input
> character streams and output characters into and out of the serving
> machine, respectively.  On the (remote) serving side, a small machine
> language program manages the communication, while on the client (user)
> side, analogous communication is handled by an Applesoft program.
>
> The methods for handling the circular buffer are quite general, and
> would be applicable to many programs.
>
> The first version of ATTACH used the Message Server, which handled the
> necessary buffering, but the latest (circular buffer) version does not
> require a Message Server, and so is applicable to even a 2-machine net.

Right. So many of the infrastructure tools to pull off distributed
development tools already exist ...

> > Anyway, enough of that and onward to more positive things and perhaps
> > a discussion point: What could be done with 1 (or perhaps 2) slave
> > machines attached to a developers primary Apple II? This is a
> > configuration I think most of us could scare up easily
>
> NadaPong was a (very) simple demonstration of two machines cooperating
> on a peer-to-peer basis to support a simple game.  It trivially extends
> to more machines with a single ball, or with more thought, to multiple
> sprites moving simultaneously.
>
> I was motivated to put it together after lunch because it was suggested
> to me that something that graphically illustrated the communication that
> was occurring would be more interesting and thought-provoking.

I remember your demo well!

> The fundamental problem with a gaming example is the natural emphasis
> on real-time interaction.  It is axiomatic that machines can do only
> one *real* real-time task at a time, and NadaNet is itself a real-time
> task, so there are tricky problems to be overcome if state spread across
> the network needs to be "displayed" in some sense on multiple machines
> in "real-time".
>
> It's fun to solve problems like this, but mainly because it *isn't*
> easy!  As a result, it's probably not the best way for a NadaNet
> programmer to get their feet wet.  ;-)

In my university days I tutored concurrent programming. We had the
students build a virtual 'widget factory' in which conveyors and
processing points where represented as message oriented queues and
threads. It was a fun contrived example, and students quickly
discovered that doing a 'fancy' real-time display of the factory
presented a more difficult problem than the simulation itself ;-)

> >                          and since we're
> > talking developers how about ...
>
> > An interactive, incremental,  Editor/Assembler/Debugger ?
>
> Such an application would be straightforward if starting from scratch,
> developing ones own apps.  It's probably not so easy to do by adapting
> existing applications.

Agreed.

> Sending an assembly or compilation to another machine raises the issue
> of a common file system.  Although my File Server provides stateless
> access, it does not support OPEN..(READ | WRITE)..CLOSE file access,
> nor does it support sector- or block-level access.  It handles the
> subset of ProDOS commands that do not require keeping state between
> operations, for which the atomicity of command execution is sufficient
> to ensure consistency.
>
> As a result, it is well suited to supporting programs written to use
> BLOADs and BSAVEs for disk I/O (which are quite sufficient), but it does
> not support programs that depend on the existing DOS or ProDOS command
> set beyond the stateless ones.
>
> If the text editor (B)loads the file into memory and (B)saves it back
> to disk, then it could easily be modified to do so using remote File
> Server(s), which could then host the assembler(s) and/or compiler(s)
> on its/their local file systems.  It would be a File Server that also
> was an Assembler Server, for example.

Right. I was more thinking of a more esoteric approach where the text
editor would send 'delta' information to the message server, which an
assembler agent then subscribed to. In this way, the assembler agent
could incrementally 'assemble' each delta against the already accrued
symbol table, and send messages back which the editor could then use
to provide edit-time notification of errors/warnings.

Note that a 'delta' could be single line, a copy/paste chunk, or an
entire file that's just been loaded. a simple command could switch you
to the monitor, where your assembled work is quickly shipped to the
correct locations and your source shipped 'elsewhere' on the NadaNet
for quick recall when you're done testing.

The 'challenge' is in keeping and then storing assembly metadata
(symbols *and* their references) in a structure suitable for fast
lookup such that a delete/inserts ramifications can be quickly
calculated. I admit though, this is an ambitious undertaking and
there's a few different ways to skin the cat ...

> A Print Server could be similarly implemented, assuming that it was
> only necessary to print disk files.  (Of course, many Apple II apps
> are not structured to place a disk file between the app and its printer,
> so that could also be an issue for existing apps.)
>
> You raise an interesting point--I have one //e that is configured for
> general use:  editing, software development, etc., with a CFFA.  I have
> so far not set up another capacious machine as a slave to run larger
> apps, like assembly and compilation.  This sounds like it could be fun!
>
> It will require modifying an editor to do its file I/O using the File
> Server, but that shouldn't be hard if it is reasonably structured...

I don't think modifying Merlin 8 to do this would be that tricky...

> The more generic solution, providing patches to ProDOS and DOS to access
> files stored on a remote machine, raises several issues (multi-machine
> state, file and file system sharing, etc.) that I have chosen not to
> address.  Perhaps someone more comfortable with the internals of these
> OSs would like to try for something more general?

I would like to see this happen. I do worry though given the
synchronous nature of I/O on an Apple II that such a feature might
quickly become too powerful to be useful, if you know what I mean...