Re: RFC : SOME IDEAS FOR THE APPLE II FPGA'ers

["Michael J. Mahon" <mjmahon@aol.com>]

Mark McDougall wrote:
> Michael J. Mahon wrote:
>
> > I spent a lot of time trying to convince a bunch of software
> > engineers that their problem--more than half of their time to release
> > a product revision spent in system reliablilty testing--could be
> > largely solved by creating a "test scaffold" to simulate the running
> > environment of each major module, and feed it errors, race
> > conditions, and load under the direction of test scripts.
>
>     (snip)
>
> > In short, I failed.  I could not convince a single project manager to
> >  take this approach, even though it provided the single best
> > opportunity to reduce time-to-market.  Too different, might not work,
> > and--the big one--"what will happen to me if it doesn't work?"
> >
> > It's much "safer" to keep doing the same old thing that works very 
> > poorly than to take a risk on a new method that could revolutionize 
> > the development process.
>
>
> My first degree was in computer science, so I had the formal education 
> similar to I guess a lot of other 'software engineers' out there. 
> Despite taking a course in theoretical computing which covered formal 
> software verification proofs, I have to admit that 'testing' was viewed 
> as a necessary evil and something done after-the-fact without much of a 
> formal process. And so my early years in computing were similar to the 
> scenario you have encountered.

Yes--another measure of the immaturity of programming as a discipline.
In fact, the way it's *usually* done, it doesn't deserve to be called
software engineering at all...

> However, I went back to study Electrical Engineering and have in more 
> recent times been working more and more with hardware and firmware. In 
> my recent projects, the emphasis on design verification has meant that 
> our testbenches are several times larger than the design we're testing, 
> and the time and effort gone into coding the tests has been at least 4 
> times that of coding the core design.

I've had the same experience--and spent some time figuring out how to do
similar "verification" of software components.

> Having gone through the process, the only conclusion I can make is that 
> it is *more* than worth the effort. We packed an FPGA with a handful of 
> IP cores, proprietary IP, and interfaced it all to a proprietary system 
> board. More than 12 man-months of work went into the whole process 
> before we finally got to power-up the design - and the effort since 
> (aside from enhancements) could be measured in a few *weeks*.

The very point I tried (apparently unconvincingly) to make.

> How often do you hear of large software projects being stuck in the last 
> 5% for twice as long as the original project span?

In fact, it's the standard "Now that we've completed 90% of the project,
we only have the second 90% to go."

> It's unfortunate that your colleagues can't be convinced otherwise (as 
> you point out I'm sure it has a lot to do with self-interest and 
> ass-covering) but it's not uncommon in the software industry - it's an 
> immature discipline and can still learn a lot of lessons from other 
> disciplines I think.

Absolutely.  But the precondition for learning is acknowledged
ignorance.  That prevents most people, and almost all organizations,
from moving forward.

> There's certainly software houses out there doing exactly what you 
> proposed - and some of them are *big*. I've never heard of anyone trying 
> what you propose and then reverting to ad-hoc debugging because it 
> didn't work. And some radical approaches such as "extreme programming" 
> also touch on it.

Right--my little corner of the world went to extreme programming after
we relocated away from the old culture.

> Maybe they didn't like your idea of having a big flashing light on the 
> cubicle of the last person to submit buggy code to the repository?!? ;)

You're right, but that's another story!  ;-)

I always liked the idea that "if you break the build, you don't go home
until it's fixed."  ;-)

-michael

Music synthesis for 8-bit Apple II's!
Home page:  http://members.aol.com/MJMahon/

"The wastebasket is our most important design
tool--and it is seriously underused."