Re: RFC : SOME IDEAS FOR THE APPLE II FPGA'ers

[Mark McDougall <markm@vl.com.au>]

Michael J. Mahon wrote:

> I spent a lot of time trying to convince a bunch of software
> engineers that their problem--more than half of their time to release
> a product revision spent in system reliablilty testing--could be
> largely solved by creating a "test scaffold" to simulate the running
> environment of each major module, and feed it errors, race
> conditions, and load under the direction of test scripts.
	(snip)
> In short, I failed.  I could not convince a single project manager to
>  take this approach, even though it provided the single best
> opportunity to reduce time-to-market.  Too different, might not work,
> and--the big one--"what will happen to me if it doesn't work?"
>
> It's much "safer" to keep doing the same old thing that works very 
> poorly than to take a risk on a new method that could revolutionize 
> the development process.

My first degree was in computer science, so I had the formal education 
similar to I guess a lot of other 'software engineers' out there. 
Despite taking a course in theoretical computing which covered formal 
software verification proofs, I have to admit that 'testing' was viewed 
as a necessary evil and something done after-the-fact without much of a 
formal process. And so my early years in computing were similar to the 
scenario you have encountered.

However, I went back to study Electrical Engineering and have in more 
recent times been working more and more with hardware and firmware. In 
my recent projects, the emphasis on design verification has meant that 
our testbenches are several times larger than the design we're testing, 
and the time and effort gone into coding the tests has been at least 4 
times that of coding the core design.

Having gone through the process, the only conclusion I can make is that 
it is *more* than worth the effort. We packed an FPGA with a handful of 
IP cores, proprietary IP, and interfaced it all to a proprietary system 
board. More than 12 man-months of work went into the whole process 
before we finally got to power-up the design - and the effort since 
(aside from enhancements) could be measured in a few *weeks*.

How often do you hear of large software projects being stuck in the last 
5% for twice as long as the original project span?

It's unfortunate that your colleagues can't be convinced otherwise (as 
you point out I'm sure it has a lot to do with self-interest and 
ass-covering) but it's not uncommon in the software industry - it's an 
immature discipline and can still learn a lot of lessons from other 
disciplines I think.

There's certainly software houses out there doing exactly what you 
proposed - and some of them are *big*. I've never heard of anyone trying 
what you propose and then reverting to ad-hoc debugging because it 
didn't work. And some radical approaches such as "extreme programming" 
also touch on it.

Maybe they didn't like your idea of having a big flashing light on the 
cubicle of the last person to submit buggy code to the repository?!? ;)

Regards,

--
Mark McDougall, Engineer
Virtual Logic Pty Ltd, <http://www.vl.com.au>
21-25 King St, Rockdale, 2216
Ph: +612-9599-3255 Fax: +612-9599-3266