Re: A 21st Century Apple II?

[mdj <mdj.mdj@gmail.com>]

On Mar 9, 9:36 pm, JB <jasonb1...@yahoo.com> wrote:

[...]

> For the 14 benchmarks available, the results on the graph displayed
> show that Java 6 -Xms64m ranges anywhere from around 40% faster (for
> binary-trees) to 900% slower (for partial-sums).  The average
> difference is probably not far off from a factor of 2 faster, but I'm
> too lazy to calculate it exactly.

To me, what is interesting is that so many of the results *cluster*
around quite a small difference.

> I see that if you use the best of the five or six variations of Java
> and compare against the best of the two versions of C++, the
> performance difference would probably narrow somewhat.  In that case,
> I'd suggest that two new versions of C++ code be produced where the
> startup code is configured to pre-allocate 64 megabytes on the heap.
> I'm guessing that this would eliminate the relative advantage that Java
> has with the -Xms64m argument.

I checked the memory size reported in the benchmark which shows
39,436KB, a good deal less than 64MB. How can this be if the memory is
pre-allocated? It can't of course. Rather than pre-allocating space,
the heap size parameters feed the VM data about the expected heap
usage. In this case, what it probably means is that for this problem
and a 64MB heap, the "Eden" size is adequate enough that the VM isn't
going to re-organise it's heap structure during the programs runtime.

It's important to recognise that the term 'heap' has somewhat
different meanings in the context of C++ vs. Java. A C/C++ 'heap' is
really a pointer to the processes data segment, which the platforms
malloc/free implementation will manage as it sees fit, informing the
OS of allocations, etc. In terms of a JVM, the process level heap is
used by many different subsystems within the VM itself (JIT compiler,
etc) as well as the running programs (of which there may be more than
one). Performance loss can be avoided by telling the VM up front that
the program its about to run is going to exceed its fairly
conservative default guesses.

In a real world program, the costs are either amortised over the
programs entire execution time or simply don't exist, depending on the
level of startup time tuning you're prepared to do.

In the binary tree benchmark, the performance difference can be
attributed to garbage collection. In fact, it's probable that
particular program completes without any collection having been done.
In a much longer run the collector would kick in, and if you've only
got one processor to do the work, you'll probably pay a higher cost
for it than the equivalent C++ program. On an SMP system, you'll
probably get a win for the Java implementation.

> In any event, I'm interested to know what Matt has to say about this,
> because maybe Im missing something.

There are a number of other things that could account for the
difference. Calls to the operating system for a chunk of ram are just
one part of the equation: the languages runtime level allocator still
has to divvy up the ram chunks to each object and track it. It could
just be that in this particular case, the JIT compiler recognised that
X number of objects was going to be created in the bounded loop, and
slab allocated the whole thing. If what you're saying about the C++
implementation is true (and I'm pretty sure it's not) then it's
certainly not doing that optimisation.

Bottom line: we don't actually know unless we profile the generated
code. And phew! we still havn't even examined the sum-file benchmark
yet!

> > My selection metric was simple: display all the results, and pick the
> > best result for each language, and use that as the basis for
> > comparision. Fairly obviously, if it's performance we want to compare,
> > this is the only useful metric.
>
> It seems to me that both of you should have made your selection
> criteria clear from the beginning.
>
> Please also keep in mind what I said about about the -Xms64m argument.

I think I've addressed it above. If not, I've probably got one more
round in me ;-)

> >> If you want to compare the performance of the GNU compiler versus the
> >> Intel compiler, then look at the average performance across all
> >> benchmarks for the GNU compiler and the Intel compiler.  If you do the
> >> math, you'll see that the Intel compiler is significantly faster.
> >> Congratulations, you have located a statistical anomaly above.
>
> > And that's exactly it. If you do the math and compare the best results
> > of the two languages you'll see that your entire conclusion is based
> > upon such a statistical anomaly.
>
> To me it looks like his conclusion is based on a different selection
> criteria for the benchmarks.

No doubt.

> > I tend to find that threads that become large and develop digressions
> > are good indicators of the level of interest in the original point and
> > don't find it particularly bothersome to filter out that which I
> > consider "noise". And whilst it may be considered an unwanted by some
> > the fact remains that if you stand on a public soapbox, you are going
> > to be heckled by the crowd. For future reference you might find it
> > easier to simply retract a fallacious remark then to continue to argue
> > it, particularly in light of your "most programmers today are crap"
> > position.
>
> I haven't read the entire thread, but it appears to me to be a central
> part of his argument that C++ is faster than Java.  Unless I missed
> something obvious, that appears to be supported by the benchmarks at
> the language shootout web site if you look at them in aggregate.

But what's the argument? Unfortunately in this case it's not a
symmetrical one (ie. party A says X is faster than Y, party B says Y
is faster than X).

My position in this discussion is the negative one: The point was made
the C++ is always faster, and that'd we'd be hard pressed to find an
example where it wasn't. I think we've refuted that fairly
comprehensively at this point. As for selecting which configuration to
benchmark? If the point is to measure performance, select the fastest
one. Anything else is just doctoring the figures.

Matt