[self-interest] Performance figures interesting

[Michael Latta]

It is definitely not a good idea to take such a micro benchmark too
seriously.  This was just what I had available that could be run on this
many different Smalltalk VMs.

I would think that the two machines are basically well matched in
performance, given similar architectures and clock rates for CPU and memory.
If anything the Mac should be faster given the 1Ghz FSB per CPU.

What did get my attention is that the Self version we have (10 years old) is
not keeping up with even the more simple implementations available from
Squeak and VisualWorks.  I expect that the Klein VM will do better, but even
that will only have research level support, so there will be lots of things
that never get done.

I am struggling with the issues of better technology vs. industry momentum
on many levels these days, and this is just one example where the lack of
resources seems to be limiting the success of the technically better
solution.  I like the concept of Self better than Smalltalk (directly
manipulating instances rather than all behavior and structure organized into
classes), and I much prefer Smalltalk to Java or C#.  Unfortunately the
expectations on all applications are rising and the Smalltalk based
technologies are not keeping pace.  The UI frameworks in Smalltalk can not
do the things that WPF in C# can do.  The level of polish on VisualWorks is
not competitive with Eclipse for Java.  At the same time I can solve
problems faster in Smalltalk and do things that the more mainstream systems
can not come close to doing.

For Smalltalk to survive it needs better connectivity to other technologies
like Java, C#, and web services, and there needs to be more invested into
making it competitive in terms of quality and performance.  While users do
not generally care about what technology a product is built on, the IT
department does.  In the enterprise space it needs to fit in.  In the
end-user space it needs to look credible compared to other applications.
Morphic under Squeak is the most polished of the UI options for Smalltalk,
but does not use native windows, and Visual Works which uses native windows,
still has 10 year old refresh bugs that make it look very unprofessional.

For standalone computers like you are building Jecel it makes more sense
than for applications trying to exist in a Microsoft/IBM dominated world.

Smalltalk is what got me into computers, and I have not found anything I
like better, I just hope I can find ways to keep using it professionally.

Michael



-----Original Message-----
From: self-interest at yahoogroups.com [mailto:self-interest at yahoogroups.com]
On Behalf Of Jecel Assumpcao Jr
Sent: Tuesday, May 09, 2006 4:54 PM
To: self-interest at yahoogroups.com
Subject: Re: [self-interest] Performance figures interesting

Michael Latta wrote on Tue, 9 May 2006 14:08:06 -0700

> Using the tinyBenchmark microbenchmark from Squeak I find
> the following figures interesting.  I wonder how the Klein VM
> will fare?  In all cases the benchmark was run several times
> to give the VM a chance to cache the compiled version if it so chose. 
> 
> 2.0Ghz Pentium/M w/2GB 533Mhz memory 

That means you are only using the non optimizing compiler. This makes a
huge difference.
 
> Squeak 
> 
>  '327,784,891 bytecodes/sec;  9,379,747 sends/sec' 

I reformated your numbers to make them easier to read.

> VisualWorks 
> 
>  '859,781,696 bytecodes/sec; 71,069,397 sends/sec' 

This is very impressive. Note that you would probably get similar
numbers using the Exupery compiler for Squeak.

> Gemstone/S 
> 
>  ' 30,563,514 bytecodes/sec;  4,467,145 sends/sec' 
> 
> PowerMac G5 2.0Ghz w/4GB 400Mhz memory 

Ah, so you are using the full compilation technology! Forget my comment
above. But it doesn't make sense to compare different language
implementations on entirely different machines.
 
> Self 
> 
>  ' 25,196,850 bytecodes/sec;   879,023 sends/sec' 

Here are some numbers I had posted at
http://minnow.cc.gatech.edu/squeak/768

600MHz Pentium III (bogomips 1193)
  Squeak 3.2
     1,637,886 sends/sec
      54,872,936 bytecodes/sec
  Self 4.2.1
      437,065 sends/sec
      5,003,187 bytecodes/sec

277MHz Ultra 5 (UltraSparc IIi)
  Squeak 3.6
      671,079 sends/sec
      18,065,656 bytecodes/sec
  Self 4.2.1
      15,517,463 sends/sec
      93,590,129 bytecodes/sec

So while on one machine (without the optimizing compiler) Self was
significantly slower than Squeak, on another machine it was just as
significantly faster. The total difference between the two machines was
a factor of 57 for bytecodes/sec and 87 for sends/sec.

But you already knew that it isn't a good idea to take benchmarks too
seriously, right?
-- Jecel


 
Yahoo! Groups Links