[self-interest] desnarfing tinySelf

[Jecel Assumpcao Jr]

The number one complaint by far that people have about Self is that
it doesn't run on their machines. So it isn't surprising that the
question I get asked most often is how do I intend to address the
portability problem.

Paul Wilson calls "snarfing" (in his excellent pages about implementing
scheme - check out http://www.cs.utexas.edu/users/oops/ ) the act of
implementing a feature in a language by simply stealing it from the
underlying system. You could, for example, implement recursion in a
Logo written in C by creating a stack array and having an explicit
stack pointer that must be carefully kept updated. Or you could
simply *snarf* the recursion already available in C.

Snarfing makes it much easier to get started by reducing the number
of things you have to worry about to obtain a working system. But it
creates a tight dependence between the implementing and implemented
systems, greatly reducing portability. And in the case of bootstrapping
(such as tinySelf on Self 4.0 or a Lisp written in Lisp) it becomes
an obstacle to be removed before the system can run "stand alone".

TinySelf 1 was written in Self 4.0 back in 1997 as a quick and dirty
test to see if parallel execution of Self programs was possible. I
haven't finished debugging it yet and it was never intended to be
the base for something better - tinySelf 2 is to be written from
scratch instead. Even so, looking at what tinySelf 1 snarfs from the
underlying Self 4 helps give us a sense of what must be done next:

  - the whole memory system. There is no separation between tinySelf 1
    objects and Self objects. So tinySelf doesn't have to worry about
    allocation, garbage collection, it has no notion of maps and uses
    mirrors to access slots and things. The need to create mirrors for
    every object it uses is the main performance problem in this
    implementation.

  - all the primitives except '_Restart'. TinySelf would just do a
    "perform: primitiveSelector" in Self to get things done except
    the Self VM doesn't like that. So each primitive is wrapped in
    a trivial method and tinySelf asks Self to "perform" that instead.

  - the parser. When you give tinySelf an expression to evaluate, it
    uses Self to parse that into an object, which is then available
    at the tinySelf level due to the shared memory system.

So if tinySelf 2 is to run on systems other than Self 4, the first
thing it must do is implement the above things itself and not snarf
them.

A second portability problem is that there are too many primitives
making Self 4's interface to the system on which it runs "too wide".
The UI is the best example - while Self exposes all of the X Window
API in the form of dozens of primitives, Squeak only requires a
primitive to copy a bitmap from an internal form object to the
frame buffer in order to work. Porting Squeak involves only writing
a simple piece of code, while making Self run on Win32 or the Mac
would involve adding a layer to essentially emulate X Windows on
them (though such layers are available, so it might not be that
much work). So one thing that would improve portability would be
to "narrow down" the interface to the implementation platform by
reducing the number of primitives. Unlike Squeak, Self doesn't not
have a performance problem of things implemented using the language
instead of primitives.

After considering several options, I decided on an alternative for
tinySelf 2 which isn't minimal but which should make it easier for
other people to port it to various platforms. The main objects
involved are:

  - compiler. This object takes bytecodes and using the current
    cpuDescription object is generates machine language code. More
    than one variation can exist, such as the current NIC and SIC.

  - cpuDescription. This encapsulates all the characteristics of a
    given cpu (X86, PowerPC, Sparc, ARM, Trimedia, etc.) so the other
    objects can be more portable. The level of detail here can be
    as high as needed - the X86 object can later be complemented with
    486, Pentium, K6, MII, PentiumPro and so on objects. This will
    boost performance as the compiler will be able to tune the
    instruction scheduling to the actual hardware.

  - platformDescription. This encapsulates the "drivers" part of
    the system. Some platforms are Win32, POSIX and BarePC. This
    is the only place in the system where non Self code, assembly
    language, in this case, is allowed. These tend to be abstract
    objects - when combined with a cpuDescription object they form
    a concrete combo called a host object.

  - emulator. Given a host object, it can "run" code written for that
    host.

  - umbilical. This object connects two executions of Self: the
    motherSelf and the babySelf. They might run on separate machines
    connected by a LAN or serial port, as two processes in a native
    OS connected by that OS's IPC or the babySelf might run in the
    emulator running inside the motherSelf. Whatever the case, this
    object allows motherSelf to have complete control of babySelf
    and includes a protocol to allow objects to flow back and forth
    between them.

  - reflectiveVM. This includes memory allocation, garbage collection,
    multithreading, message passing, etc. all written in Self.

So the first step is to get all this (except for the reflectiveVM)
running in Self 4.0. A host object containing the X86 cpuDescription
and the POSIX platformDescription must be created and the emulator
is started up. The compiler, running in Self 4, gets to work on
the umbilical object and the resulting X86 code is used to "boot"
the emulator. Then the compiler generates code for the reflectiveVM
and the compiler itself and sends it over the umbilical to the
tinySelf 2 running inside the emulator. At some point this tinySelf
is complete enough that saving a snapshot should result in a file
that can actually run by itself on an Intel Linux machine!

Note that the emulator isn't strictly necessary. The previous steps
could have been done by connecting the umbilical to an actual Intel
Linux machine instead. But doing it via an emulator is much more
convenient since you can single step, examine memory, etc. Having
two paths to babySelf (the umbilical object and the emulator's UI)
makes it much easier to see what is going on. Of course, if the
emulator isn't accurate then things will crash when trying it out
on the actual hardware. But these bugs will be eliminated in time
and it will become easier and easier to do new ports.

It is interesting to compare this with the Squeak approach. They need
an external system (the C compiler) to avoid having to deal with
different CPUs. The problem is that this external system is only
available when generating an new VM, not at runtime like the compiler
described here. They moved most platform stuff into "black boxes" -
some hand written C files. In contrast, the system I proposed is
totally self contained (like many Forths) and takes full advantage
of the power of Self (like multiple inheritance, polymorphism...).

I still have a few things to do before I start working on this, but
it is my hope that someday you will have tinySelf coming soon to a
computer near you.
-- Jecel



------------------------------------------------------------------------
eGroup home: http://www.eGroups.com/list/self-interest
Free Web-based e-mail groups by eGroups.com