[Self-interest] Proposal for a modernized Self dialect

Eric Nijakowski eric.nijakowski at web.de
Mon Dec 13 23:07:52 UTC 2021


Hi Randy,

very interesting indeed! What you're saying makes sense, I too have 
experienced these confused stares a lot :D

However, I wouldn't trace this back to the complexity of the idea - some 
ideas are just way ahead of their time, but not complex.

Conceptually, there isn't a big difference between a Self world that 
runs on one computer and a Self world that runs on a network of 
computers. The only thing where people get confused is if you explain 
the technical details instead of showing them what you can do with it. 
At least, that's my experience.

Just like we don't care about the sectors that a file on our hard disk 
rests on, most users shouldn't care about the tricks the VM pulls on 
them to provide the illusion of being able to create a direct pointer to 
an object on a different computer. Actually, that's also the principle 
the Self VM has used a lot: The existence of "maps" isn't very important 
to the user since the fact that objects are not simply implemented as 
key-value-lists is not leaked to the Self programmer at any time. In 
this sense, the existence of byte-based networking protocols and hard 
disks should also be completely abstracted away.

Now, I'd like to move on to the problems you mentioned in your response. 
From my current standpoint I think the reason why your early 
undertakings in this area failed is that you were limited by the 
hardware and software capabilities of the time. These days, there is an 
increased use of technologies like gRPC and Protocol Buffers, and I'd 
definitely say that it's time to give the idea of a "networking Self" a 
new shot (which is essentially what I'm trying to do with my dialect of 
Self, Eco).

The way I understand it, the fact that Smalltalk treats "ifTrue:" 
differently is mostly a compiler issue. What I mean by that is that 
these messages are usually compiled directly into JUMP bytecodes - which 
is a problem when it comes to networking, especially since both the 
bytecode dispatcher and the compiler need a lot of information about the 
objects in order to decide e.g. whether to branch to a different 
instruction or not.

Some problems that I'd assume have appeared in your prototypes are:

  - Waiting for a computer to respond should not halt the entire VM. 
When accessing a slot or a method of an object on another system, 
waiting for an incoming message has to be done in a way that allows us 
to handle errors while at the same time providing a fast and secure 
environment.
  - Garbage collection becomes incredibly difficult if the object graph 
spans multiple computers on the network. Somehow, groups of objects need 
to be found that can be freed, which means that the garbage collectors 
have to work together. One hanging implementation might bring down the 
entire network, and if the communication doesn't happen fast enough the 
entire world will just stop.
  - Passing a block closure or stack snapshots from one computer on the 
network to another is also a very difficult task, especially if the 
systems are fundamentally different regarding word size and memory layout.
  - The Smalltalk VM was not designed to support collaborative 
networking, so many fundamental concepts might have to be changed (e.g. 
how to share metaclasses) or hacked away.

There are many more issues that I can think of, and I'm sure that you're 
familiar with them, too. The way I'm approaching the problem is the 
following:

  1) Networking capabilities should be built directly into the "slow" 
message dispatching function (the function that is called by the SEND 
bytecode). Optimized code examines the object's map pointer in order to 
determine if the object type is unexpected and the system should 
deoptimize. Every local representative of a "network object" therefore 
has its own map pointer, so that if a message is sent to a networked 
object from within the optimized code, the system will quickly switch to 
the deoptimized version, call the "slow send function", and tell the 
scheduler to pause the current thread until the result of the message 
send is received over the network. During that idle time, the local 
garbage collector can do its work.
  2) Garbage collection for local objects is implemented in C, garbage 
collection for remote objects is implemented in Eco itself (which, due 
to the design of the VM, is unaffected by the complexities arising from 
the difference between a local pointer and a pointer to a remote 
object). The VM supports running Eco code in a "hidden administrator 
thread mode" that enables it to examine these local representatives of 
remote objects and find unreferenced subgraphs.
  3) Immutable objects can be copied over and become a local object. 
This is quite useful for bytecode objects. Also, data structures that 
consist of multiple related objects (densely connected subgraphs) should 
not be split between two VMs. I have thought about a few ways to share 
closures and continuations too, but I don't want to get into too much 
detail on that right now.

I'd really like to hear your take on this approach, especially in the 
context of your experience, and if you think that it could work out. 
Also, I'm very interested in your "Us" language. Do you still have any 
documents from that time? I'd love to take a look at them :)

Thanks a lot!
Eric

> Eric, Thanks for this well thought-out and well-written (with references!) note!
>
> Just a comment on one of your points.
>
>> I'd therefore suggest the implementation of a mechanism that allowed
>> Self objects to be stored, referenced and accessed from everywhere
>> (without destroying the illusion of all objects being "live" and in the same world)
> When I was in the Smalltalk group at PARC, (1980’s) Adele Goldberg encouraged us to think about collaborative computing, and so I wondered what one might do at the language level to support it. (I think this was just before Dave came along and we started up Self ideas.) There were a number of brilliant people around, so it was fun to talk with them. (e.g., Mark Miller, Peter Deutsch, and Danny Bobrow among those I recall especially.)  My approach of course was to try to get a simple and uniform underlying model for objects all over the world, sharing references as though in one giant, globe-spanning VM. I did some hacks with “transparent forwarders” and object migration in the PARC Smalltalk-80 world, but ran aground of numerous issues some virtually impossible to really solve in Smalltalk-80 (for example "ifTrue:" isn’t even a message) and others that are easy to enumerate once one starts thinking about it. Nothing came of that line of work, we are talking like 1987 or something and there were performance problems that made the whole enterprise questionable for the foreseeable future.
>
> While at Sun I worked out a paper-design for a language called “Us,”  (so named as to emphasize the inherited ideas from Self now applied to a world of multiple viewpoints). The fundamental approach in Us was to reify as objects and therefore embrace the issues of multiple perspectives that collaborative systems immediately raise. However, once I had worked out a story, I found that in trying to describe what I had hoped to be a simple and uniform vision I would be greeted with patient but usually silent confused stares. This experience stood in stark contrast to describing Self which would often bring smiles and nods.  I started to feel that the complexity was simply too much and grew discouraged.
>
> Dave and I briefly discussed taking Self in various directions like this once Sun got committed to Java and grew cool on Self. It would be a major paradigm shift and I’m sure much thinking along these lines has been done by now many times over, and I confess I haven’t tracked the thinking. I’d be interested in thoughts of others and/or in a summery of what has been done or at least thought of along these lines.
>
> Thanks for listening!
>
> —Randy
>   
>


More information about the Self-interest mailing list