[Self-interest] Proposal for a modernized Self dialect
Eric Nijakowski
eric.nijakowski at web.de
Mon Dec 13 23:07:52 UTC 2021
Hi Randy,
very interesting indeed! What you're saying makes sense, I too have
experienced these confused stares a lot :D
However, I wouldn't trace this back to the complexity of the idea - some
ideas are just way ahead of their time, but not complex.
Conceptually, there isn't a big difference between a Self world that
runs on one computer and a Self world that runs on a network of
computers. The only thing where people get confused is if you explain
the technical details instead of showing them what you can do with it.
At least, that's my experience.
Just like we don't care about the sectors that a file on our hard disk
rests on, most users shouldn't care about the tricks the VM pulls on
them to provide the illusion of being able to create a direct pointer to
an object on a different computer. Actually, that's also the principle
the Self VM has used a lot: The existence of "maps" isn't very important
to the user since the fact that objects are not simply implemented as
key-value-lists is not leaked to the Self programmer at any time. In
this sense, the existence of byte-based networking protocols and hard
disks should also be completely abstracted away.
Now, I'd like to move on to the problems you mentioned in your response.
From my current standpoint I think the reason why your early
undertakings in this area failed is that you were limited by the
hardware and software capabilities of the time. These days, there is an
increased use of technologies like gRPC and Protocol Buffers, and I'd
definitely say that it's time to give the idea of a "networking Self" a
new shot (which is essentially what I'm trying to do with my dialect of
Self, Eco).
The way I understand it, the fact that Smalltalk treats "ifTrue:"
differently is mostly a compiler issue. What I mean by that is that
these messages are usually compiled directly into JUMP bytecodes - which
is a problem when it comes to networking, especially since both the
bytecode dispatcher and the compiler need a lot of information about the
objects in order to decide e.g. whether to branch to a different
instruction or not.
Some problems that I'd assume have appeared in your prototypes are:
- Waiting for a computer to respond should not halt the entire VM.
When accessing a slot or a method of an object on another system,
waiting for an incoming message has to be done in a way that allows us
to handle errors while at the same time providing a fast and secure
environment.
- Garbage collection becomes incredibly difficult if the object graph
spans multiple computers on the network. Somehow, groups of objects need
to be found that can be freed, which means that the garbage collectors
have to work together. One hanging implementation might bring down the
entire network, and if the communication doesn't happen fast enough the
entire world will just stop.
- Passing a block closure or stack snapshots from one computer on the
network to another is also a very difficult task, especially if the
systems are fundamentally different regarding word size and memory layout.
- The Smalltalk VM was not designed to support collaborative
networking, so many fundamental concepts might have to be changed (e.g.
how to share metaclasses) or hacked away.
There are many more issues that I can think of, and I'm sure that you're
familiar with them, too. The way I'm approaching the problem is the
following:
1) Networking capabilities should be built directly into the "slow"
message dispatching function (the function that is called by the SEND
bytecode). Optimized code examines the object's map pointer in order to
determine if the object type is unexpected and the system should
deoptimize. Every local representative of a "network object" therefore
has its own map pointer, so that if a message is sent to a networked
object from within the optimized code, the system will quickly switch to
the deoptimized version, call the "slow send function", and tell the
scheduler to pause the current thread until the result of the message
send is received over the network. During that idle time, the local
garbage collector can do its work.
2) Garbage collection for local objects is implemented in C, garbage
collection for remote objects is implemented in Eco itself (which, due
to the design of the VM, is unaffected by the complexities arising from
the difference between a local pointer and a pointer to a remote
object). The VM supports running Eco code in a "hidden administrator
thread mode" that enables it to examine these local representatives of
remote objects and find unreferenced subgraphs.
3) Immutable objects can be copied over and become a local object.
This is quite useful for bytecode objects. Also, data structures that
consist of multiple related objects (densely connected subgraphs) should
not be split between two VMs. I have thought about a few ways to share
closures and continuations too, but I don't want to get into too much
detail on that right now.
I'd really like to hear your take on this approach, especially in the
context of your experience, and if you think that it could work out.
Also, I'm very interested in your "Us" language. Do you still have any
documents from that time? I'd love to take a look at them :)
Thanks a lot!
Eric
> Eric, Thanks for this well thought-out and well-written (with references!) note!
>
> Just a comment on one of your points.
>
>> I'd therefore suggest the implementation of a mechanism that allowed
>> Self objects to be stored, referenced and accessed from everywhere
>> (without destroying the illusion of all objects being "live" and in the same world)
> When I was in the Smalltalk group at PARC, (1980’s) Adele Goldberg encouraged us to think about collaborative computing, and so I wondered what one might do at the language level to support it. (I think this was just before Dave came along and we started up Self ideas.) There were a number of brilliant people around, so it was fun to talk with them. (e.g., Mark Miller, Peter Deutsch, and Danny Bobrow among those I recall especially.) My approach of course was to try to get a simple and uniform underlying model for objects all over the world, sharing references as though in one giant, globe-spanning VM. I did some hacks with “transparent forwarders” and object migration in the PARC Smalltalk-80 world, but ran aground of numerous issues some virtually impossible to really solve in Smalltalk-80 (for example "ifTrue:" isn’t even a message) and others that are easy to enumerate once one starts thinking about it. Nothing came of that line of work, we are talking like 1987 or something and there were performance problems that made the whole enterprise questionable for the foreseeable future.
>
> While at Sun I worked out a paper-design for a language called “Us,” (so named as to emphasize the inherited ideas from Self now applied to a world of multiple viewpoints). The fundamental approach in Us was to reify as objects and therefore embrace the issues of multiple perspectives that collaborative systems immediately raise. However, once I had worked out a story, I found that in trying to describe what I had hoped to be a simple and uniform vision I would be greeted with patient but usually silent confused stares. This experience stood in stark contrast to describing Self which would often bring smiles and nods. I started to feel that the complexity was simply too much and grew discouraged.
>
> Dave and I briefly discussed taking Self in various directions like this once Sun got committed to Java and grew cool on Self. It would be a major paradigm shift and I’m sure much thinking along these lines has been done by now many times over, and I confess I haven’t tracked the thinking. I’d be interested in thoughts of others and/or in a summery of what has been done or at least thought of along these lines.
>
> Thanks for listening!
>
> —Randy
>
>
More information about the Self-interest
mailing list