Selfers,
The past few months I have been playing around with the latest VM code, and building some extensions to aid in building networked objects that take advantage of modern OS infrastructure. Back when Self was under active development by Sun, none of the current kernel level APIs designed to address the 10k problem for Mac (BSD), Linux, Solaris even existed. And if you've ever programed highly distributed systems, you'll know that kqueue, epoll, and the like are your friends.
This past week I got to thinking about how I would do message sends in Self to proxy objects with a URI naming convention similar to dself, but with a view towards making to accessible to non-self APIs as well. It struck me that HTTP alone could be used to send messages using a simple scheme:
SELF uri-encoded-string HTTP/1.1\r\n\r\n
Where the handler for this invocation merely decodes the string and then performs it using a delegate. The delegate's enclosing scope would contain a reference to the sender of the message so that the object could reply directly by sending a message back:
HTTP/1.1 200 OK\r\n\r\n SELF uri-encoded-method-call HTTP/1.1\r\n\r\n
Since we are going to use keepalive by default, we can decouple the ACK code from the message send, and control our behavior and security on both sides.
If we want to restrict method sends we can use a auth scheme that includes a magic cookie in the headers and use the same P2P request scheme to make use of the socket.
SELF uri-encoded-method-invocation HTTP/1.1 X-Self-Magic-Cookie: sha1-hashed-shared-secret
And if the shared secret does not match a config that matches the client IP/MAC address we could:
HTTP 403 ACCESS DENIED
From the point of view of a AJAX component hitting this API from a browser this would provide a means to run code, but not necessarily process anything of a meaningful response. To do that would require the Self object understand how to invoke code on the JavaScript side. The easiest way to do this is to formalize an object interface that processes the body of the OK response as:
this method args
Where the invocation code is
this[method].apply(this,args)
The entire XHR request could easily be encapsulated in Object.prototype.self = function(){} and be made available complete with self method names including the : such as
on:Print: aHTMLElement aString
The internalized string is a valid method name, just not accessible via dot notation in JS
This approach matched with some VM improvements make self a great language for building dynamic web apps that can also run in native self apps as well.
Dave
Hi Dave,
My question is whether you want to make the language Self worlds use to speak to themselves the same language as web browsers use to speak to servers. It sounds attractive but will it be limiting?
One idea I have had is if we added an interface to the ZeroMQ library we could build a messaging system with multiple transports and both call-response and multicasting of messages, and then for web apps just put Mongrel2 or similar on the front of it. It would be really nice if a single messaging mechanism could handle all IO from a running Self world but that might be too much to ask for. :)
Russell
On 11/04/2011, at 12:45 AM, David Goehrig dave@nexttolast.com wrote:
Selfers,
The past few months I have been playing around with the latest VM code, and building some extensions to aid in building networked objects that take advantage of modern OS infrastructure. Back when Self was under active development by Sun, none of the current kernel level APIs designed to address the 10k problem for Mac (BSD), Linux, Solaris even existed. And if you've ever programed highly distributed systems, you'll know that kqueue, epoll, and the like are your friends.
This past week I got to thinking about how I would do message sends in Self to proxy objects with a URI naming convention similar to dself, but with a view towards making to accessible to non-self APIs as well. It struck me that HTTP alone could be used to send messages using a simple scheme:
SELF uri-encoded-string HTTP/1.1\r\n\r\n
Where the handler for this invocation merely decodes the string and then performs it using a delegate. The delegate's enclosing scope would contain a reference to the sender of the message so that the object could reply directly by sending a message back:
HTTP/1.1 200 OK\r\n\r\n SELF uri-encoded-method-call HTTP/1.1\r\n\r\n
Since we are going to use keepalive by default, we can decouple the ACK code from the message send, and control our behavior and security on both sides.
If we want to restrict method sends we can use a auth scheme that includes a magic cookie in the headers and use the same P2P request scheme to make use of the socket.
SELF uri-encoded-method-invocation HTTP/1.1 X-Self-Magic-Cookie: sha1-hashed-shared-secret
And if the shared secret does not match a config that matches the client IP/MAC address we could:
HTTP 403 ACCESS DENIED
From the point of view of a AJAX component hitting this API from a browser this would provide a means to run code, but not necessarily process anything of a meaningful response. To do that would require the Self object understand how to invoke code on the JavaScript side. The easiest way to do this is to formalize an object interface that processes the body of the OK response as:
this method args
Where the invocation code is
this[method].apply(this,args)
The entire XHR request could easily be encapsulated in Object.prototype.self = function(){} and be made available complete with self method names including the : such as
on:Print: aHTMLElement aString
The internalized string is a valid method name, just not accessible via dot notation in JS
This approach matched with some VM improvements make self a great language for building dynamic web apps that can also run in native self apps as well.
Dave
Hi Russel,
My question is whether you want to make the language Self worlds use to
speak to themselves the same language as web browsers use to speak to servers. It sounds attractive but will it be limiting?
Well what I'm thinking is the language of communication is just Self. I've done Forth systems that used Forth as their communication protocol and it worked beautifully. I see no reason why sending a message shouldn't just be modeled directly as a message send :)
One idea I have had is if we added an interface to the ZeroMQ library we
could build a messaging system with multiple transports and both call-response and multicasting of messages, and then for web apps just put Mongrel2 or similar on the front of it.
I've done a lot of apps with XMPP, AMQP, and custom communication protocols over the years, and the lesson I've learned is that it isn't the transport mechanism that matters. What really matters is what are you shuffling across the wire? If you're just sending state, you're setting yourself up for failure.. You really have to be sending behavior back and forth to make the system scale. ESB get this to a degree. Adding ZeroMQ & Mongrel2, or Ejabberd & Mochweb, or X & Y, is far less important than designing a programming model where pushing a button no a web form literally sends a message to an object on a remote server.
It would be really nice if a single messaging mechanism could handle all IO from a running Self world but that might be too much to ask for. :)
I've actually been building an object to map Self sent over the wire via sockets to Erlang style processes. It listens on any number of sockets, files, or process ids, and sends a message to the registered delegates. Globals can be referenced in the messages by name, and sandboxing is done by just using objects which never refer to lobby but use a sandbox* reference to a minimal Self world.
My goal with the JS side of things for this is to just have the Self code necessary to render a UI just directly run in JS using the translation scheme I mentioned. HTML5 canvas + Self + this proxy object => self in the browser. I'm also thinking about doing a Webkit plugin and just bind the Self VM as a plugin, using this Self <-> JS bridge code to once again embed Self in the browser.
Dave
Hi Dave,
Well what I'm thinking is the language of communication is just Self. I've done Forth systems that used Forth as their communication protocol and it worked beautifully. I see no reason why sending a message shouldn't just be modeled directly as a message send :)
Sounds good.
I've done a lot of apps with XMPP, AMQP, and custom communication protocols over the years, and the lesson I've learned is that it isn't the transport mechanism that matters. What really matters is what are you shuffling across the wire? If you're just sending state, you're setting yourself up for failure.. You really have to be sending behavior back and forth to make the system scale. ESB get this to a degree. Adding ZeroMQ & Mongrel2, or Ejabberd & Mochweb, or X & Y, is far less important than designing a programming model where pushing a button no a web form literally sends a message to an object on a remote server.
I think I agree, though I see ZeroMQ as a replacement for sockets/etc rather than a full featured system like AMQP. I think in ZeroMQ the actual messages are blobs - ZeroMQ only does routing. Which means that you could run a pure Self messaging system over it and take advantage of the handling of the messy underlying connection details.
I've actually been building an object to map Self sent over the wire via sockets to Erlang style processes. It listens on any number of sockets, files, or process ids, and sends a message to the registered delegates. Globals can be referenced in the messages by name, and sandboxing is done by just using objects which never refer to lobby but use a sandbox* reference to a minimal Self world.
I'm very interested in this. Are you intending to share it?
I have played around with similar ideas and sandboxing turned out to be harder than I thought. In particular to do a proper job, my conclusion was there needed to be VM support for restricting access to primitives and a restructuring of how numbers and strings (amongst other) objects were structured.
ie at the moment you can do things like:
42 lobby blowUpWorld
'Hello' _BlowUpWorld
These are certainly fixable.
What are you using for a serialisation/deserialisation mechanism? It would be nice but not necessary to use the normal way of defining Self objects. Unfortunately the Self parser is in the VM and isn't securable, ie
(| key = blowUpWorld |)
There is code to build an object tree from Self code in the system but I don't think it does the next step of generating actual Self methods.
My goal with the JS side of things for this is to just have the Self code necessary to render a UI just directly run in JS using the translation scheme I mentioned. HTML5 canvas + Self + this proxy object => self in the browser. I'm also thinking about doing a Webkit plugin and just bind the Self VM as a plugin, using this Self <-> JS bridge code to once again embed Self in the browser.
Or the Google nativeclient, though the vm does funky things so mightn't play well in a sandbox.
If you have a HTML5/javascript implementation of the Self morphic 'canvas' which talks over the wire to a Self world then it might not be too slow even if the world is in the cloud. You could also take something like QT and its inbuilt javascript interpreter and do the same thing to make 'native' apps.
Cheers,
Russell
On Mon, Apr 11, 2011 at 7:32 PM, Russell Allen mail@russell-allen.comwrote:
I think I agree, though I see ZeroMQ as a replacement for sockets/etc rather than a full featured system like AMQP. I think in ZeroMQ the actual messages are blobs – ZeroMQ only does routing. Which means that you could run a pure Self messaging system over it and take advantage of the handling of the messy underlying connection details.
ZeroMQ send length coded strings, like most Erlang sockets do. ZeroMQ's pub-sub-hub model works well only if you are worried about routing payloads to well known endpoints, and is just another transport layer. It tends to fall flat when you want an actual broadcast mechanism, and want to receive from unknown endpoints (like with email).
I’m very interested in this. Are you intending to share it?
I was planning on releasing it on github with a BSD style license.
my conclusion was there needed to be VM support for restricting access to primitives and a restructuring of how numbers and strings (amongst other) objects were structured.
I'm certain that is true. The primitives are too dangerous to expose to other people's code. One of the things that I'd like to do is send full objects to remote nodes for processing.
What are you using for a serialisation/deserialisation mechanism? It would
be nice but not necessary to use the normal way of defining Self objects.
I was planning on using the Transporter, and an extra bit or two.
If you have a HTML5/javascript implementation of the Self morphic ‘canvas’ which talks over the wire to a Self world then it might not be too slow even if the world is in the cloud. You could also take something like QT and its inbuilt javascript interpreter and do the same thing to make ‘native’ apps.
Bingo!
I was also thinking WebGL + Self might be a very nice to build a virtual world :)
Dave
self-interest@lists.selflanguage.org
-
David Goehrig -
Russell Allen