...

The system keeps dependency information to keep track of the fact that the machine code for bar depends on b being a data slot. During the _Define:, bar's code is marked as "outdated", and it's stack frame is marked. When control returns to bar (after the change), a trap handler is invoked which recompiles bar before continuing execution.

It seems to me that depending on what the compiler has inlined, and how it moved code around and optimized some of it away, that it isn't easy to continue execution after recompiling a method. I haven't had time to think this through, but the new compilation of the method might organize its stack/registers differently. Just remapping the instruction pointer

...

From the old method to the new seems quite complicated.

This is true; a local variable which was in a register in the old version of the compiled method might now reside in another register or on the stack; stack frames may grow or shrink, and a single stack frame might be split into several stack frames (because less inlining is done in the newer version of the code). The system uses the same debugging information that is used to display the stack to find out about the old and new locations of things. We'll try to write a paper soon about these questions; our current solution is actually quite simple.

the compiler might play. The OOPSLA'89 paper mentions scope description and bytecode mappings as the solution. How much overhead do you get for keeping all that information. Would it be too costly to throw it away and, when you need it, to recompile the old version of the method to regenerate all this?

The space overhead does indeed exist, and we've been thinking about reducing it. The current implementation doesn't try to save space - a relatively simple change to a denser encoding would probably halve the space requirements, but we just haven't gotten around to doing it. Generating the information "on demand" a la Smalltalk is possible. However, things get tricky after programming changes - how do you recompile (and re-inline) a method if its receiver looks entirely different now? So, in the general case, some information has to be generated always. Note that you need the debugging info precisely after programming changes to recompile methods on the stack as mentioned above.

-Urs